Evals / image

Metadata and privacy

4 scenarios, 4 run by an agent. Each run shows its result, time and new tokens.

ScenarioCodexwithout deftCodexwith deft, skill installedCodexwith deft, skill as agents.mdClaude Opus 5.5without deftClaude Opus 5.5with deft, skill installedClaude Codewithout deftClaude Codewith deft
Answer a question about a photoWhat are the displayed width and height of landscape_6.jpg, after its orientation is applied? Write them as WIDTHxHEIGHT to size.txt.pass11s · 3.4kpass12s · 4.7kpass12s · 9.9kpass13s · 10.2kpass17s · 12.0knot runnot run
Where was this takenWhere was DSCN0010.jpg taken? Write the latitude and longitude as decimal degrees to place.txt, one per line.pass16s · 4.5kpass18s · 6.1knot runpass14s · 9.4kpass17s · 12.1knot runnot run
Remove GPS before sharingI want to share DSCN0010.jpg but it must not reveal where it was taken. Save a safe copy as clean.jpg.pass19s · 7.2kpass20s · 10.0kpass19s · 13.9kpass19s · 10.5kpass18s · 11.7knot runnot run
Blur part of a photoBlur the top-left 64 by 48 pixel corner of hopper.png and save it as hidden.png.pass12s · 3.6kpass17s · 7.4knot runpass11s · 8.8kpass14s · 10.3knot runnot run