Evals / pdf

Forms

2 scenarios, 2 run by an agent. Each run shows its result, time and new tokens.

ScenarioCodexwithout deftCodexwith deft, skill installedCodexwith deft, skill as agents.mdClaude Opus 5.5without deftClaude Opus 5.5with deft, skill installedClaude Codewithout deftClaude Codewith deft
Fill a PDF formFill in the name field on fw9.pdf with Ada Lovelace and save it as w9-filled.pdf.pass62s · 20.6kpass18s · 6.8kfail41s · 17.2kpass71s · 26.9kpass17s · 11.6knot runnot run
Flatten a filled formFill the first text field of fw9.pdf with Ada Lovelace and make the result uneditable. Save it as w9-flat.pdf.pass130s · 57.5kpass23s · 13.2knot runpass56s · 15.7kpass13s · 11.8knot runnot run