Evals / pdf

Convert a PDF

3 scenarios, 3 run by an agent. Each run shows its result, time and new tokens.

ScenarioCodexwithout deftCodexwith deft, skill installedCodexwith deft, skill as agents.mdClaude Opus 5.5without deftClaude Opus 5.5with deft, skill installedClaude Codewithout deftClaude Codewith deft
Every page to JPEGTurn every page of f1040.pdf into a JPEG in a folder called pages.pass15s · 6.3kpass16s · 5.2kpass13s · 10.4kpass12s · 8.8kpass13s · 9.8knot runnot run
Render a cover imageSave page 1 of paper.pdf as a 150 dpi image called cover.png.pass16s · 8.9kpass12s · 4.9kpass12s · 6.0kpass9s · 8.6kpass15s · 11.2knot runnot run
Pull tables out of a PDFtables.pdf has two tables among its text. Put them in a workbook called table.xlsx, one table per sheet.pass41s · 19.7kpass18s · 8.4knot runpass17s · 11.5kpass28s · 12.8knot runnot run