Evals / video

See inside a video

4 scenarios, 4 run by an agent. Each run shows its result, time and new tokens.

ScenarioCodexwithout deftCodexwith deft, skill installedCodexwith deft, skill as agents.mdClaude Opus 5.5without deftClaude Opus 5.5with deft, skill installedClaude Codewithout deftClaude Codewith deft
Sample framesSave one frame every 2 seconds of bunny.mp4 as images in a folder called frames.pass12s · 3.3kpass11s · 4.3kpass13s · 5.9kpass11s · 8.5kpass9s · 9.9knot runnot run
Sample frames from a pointStarting at 4 seconds into bunny.mp4, save one frame every 2 seconds into a folder called frames.pass12s · 6.0kpass11s · 4.4knot runpass15s · 8.9kpass17s · 10.3knot runnot run
Grab one frameSave the frame at 5 seconds in bunny.mp4 as frame.jpg.pass7s · 4.1kpass17s · 5.7kpass9s · 5.2kpass11s · 10.1kpass15s · 9.8knot runnot run
Answer a question about a videoHow long is jellyfish.mp4 in seconds, and what is its resolution? Write the answer to info.txt.pass11s · 3.4kpass15s · 7.6kpass13s · 5.6kpass13s · 8.9kpass14s · 10.5knot runnot run