Evals / video

Multi-step jobs

3 scenarios, 3 run by an agent. Each run shows its result, time and new tokens.

ScenarioCodexwithout deftCodexwith deft, skill installedClaude Opus 5.5without deftClaude Opus 5.5with deft, skill installedClaude Codewithout deftClaude Codewith deft
Make a captioned vertical clipTurn the first 6 seconds of bunny.mp4 into a vertical clip for Reels: 1080 by 1920 with the whole picture showing, the subtitles from talk.srt burned in, and under 2MB. Save it as reel.mp4.pass27s · 8.8kpass34s · 16.8kpass21s · 11.6kpass35s · 13.6knot runnot run
Prepare a landing page videomovie.mp4 will autoplay as the hero video on our landing page. It has to be silent, under 1MB and no taller than 720 pixels, saved as hero.mp4. Also save its first frame as poster.jpg.pass19s · 7.0kpass35s · 14.1kpass19s · 9.8kpass27s · 12.1knot runnot run
See a video at a glanceI need to see what happens in bunny.mp4 without watching it. Make one picture, overview.jpg, with a frame from every 2 seconds laid out in a grid.pass14s · 3.7kpass20s · 6.0kpass17s · 11.4kpass18s · 13.7knot runnot run