Evals / video

Subtitles

2 scenarios, 2 run by an agent. Each run shows its result, time and new tokens.

ScenarioCodexwithout deftCodexwith deft, skill installedCodexwith deft, skill as agents.mdClaude Opus 5.5without deftClaude Opus 5.5with deft, skill installedClaude Codewithout deftClaude Codewith deft
Burn subtitles into a videoBurn the subtitles from talk.srt into bunny.mp4 so they are always visible. Save it as subbed.mp4.pass12s · 6.5kpass15s · 12.0kpass15s · 7.3kpass27s · 12.4kpass12s · 10.7knot runnot run
Add a subtitle track viewers can switch offAdd talk.srt to bunny.mp4 as a subtitle track that viewers can turn on and off. Do not re-encode the video. Save it as soft.mp4.pass10s · 4.8kpass17s · 8.4kpass16s · 11.5kpass13s · 9.5kpass17s · 10.8knot runnot run