v1EffectsPrompt only

Lava lamp

The first CmdBench task: one prompt, every published run. Its runs are never compared with newer benchmark versions. Two GPT-6.1 Terra runs, xhigh and max (26.10.2026), used a copy of the runner with xhigh added to its allowed efforts; the prompt and checks were the same.

60 published runs · 2 agents · last run 28.10.2026 14:40 Tallinn

Fastest working build
29.7 s
GPT-5.6 Luna, low effort
Fewest output tokens
3.4k
GPT-5.6 Luna, low effort
Lowest API cost
$0.02
Claude Haiku 5.5, low effort. Only Claude Code reports cost.
Failing runs
5 of 60
Each is marked in the grid and the run list.

Model × effort

Every model at every effort.

One model at one effort per cell, with its newest working build. Down a column: models at one effort. Along a row: what more effort buys. Tick builds to compare them.

Number shown

Build timeOutput tokensOutput speedAPI cost

Lower is better. Median generation time to a finished build.Lower is better. Median tokens the model wrote.Higher is better. Median output tokens per second.Lower is better. The CLI’s list-price estimate, not a charge.

Model	Low12 models	Medium12 models	High12 models	Extra high12 models	Max12 models
Claude Code
Claude Fable 5.15 runs · 5 of 5 workCompare its efforts	Works 2m 50s 1 run 13.5k 1 run 79 tok/s 1 run $1.05 1 run Compare	Works 4m 53s 1 run 23.5k 1 run 80 tok/s 1 run $1.77 1 run Compare	Works 14m 06s 1 run 63.1k 1 run 75 tok/s 1 run $4.42 1 run Compare	Works 32m 58s 1 run 145.7k 1 run 74 tok/s 1 run $9.06 1 run Compare	Works 36m 52s 1 run 172.0k 1 run 78 tok/s 1 run $11.33 1 run Compare
Claude Haiku 5.55 runs · 5 of 5 workCompare its efforts	Works 2m 36s 1 run 34.3k 1 run 220 tok/sFastest output 1 run $0.02Cheapest 1 run Compare	Works 5m 06s 1 run 69.5k 1 run 227 tok/sFastest output 1 run $0.05Cheapest 1 run Compare	Works 5m 22s 1 run 75.5k 1 run 235 tok/sFastest output 1 run $0.06Cheapest 1 run Compare	Works 8m 28s 1 run 117.3k 1 run 231 tok/sFastest output 1 run $0.20Cheapest 1 run Compare	Works 20m 00s 1 run 263.9k 1 run 220 tok/sFastest output 1 run $1.37Cheapest 1 run Compare
Claude Opus 5.55 runs · 5 of 5 workCompare its efforts	Works 1m 42s 1 run 10.6k 1 run 104 tok/s 1 run $0.33 1 run Compare	Works 8m 56s 1 run 54.5k 1 run 102 tok/s 1 run $2.01 1 run Compare	Works 13m 48s 1 run 80.9k 1 run 98 tok/s 1 run $2.40 1 run Compare	Works 28m 09s 1 run 151.6k 1 run 90 tok/s 1 run $6.40 1 run Compare	Works 66m 39s 1 run 402.8k 1 run 101 tok/s 1 run $13.82 1 run Compare
Claude Sonnet 5.55 runs · 5 of 5 workCompare its efforts	Works 2m 37s 1 run 19.4k 1 run 124 tok/s 1 run $0.29 1 run Compare	Works 4m 44s 1 run 34.8k 1 run 122 tok/s 1 run $0.60 1 run Compare	Works 6m 45s 1 run 52.7k 1 run 130 tok/s 1 run $0.71 1 run Compare	Works 21m 18s 1 run 168.4k 1 run 132 tok/s 1 run $2.92 1 run Compare	Works 87m 03s 1 run 479.7k 1 run 92 tok/s 1 run $12.68 1 run Compare
Codex CLI
GPT-5.6 Luna5 runs · 5 of 5 workCompare its efforts	Works 29.7 sFastest 1 run 3.4kFewest 1 run 114 tok/s 1 run not reported 1 run Compare	Works 59.1 s 1 run 6.7k 1 run 114 tok/s 1 run not reported 1 run Compare	Works 1m 04s 1 run 8.0k 1 run 125 tok/s 1 run not reported 1 run Compare	Works 1m 25s 1 run 10.5k 1 run 125 tok/s 1 run not reported 1 run Compare	Works 2m 22sFastest 1 run 22.4kFewest 1 run 157 tok/s 1 run not reported 1 run Compare
GPT-5.6 Sol5 runs · 2 of 5 workCompare its efforts	Works 1m 46s 1 run 5.2k 1 run 50 tok/s 1 run not reported 1 run Compare	Fails 2m 20s 1 run 7.3k 1 run 52 tok/s 1 run not reported 1 run Compare	Works 2m 34s 1 run 11.2k 1 run 73 tok/s 1 run not reported 1 run Compare	Fails 4m 05s 1 run 17.1k 1 run 70 tok/s 1 run not reported 1 run Compare	Fails 7m 53s 1 run 27.6k 1 run 58 tok/s 1 run not reported 1 run Compare
GPT-5.6 Terra5 runs · 4 of 5 workCompare its efforts	Works 1m 38s 1 run 3.6k 1 run 36 tok/s 1 run not reported 1 run Compare	Works 1m 48s 1 run 3.5kFewest 1 run 32 tok/s 1 run not reported 1 run Compare	Works 1m 00sFastest 1 run 7.5kFewest 1 run 126 tok/s 1 run not reported 1 run Compare	Works 3m 50s 1 run 7.8kFewest 1 run 34 tok/s 1 run not reported 1 run Compare	Fails 4m 04s 1 run 27.6k 1 run 113 tok/s 1 run not reported 1 run Compare
GPT-6 Astra5 runs · 4 of 4 workCompare its efforts	No capture published Not checked 4m 17s 1 run 7.7k 1 run 30 tok/s 1 run not reported 1 run Compare	Works 4m 15s 1 run 8.4k 1 run 33 tok/s 1 run not reported 1 run Compare	Works 7m 30s 1 run 11.1k 1 run 25 tok/s 1 run not reported 1 run Compare	Works 25m 55s 1 run 36.6k 1 run 24 tok/s 1 run not reported 1 run Compare	Works 36m 32s 1 run 47.1k 1 run 21 tok/s 1 run not reported 1 run Compare
GPT-6 Luna5 runs · 4 of 5 workCompare its efforts	Fails 51.5 s 1 run 5.3k 1 run 103 tok/s 1 run not reported 1 run Compare	Works 1m 21s 1 run 9.2k 1 run 114 tok/s 1 run not reported 1 run Compare	Works 1m 27s 1 run 8.2k 1 run 93 tok/s 1 run not reported 1 run Compare	Works 4m 24s 1 run 26.3k 1 run 100 tok/s 1 run not reported 1 run Compare	Works 4m 38s 1 run 25.0k 1 run 90 tok/s 1 run not reported 1 run Compare
GPT-6 Sol5 runs · 5 of 5 workCompare its efforts	Works 2m 39s 1 run 8.0k 1 run 50 tok/s 1 run not reported 1 run Compare	Works not recorded 1 run not reported 1 run not reported 1 run not reported 1 run Compare	Works 4m 12s 1 run 16.9k 1 run 67 tok/s 1 run not reported 1 run Compare	Works 7m 11s 1 run 19.7k 1 run 46 tok/s 1 run not reported 1 run Compare	Works 13m 48s 1 run 37.4k 1 run 45 tok/s 1 run not reported 1 run Compare
GPT-6.1 Sol5 runs · 5 of 5 workCompare its efforts	Works 3m 11s 1 run 7.3k 1 run 38 tok/s 1 run not reported 1 run Compare	Works 4m 52s 1 run 8.2k 1 run 28 tok/s 1 run not reported 1 run Compare	Works 5m 15s 1 run 15.3k 1 run 49 tok/s 1 run not reported 1 run Compare	Works 11m 37s 1 run 31.1k 1 run 45 tok/s 1 run not reported 1 run Compare	Works 16m 56s 1 run 44.6k 1 run 44 tok/s 1 run not reported 1 run Compare
GPT-6.1 Terra5 runs · 5 of 5 workCompare its efforts	Works 42.9 s 1 run 6.7k 1 run 157 tok/s 1 run not reported 1 run Compare	Works 47.9 sFastest 1 run 9.0k 1 run 188 tok/s 1 run not reported 1 run Compare	Works 1m 12s 1 run 7.8k 1 run 109 tok/s 1 run not reported 1 run Compare	Works 1m 15sFastest 1 run 12.8k 1 run 171 tok/s 1 run not reported 1 run Compare	Works 4m 55s 1 run 41.5k 1 run 141 tok/s 1 run not reported 1 run Compare

The number is the median; each dot is one run. worksfailsnot checkedmedian Each effort column has its own scale; “not reported” is never read as zero.

Effort ladders

What more effort buys.

Median build time per effort, on one shared scale.

Claude Fable 5.1Low → Max
Claude Fable 5.1: median build time per effort
Low 2m 50s
Medium 4m 53s
High 14m 06s
Extra high 32m 58s
Max 36m 52s
Low → Max: Time ×13.0 · Tokens ×12.8 · Cost ×10.8
Works at every effort it ran.
Claude Haiku 5.5Low → Max
Claude Haiku 5.5: median build time per effort
Low 2m 36s
Medium 5m 06s
High 5m 22s
Extra high 8m 28s
Max 20m 00s
Low → Max: Time ×7.7 · Tokens ×7.7 · Cost ×58.2
Works at every effort it ran.
Claude Opus 5.5Low → Max
Claude Opus 5.5: median build time per effort
Low 1m 42s
Medium 8m 56s
High 13m 48s
Extra high 28m 09s
Max 66m 39s
Low → Max: Time ×39.3 · Tokens ×37.9 · Cost ×41.6
Works at every effort it ran.
Claude Sonnet 5.5Low → Max
Claude Sonnet 5.5: median build time per effort
Low 2m 37s
Medium 4m 44s
High 6m 45s
Extra high 21m 18s
Max 87m 03s
Low → Max: Time ×33.3 · Tokens ×24.7 · Cost ×43.0
Works at every effort it ran.
GPT-5.6 LunaLow → Max
GPT-5.6 Luna: median build time per effort
Low 29.7 s
Medium 59.1 s
High 1m 04s
Extra high 1m 25s
Max 2m 22s
Low → Max: Time ×4.8 · Tokens ×6.6
Works at every effort it ran.
GPT-5.6 SolLow → Max
GPT-5.6 Sol: median build time per effort
Low 1m 46s
Medium 2m 20s
High 2m 34s
Extra high 4m 05s
Max 7m 53s
Low → Max: Time ×4.5 · Tokens ×5.3
Works at low, high; fails at medium, fails at extra high, fails at max.
GPT-5.6 TerraLow → Max
GPT-5.6 Terra: median build time per effort
Low 1m 38s
Medium 1m 48s
High 1m 00s
Extra high 3m 50s
Max 4m 04s
Low → Max: Time ×2.5 · Tokens ×7.8
Works at low, medium, high, extra high; fails at max.
GPT-6 AstraLow → Max
GPT-6 Astra: median build time per effort
Low 4m 17s
Medium 4m 15s
High 7m 30s
Extra high 25m 55s
Max 36m 32s
Low → Max: Time ×8.5 · Tokens ×6.1
Works at medium, high, extra high, max.
GPT-6 LunaLow → Max
GPT-6 Luna: median build time per effort
Low 51.5 s
Medium 1m 21s
High 1m 27s
Extra high 4m 24s
Max 4m 38s
Low → Max: Time ×5.4 · Tokens ×4.7
Works at medium, high, extra high, max; fails at low.
GPT-6 SolLow → Max
GPT-6 Sol: median build time per effort
Low 2m 39s
Medium no time
High 4m 12s
Extra high 7m 11s
Max 13m 48s
Low → Max: Time ×5.2 · Tokens ×4.7
Works at every effort it ran.
GPT-6.1 SolLow → Max
GPT-6.1 Sol: median build time per effort
Low 3m 11s
Medium 4m 52s
High 5m 15s
Extra high 11m 37s
Max 16m 56s
Low → Max: Time ×5.3 · Tokens ×6.1
Works at every effort it ran.
GPT-6.1 TerraLow → Max
GPT-6.1 Terra: median build time per effort
Low 42.9 s
Medium 47.9 s
High 1m 12s
Extra high 1m 15s
Max 4m 55s
Low → Max: Time ×6.9 · Tokens ×6.2
Works at every effort it ran.

Every agent config, with its own medianstier, cancelled and unchecked runs

Each agent config on Lava lamp v1: runs, works, median build time, output tokens, output speed and API cost
Model	Effort	Runs	Works	Build time	Output tokens	Speed	API cost	Notes
Claude Fable 5.1	Low	1	1 of 1 work	2m 50s	13.5k	79 tok/s	$1.05
Claude Fable 5.1	Medium	1	1 of 1 work	4m 53s	23.5k	80 tok/s	$1.77
Claude Fable 5.1	High	1	1 of 1 work	14m 06s	63.1k	75 tok/s	$4.42
Claude Fable 5.1	Extra high	1	1 of 1 work	32m 58s	145.7k	74 tok/s	$9.06
Claude Fable 5.1	Max	1	1 of 1 work	36m 52s	172.0k	78 tok/s	$11.33
Claude Haiku 5.5	Low	1	1 of 1 work	2m 36s	34.3k	220 tok/s	$0.02
Claude Haiku 5.5	Medium	1	1 of 1 work	5m 06s	69.5k	227 tok/s	$0.05
Claude Haiku 5.5	High	1	1 of 1 work	5m 22s	75.5k	235 tok/s	$0.06
Claude Haiku 5.5	Extra high	1	1 of 1 work	8m 28s	117.3k	231 tok/s	$0.20
Claude Haiku 5.5	Max	1	1 of 1 work	20m 00s	263.9k	220 tok/s	$1.37	1 stopped at the limit
Claude Opus 5.5	Low	1	1 of 1 work	1m 42s	10.6k	104 tok/s	$0.33
Claude Opus 5.5	Medium	1	1 of 1 work	8m 56s	54.5k	102 tok/s	$2.01
Claude Opus 5.5	High	1	1 of 1 work	13m 48s	80.9k	98 tok/s	$2.40
Claude Opus 5.5	Extra high	1	1 of 1 work	28m 09s	151.6k	90 tok/s	$6.40
Claude Opus 5.5	Max	1	1 of 1 work	66m 39s	402.8k	101 tok/s	$13.82
Claude Sonnet 5.5	Low	1	1 of 1 work	2m 37s	19.4k	124 tok/s	$0.29
Claude Sonnet 5.5	Medium	1	1 of 1 work	4m 44s	34.8k	122 tok/s	$0.60
Claude Sonnet 5.5	High	1	1 of 1 work	6m 45s	52.7k	130 tok/s	$0.71
Claude Sonnet 5.5	Extra high	1	1 of 1 work	21m 18s	168.4k	132 tok/s	$2.92
Claude Sonnet 5.5	Max	1	1 of 1 work	87m 03s	479.7k	92 tok/s	$12.68
GPT-5.6 Luna	Low	1	1 of 1 work	29.7 s	3.4k	114 tok/s	not reported
GPT-5.6 Luna	Medium	1	1 of 1 work	59.1 s	6.7k	114 tok/s	not reported
GPT-5.6 Luna	High	1	1 of 1 work	1m 04s	8.0k	125 tok/s	not reported
GPT-5.6 Luna	Extra high	1	1 of 1 work	1m 25s	10.5k	125 tok/s	not reported
GPT-5.6 Luna	Max	1	1 of 1 work	2m 22s	22.4k	157 tok/s	not reported
GPT-5.6 Sol	Low	1	1 of 1 work	1m 46s	5.2k	50 tok/s	not reported
GPT-5.6 Sol	Medium	1	0 of 1 work	2m 20s	7.3k	52 tok/s	not reported	failed: visible autonomous animation; failed: 390 px mobile layout has no horizontal overflow
GPT-5.6 Sol	High	1	1 of 1 work	2m 34s	11.2k	73 tok/s	not reported
GPT-5.6 Sol	Extra high	1	0 of 1 work	4m 05s	17.1k	70 tok/s	not reported	failed: visible autonomous animation; failed: no browser errors
GPT-5.6 Sol	Max	1	0 of 1 work	7m 53s	27.6k	58 tok/s	not reported	failed: visible autonomous animation; failed: no browser errors
GPT-5.6 Terra	Low	1	1 of 1 work	1m 38s	3.6k	36 tok/s	not reported
GPT-5.6 Terra	Medium	1	1 of 1 work	1m 48s	3.5k	32 tok/s	not reported
GPT-5.6 Terra	High	1	1 of 1 work	1m 00s	7.5k	126 tok/s	not reported
GPT-5.6 Terra	Extra high	1	1 of 1 work	3m 50s	7.8k	34 tok/s	not reported
GPT-5.6 Terra	Max	1	0 of 1 work	4m 04s	27.6k	113 tok/s	not reported	failed: visible autonomous animation; failed: no browser errors
GPT-6 Astra	Low	1	not checked	4m 17s	7.7k	30 tok/s	not reported	1 not checked
GPT-6 Astra	Medium	1	1 of 1 work	4m 15s	8.4k	33 tok/s	not reported
GPT-6 Astra	High	1	1 of 1 work	7m 30s	11.1k	25 tok/s	not reported
GPT-6 Astra	Extra high	1	1 of 1 work	25m 55s	36.6k	24 tok/s	not reported
GPT-6 Astra	Max	1	1 of 1 work	36m 32s	47.1k	21 tok/s	not reported
GPT-6 Luna	Low	1	0 of 1 work	51.5 s	5.3k	103 tok/s	not reported	failed: visible autonomous animation
GPT-6 Luna	Medium	1	1 of 1 work	1m 21s	9.2k	114 tok/s	not reported
GPT-6 Luna	High	1	1 of 1 work	1m 27s	8.2k	93 tok/s	not reported
GPT-6 Luna	Extra high	1	1 of 1 work	4m 24s	26.3k	100 tok/s	not reported
GPT-6 Luna	Max	1	1 of 1 work	4m 38s	25.0k	90 tok/s	not reported
GPT-6 Sol	Low	1	1 of 1 work	2m 39s	8.0k	50 tok/s	not reported
GPT-6 Sol	Medium	1	1 of 1 work	—	—	—	not reported	1 stopped at the limit
GPT-6 Sol	High	1	1 of 1 work	4m 12s	16.9k	67 tok/s	not reported
GPT-6 Sol	Extra high	1	1 of 1 work	7m 11s	19.7k	46 tok/s	not reported
GPT-6 Sol	Max	1	1 of 1 work	13m 48s	37.4k	45 tok/s	not reported
GPT-6.1 Sol	Low	1	1 of 1 work	3m 11s	7.3k	38 tok/s	not reported
GPT-6.1 Sol	Medium	1	1 of 1 work	4m 52s	8.2k	28 tok/s	not reported
GPT-6.1 Sol	High	1	1 of 1 work	5m 15s	15.3k	49 tok/s	not reported
GPT-6.1 Sol	Extra high	1	1 of 1 work	11m 37s	31.1k	45 tok/s	not reported
GPT-6.1 Sol	Max	1	1 of 1 work	16m 56s	44.6k	44 tok/s	not reported
GPT-6.1 Terra	Low	1	1 of 1 work	42.9 s	6.7k	157 tok/s	not reported
GPT-6.1 Terra	Medium	1	1 of 1 work	47.9 s	9.0k	188 tok/s	not reported
GPT-6.1 Terra	High	1	1 of 1 work	1m 12s	7.8k	109 tok/s	not reported
GPT-6.1 Terra	Extra high	1	1 of 1 work	1m 15s	12.8k	171 tok/s	not reported
GPT-6.1 Terra	Max	1	1 of 1 work	4m 55s	41.5k	141 tok/s	not reported

Brief & checks

The exact prompt

Build a lava lamp as a single index.html file with no external requests. Show a glass lamp on a desk. Wax blobs rise as they warm, sink as they cool, and merge and split when they touch. Add a switch that turns the lamp off and on. It should hold 60 fps and fit a phone screen.

Prompt SHA-256: 41373a84940d431918ef0e93e9325d0d5f1e69b3300d30825166abeac9ff2a47
Inputs: None — prompt only
Output: One self-contained index.html, no network
Contract: Legacy v1 — never paired with v2 runs

Checks × runs

460 / 480 passed

Saved check results: rows are checks, columns are runs
Check	Claude Code · high, round 1	Claude Code · xhigh, round 2	Claude Code · max, round 3	Claude Code · low, round 4	Claude Code · medium, round 5	Claude Code · low, round 1	Claude Code · medium, round 1	Claude Code · high, round 2	Claude Code · xhigh, round 3	Claude Code · max, round 4	Claude Code · low, round 1	Claude Code · medium, round 1	Claude Code · high, round 2	Claude Code · xhigh, round 3	Claude Code · max, round 4	Codex CLI · medium, round 1	Codex CLI · low, round 2	Codex CLI · high, round 3	Codex CLI · xhigh, round 4	Codex CLI · max, round 5	Codex CLI · low, round 1	Codex CLI · medium, round 2	Codex CLI · high, round 3	Codex CLI · xhigh, round 4	Codex CLI · max, round 5	Claude Code · low, round 1	Claude Code · medium, round 2	Claude Code · high, round 3	Claude Code · xhigh, round 4	Claude Code · max, round 5	Codex CLI · low, round 1	Codex CLI · medium, round 2	Codex CLI · high, round 3	Codex CLI · xhigh, round 4	Codex CLI · max, round 5	Codex CLI · low, round 1	Codex CLI · medium, round 2	Codex CLI · high, round 3	Codex CLI · xhigh, round 4	Codex CLI · max, round 5	Codex CLI · low, round 1	Codex CLI · medium, round 2	Codex CLI · high, round 3	Codex CLI · xhigh, round 4	Codex CLI · max, round 5	Codex CLI · low, round 1	Codex CLI · medium, round 2	Codex CLI · high, round 3	Codex CLI · xhigh, round 4	Codex CLI · max, round 5	Codex CLI · low, round 1	Codex CLI · medium, round 2	Codex CLI · high, round 3	Codex CLI · xhigh, round 4	Codex CLI · max, round 5	Codex CLI · low, round 1	Codex CLI · medium, round 2	Codex CLI · xhigh, round 3	Codex CLI · max, round 4	Codex CLI · high, round 5
Self-contained index.html is present blocker	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	?	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Visible page content blocker	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	?	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Visible autonomous animation blocker	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	?	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	×	✓	×	×	✓	✓	✓	✓	×	×	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Accessible on/off control	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	?	✓	✓	✓	✓	✓	×	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
390 px mobile layout has no horizontal overflow	✓	✓	✓	✓	✓	✓	✓	×	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	?	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	×	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
Local animation frame performance	✓	✓	×	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	?	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
No browser errors blocker	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	?	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	×	×	✓	✓	✓	✓	×	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓
No attempted external or linked-resource requests blocker	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	?	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓

460 of 480 check results passed. 8 of 60 runs failed at least one check, 5 on a blocker. 8 results are not recorded. Looks are decided in blind review — 0 of 60 runs reviewed.

All 60 runs, one by oneoldest first

Published runs of Lava lamp v1
Run	Agent config	Works	Checks	Episode	Started
Claude Code, medium effort, 30.09.2026 16:03	claude-sonnet-5-5 · medium effort	works	8/8	none	30.09.2026 16:03
Claude Code, low effort, 30.09.2026 16:08	claude-sonnet-5-5 · low effort	works	8/8	none	30.09.2026 16:08
Claude Code, medium effort, 30.09.2026 19:41	claude-opus-5-5 · medium effort	works	8/8	none	30.09.2026 19:41
Claude Code, low effort, 30.09.2026 19:51	claude-opus-5-5 · low effort	works	8/8	none	30.09.2026 19:51
Codex CLI, medium effort, 02.10.2026 05:23	gpt-6.1-sol · medium effort	works	8/8	none	02.10.2026 05:23
Codex CLI, low effort, 02.10.2026 05:47	gpt-6-sol · low effort	works	8/8	none	02.10.2026 05:47
Codex CLI, medium effort, 02.10.2026 05:51	gpt-6-sol · medium effort	works	8/8	none	02.10.2026 05:51
Codex CLI, low effort, 02.10.2026 06:02	gpt-5.6-sol · low effort	works	8/8	none	02.10.2026 06:02
Codex CLI, medium effort, 02.10.2026 06:05	gpt-5.6-sol · medium effort	does not work	6/8	none	02.10.2026 06:05
Claude Code, high effort, 07.10.2026 21:22	claude-haiku-5-5 · high effort	works	8/8	none	07.10.2026 21:22
Claude Code, xhigh effort, 07.10.2026 21:28	claude-haiku-5-5 · xhigh effort	works	8/8	none	07.10.2026 21:28
Claude Code, max effort, 07.10.2026 21:39	claude-haiku-5-5 · max effort	works	7/8	none	07.10.2026 21:39
Codex CLI, low effort, 08.10.2026 19:01	gpt-6-astra · low effort	pending	0/0	none	08.10.2026 19:01
Codex CLI, low effort, 14.10.2026 07:24	gpt-6.1-sol · low effort	works	8/8	none	14.10.2026 07:24
Codex CLI, low effort, 26.10.2026 20:45	gpt-6.1-terra · low effort	works	8/8	none	26.10.2026 20:45
Codex CLI, medium effort, 26.10.2026 20:46	gpt-6.1-terra · medium effort	works	8/8	none	26.10.2026 20:46
Codex CLI, xhigh effort, 26.10.2026 20:49	gpt-6.1-terra · xhigh effort	works	8/8	none	26.10.2026 20:49
Codex CLI, max effort, 26.10.2026 21:00	gpt-6.1-terra · max effort	works	8/8	none	26.10.2026 21:00
Codex CLI, high effort, 28.10.2026 07:05	gpt-6.1-sol · high effort	works	8/8	none	28.10.2026 07:05
Codex CLI, high effort, 28.10.2026 07:33	gpt-6.1-terra · high effort	works	8/8	none	28.10.2026 07:33
Codex CLI, high effort, 28.10.2026 07:36	gpt-6-sol · high effort	works	8/8	none	28.10.2026 07:36
Codex CLI, xhigh effort, 28.10.2026 07:42	gpt-6.1-sol · xhigh effort	works	8/8	none	28.10.2026 07:42
Claude Code, low effort, 28.10.2026 07:44	claude-haiku-5-5 · low effort	works	8/8	none	28.10.2026 07:44
Claude Code, medium effort, 28.10.2026 07:47	claude-haiku-5-5 · medium effort	works	8/8	none	28.10.2026 07:47
Claude Code, high effort, 28.10.2026 07:54	claude-sonnet-5-5 · high effort	works	7/8	none	28.10.2026 07:54
Codex CLI, max effort, 28.10.2026 07:55	gpt-6.1-sol · max effort	works	8/8	none	28.10.2026 07:55
Claude Code, xhigh effort, 28.10.2026 08:01	claude-sonnet-5-5 · xhigh effort	works	8/8	none	28.10.2026 08:01
Codex CLI, xhigh effort, 28.10.2026 08:12	gpt-6-sol · xhigh effort	works	8/8	none	28.10.2026 08:12
Codex CLI, max effort, 28.10.2026 08:21	gpt-6-sol · max effort	works	8/8	none	28.10.2026 08:21
Codex CLI, high effort, 28.10.2026 08:35	gpt-5.6-sol · high effort	works	8/8	none	28.10.2026 08:35
Codex CLI, xhigh effort, 28.10.2026 08:39	gpt-5.6-sol · xhigh effort	does not work	6/8	none	28.10.2026 08:39
Codex CLI, max effort, 28.10.2026 08:44	gpt-5.6-sol · max effort	does not work	6/8	none	28.10.2026 08:44
Codex CLI, medium effort, 28.10.2026 08:53	gpt-6-astra · medium effort	works	8/8	none	28.10.2026 08:53
Codex CLI, high effort, 28.10.2026 08:58	gpt-6-astra · high effort	works	8/8	none	28.10.2026 08:58
Codex CLI, xhigh effort, 28.10.2026 09:07	gpt-6-astra · xhigh effort	works	8/8	none	28.10.2026 09:07
Claude Code, high effort, 28.10.2026 09:25	claude-opus-5-5 · high effort	works	8/8	none	28.10.2026 09:25
Codex CLI, max effort, 28.10.2026 09:34	gpt-6-astra · max effort	works	8/8	none	28.10.2026 09:34
Claude Code, xhigh effort, 28.10.2026 09:39	claude-opus-5-5 · xhigh effort	works	8/8	none	28.10.2026 09:39
Codex CLI, low effort, 28.10.2026 10:11	gpt-6-luna · low effort	does not work	7/8	none	28.10.2026 10:11
Codex CLI, medium effort, 28.10.2026 10:13	gpt-6-luna · medium effort	works	8/8	none	28.10.2026 10:13
Codex CLI, high effort, 28.10.2026 10:15	gpt-6-luna · high effort	works	8/8	none	28.10.2026 10:15
Codex CLI, xhigh effort, 28.10.2026 10:18	gpt-6-luna · xhigh effort	works	8/8	none	28.10.2026 10:18
Codex CLI, max effort, 28.10.2026 10:23	gpt-6-luna · max effort	works	8/8	none	28.10.2026 10:23
Codex CLI, low effort, 28.10.2026 10:29	gpt-5.6-terra · low effort	works	8/8	none	28.10.2026 10:29
Codex CLI, medium effort, 28.10.2026 10:31	gpt-5.6-terra · medium effort	works	8/8	none	28.10.2026 10:31
Codex CLI, high effort, 28.10.2026 10:34	gpt-5.6-terra · high effort	works	8/8	none	28.10.2026 10:34
Codex CLI, xhigh effort, 28.10.2026 10:36	gpt-5.6-terra · xhigh effort	works	8/8	none	28.10.2026 10:36
Codex CLI, max effort, 28.10.2026 10:41	gpt-5.6-terra · max effort	does not work	6/8	none	28.10.2026 10:41
Codex CLI, low effort, 28.10.2026 10:46	gpt-5.6-luna · low effort	works	8/8	none	28.10.2026 10:46
Codex CLI, medium effort, 28.10.2026 10:48	gpt-5.6-luna · medium effort	works	8/8	none	28.10.2026 10:48
Codex CLI, high effort, 28.10.2026 10:50	gpt-5.6-luna · high effort	works	8/8	none	28.10.2026 10:50
Codex CLI, xhigh effort, 28.10.2026 10:52	gpt-5.6-luna · xhigh effort	works	8/8	none	28.10.2026 10:52
Codex CLI, max effort, 28.10.2026 10:54	gpt-5.6-luna · max effort	works	8/8	none	28.10.2026 10:54
Claude Code, low effort, 28.10.2026 11:10	claude-fable-5-1 · low effort	works	8/8	none	28.10.2026 11:10
Claude Code, medium effort, 28.10.2026 11:14	claude-fable-5-1 · medium effort	works	7/8	none	28.10.2026 11:14
Claude Code, high effort, 28.10.2026 11:20	claude-fable-5-1 · high effort	works	8/8	none	28.10.2026 11:20
Claude Code, xhigh effort, 28.10.2026 11:35	claude-fable-5-1 · xhigh effort	works	8/8	none	28.10.2026 11:35
Claude Code, max effort, 28.10.2026 12:09	claude-fable-5-1 · max effort	works	8/8	none	28.10.2026 12:09
Claude Code, max effort, 28.10.2026 13:12	claude-sonnet-5-5 · max effort	works	8/8	none	28.10.2026 13:12
Claude Code, max effort, 28.10.2026 14:40	claude-opus-5-5 · max effort	works	8/8	none	28.10.2026 14:40

Identity of this versioncontract v1 · prompt hash

This task predates frozen benchmark packages: its checks are the original ones each run lists, and the prompt hash is its only identity.

Identity

contract v1 · SHA-256

Prompt: 41373a84940d431918ef0e93e9325d0d5f1e69b3300d30825166abeac9ff2a47

Machine-readableDownload benchmark.json