Agent
Codex CLI
Codex CLI is an AI coding agent made by OpenAI.
- Episodes
- 0
- Published runs
- 40
What it built · Lava lamp v1
8 models, 40 runs.
| Model | Low8 models | Medium8 models | High8 models | Extra high8 models | Max8 models |
|---|---|---|---|---|---|
| Codex CLI | |||||
| GPT-5.6 Luna5 runs · 5 of 5 work |  Works 29.7 sFastest 1 run |  Works 59.1 s 1 run |  Works 1m 04s 1 run |  Works 1m 25s 1 run |  Works 2m 22sFastest 1 run |
| GPT-5.6 Sol5 runs · 2 of 5 work |  Works 1m 46s 1 run |  Fails 2m 20s 1 run |  Works 2m 34s 1 run |  Fails 4m 05s 1 run |  Fails 7m 53s 1 run |
| GPT-5.6 Terra5 runs · 4 of 5 work |  Works 1m 38s 1 run |  Works 1m 48s 1 run |  Works 1m 00sFastest 1 run |  Works 3m 50s 1 run |  Fails 4m 04s 1 run |
| GPT-6 Astra5 runs · 4 of 4 work | No capture published Not checked 4m 17s 1 run |  Works 4m 15s 1 run |  Works 7m 30s 1 run |  Works 25m 55s 1 run |  Works 36m 32s 1 run |
| GPT-6 Luna5 runs · 4 of 5 work |  Fails 51.5 s 1 run |  Works 1m 21s 1 run |  Works 1m 27s 1 run |  Works 4m 24s 1 run |  Works 4m 38s 1 run |
| GPT-6 Sol5 runs · 5 of 5 work |  Works 2m 39s 1 run |  Works not recorded 1 run |  Works 4m 12s 1 run |  Works 7m 11s 1 run |  Works 13m 48s 1 run |
| GPT-6.1 Sol5 runs · 5 of 5 work |  Works 3m 11s 1 run |  Works 4m 52s 1 run |  Works 5m 15s 1 run |  Works 11m 37s 1 run |  Works 16m 56s 1 run |
| GPT-6.1 Terra5 runs · 5 of 5 work |  Works 42.9 s 1 run |  Works 47.9 sFastest 1 run |  Works 1m 12s 1 run |  Works 1m 15sFastest 1 run |  Works 4m 55s 1 run |
Low effort · 8 models · fastest first
GPT-5.6 Luna29.7 s
Works
GPT-6.1 Terra42.9 s
Works
GPT-6 Luna51.5 s
Fails
GPT-5.6 Terra1m 38s
Works
GPT-5.6 Sol1m 46s
Works
GPT-6 Sol2m 39s
Works
GPT-6.1 Sol3m 11s
Works
- No capture published
GPT-6 Astra4m 17s
Not checked
Medium effort · 8 models · fastest first
GPT-6.1 Terra47.9 s
Works
GPT-5.6 Luna59.1 s
Works
GPT-6 Luna1m 21s
Works
GPT-5.6 Terra1m 48s
Works
GPT-5.6 Sol2m 20s
Fails
GPT-6 Astra4m 15s
Works
GPT-6.1 Sol4m 52s
Works
GPT-6 Solno time
Works
High effort · 8 models · fastest first
GPT-5.6 Terra1m 00s
Works
GPT-5.6 Luna1m 04s
Works
GPT-6.1 Terra1m 12s
Works
GPT-6 Luna1m 27s
Works
GPT-5.6 Sol2m 34s
Works
GPT-6 Sol4m 12s
Works
GPT-6.1 Sol5m 15s
Works
GPT-6 Astra7m 30s
Works
Extra high effort · 8 models · fastest first
GPT-6.1 Terra1m 15s
Works
GPT-5.6 Luna1m 25s
Works
GPT-5.6 Terra3m 50s
Works
GPT-5.6 Sol4m 05s
Fails
GPT-6 Luna4m 24s
Works
GPT-6 Sol7m 11s
Works
GPT-6.1 Sol11m 37s
Works
GPT-6 Astra25m 55s
Works
Max effort · 8 models · fastest first
GPT-5.6 Luna2m 22s
Works
GPT-5.6 Terra4m 04s
Fails
GPT-6 Luna4m 38s
Works
GPT-6.1 Terra4m 55s
Works
GPT-5.6 Sol7m 53s
Fails
GPT-6 Sol13m 48s
Works
GPT-6.1 Sol16m 56s
Works
GPT-6 Astra36m 32s
Works
Build time: the number is the median, each dot one run, on one scale per effort.
About Codex CLI
About
- Vendor
- OpenAI
- Official site
- https://learn.chatgpt.com/docs/codex/cli
- CmdBench support
- runner
What this repository records
Only facts a file in this repository proves, with that file. Each run records its own harness version, binary hash, model, effort and plan; nothing on this page overrides a run.
- CmdBench runner support exists (provider "codex", harness label "Codex CLI").Source: src/server.ts, src/runner/codex.ts
- Approved harness build is Codex 0.159.2 (aarch64-apple-darwin), SHA-256 16593cc2f422d5f398a8e40f550ebbaf1245392528957be342c295920a300704; every attempt rechecks it.Source: docs/benchmark-contract-v2.md
- Reference images are passed as native --image attachments in manifest order.Source: docs/benchmark-contract-v2.md, src/runner/codex.ts
- Native video attachment is unavailable; a clip plus timed still frames is the declared transport.Source: docs/benchmark-contract-v2.md
- Runner model constant is gpt-6.1-sol.Source: src/runner/codex.ts
Not recorded yet
- harness version for cb-001 (spec allows 0.159.2 or 0.163.0; not decided)
- plan used for runs (subscription or API)
- usage fields reported (tokens, cost, plan usage)
Versions and models tested
| Harness version | Model | Effort | Runs | First run | Last run |
|---|---|---|---|---|---|
| codex-cli 0.165.0 | gpt-5.6-luna | max | 1 | 28.10.2026 10:54 | 28.10.2026 10:54 |
| codex-cli 0.165.0 | gpt-5.6-luna | xhigh | 1 | 28.10.2026 10:52 | 28.10.2026 10:52 |
| codex-cli 0.165.0 | gpt-5.6-luna | high | 1 | 28.10.2026 10:50 | 28.10.2026 10:50 |
| codex-cli 0.165.0 | gpt-5.6-luna | medium | 1 | 28.10.2026 10:48 | 28.10.2026 10:48 |
| codex-cli 0.165.0 | gpt-5.6-luna | low | 1 | 28.10.2026 10:46 | 28.10.2026 10:46 |
| codex-cli 0.165.0 | gpt-5.6-terra | max | 1 | 28.10.2026 10:41 | 28.10.2026 10:41 |
| codex-cli 0.165.0 | gpt-5.6-terra | xhigh | 1 | 28.10.2026 10:36 | 28.10.2026 10:36 |
| codex-cli 0.165.0 | gpt-5.6-terra | high | 1 | 28.10.2026 10:34 | 28.10.2026 10:34 |
| codex-cli 0.165.0 | gpt-5.6-terra | medium | 1 | 28.10.2026 10:31 | 28.10.2026 10:31 |
| codex-cli 0.165.0 | gpt-5.6-terra | low | 1 | 28.10.2026 10:29 | 28.10.2026 10:29 |
| codex-cli 0.165.0 | gpt-6-luna | max | 1 | 28.10.2026 10:23 | 28.10.2026 10:23 |
| codex-cli 0.165.0 | gpt-6-luna | xhigh | 1 | 28.10.2026 10:18 | 28.10.2026 10:18 |
| codex-cli 0.165.0 | gpt-6-luna | high | 1 | 28.10.2026 10:15 | 28.10.2026 10:15 |
| codex-cli 0.165.0 | gpt-6-luna | medium | 1 | 28.10.2026 10:13 | 28.10.2026 10:13 |
| codex-cli 0.165.0 | gpt-6-luna | low | 1 | 28.10.2026 10:11 | 28.10.2026 10:11 |
| codex-cli 0.165.0 | gpt-6-astra | max | 1 | 28.10.2026 09:34 | 28.10.2026 09:34 |
| codex-cli 0.165.0 | gpt-6-astra | xhigh | 1 | 28.10.2026 09:07 | 28.10.2026 09:07 |
| codex-cli 0.165.0 | gpt-6-astra | high | 1 | 28.10.2026 08:58 | 28.10.2026 08:58 |
| codex-cli 0.165.0 | gpt-6-astra | medium | 1 | 28.10.2026 08:53 | 28.10.2026 08:53 |
| codex-cli 0.165.0 | gpt-5.6-sol | max | 1 | 28.10.2026 08:44 | 28.10.2026 08:44 |
| codex-cli 0.165.0 | gpt-5.6-sol | xhigh | 1 | 28.10.2026 08:39 | 28.10.2026 08:39 |
| codex-cli 0.165.0 | gpt-5.6-sol | high | 1 | 28.10.2026 08:35 | 28.10.2026 08:35 |
| codex-cli 0.165.0 | gpt-6-sol | max | 1 | 28.10.2026 08:21 | 28.10.2026 08:21 |
| codex-cli 0.165.0 | gpt-6-sol | xhigh | 1 | 28.10.2026 08:12 | 28.10.2026 08:12 |
| codex-cli 0.165.0 | gpt-6.1-sol | max | 1 | 28.10.2026 07:55 | 28.10.2026 07:55 |
| codex-cli 0.165.0 | gpt-6.1-sol | xhigh | 1 | 28.10.2026 07:42 | 28.10.2026 07:42 |
| codex-cli 0.165.0 | gpt-6-sol | high | 1 | 28.10.2026 07:36 | 28.10.2026 07:36 |
| codex-cli 0.165.0 | gpt-6.1-terra | high | 1 | 28.10.2026 07:33 | 28.10.2026 07:33 |
| codex-cli 0.165.0 | gpt-6.1-sol | high | 1 | 28.10.2026 07:05 | 28.10.2026 07:05 |
| codex-cli 0.164.0 | gpt-6.1-terra | max | 1 | 26.10.2026 21:00 | 26.10.2026 21:00 |
| codex-cli 0.164.0 | gpt-6.1-terra | xhigh | 1 | 26.10.2026 20:49 | 26.10.2026 20:49 |
| codex-cli 0.164.0 | gpt-6.1-terra | medium | 1 | 26.10.2026 20:46 | 26.10.2026 20:46 |
| codex-cli 0.164.0 | gpt-6.1-terra | low | 1 | 26.10.2026 20:45 | 26.10.2026 20:45 |
| codex-cli 0.162.1 | gpt-6.1-sol | low | 1 | 14.10.2026 07:24 | 14.10.2026 07:24 |
| codex-cli 0.161.0 | gpt-6-astra | low | 1 | 08.10.2026 19:01 | 08.10.2026 19:01 |
| codex-cli 0.160.0 | gpt-5.6-sol | medium | 1 | 02.10.2026 06:05 | 02.10.2026 06:05 |
| codex-cli 0.160.0 | gpt-5.6-sol | low | 1 | 02.10.2026 06:02 | 02.10.2026 06:02 |
| codex-cli 0.160.0 | gpt-6-sol | medium | 1 | 02.10.2026 05:51 | 02.10.2026 05:51 |
| codex-cli 0.160.0 | gpt-6-sol | low | 1 | 02.10.2026 05:47 | 02.10.2026 05:47 |
| codex-cli 0.160.0 | gpt-6.1-sol | medium | 1 | 02.10.2026 05:23 | 02.10.2026 05:23 |