About CmdBench

CmdBench is not related to the cmdbench Python package or to CMDBench, the Megagon Labs data-discovery benchmark.

CmdBench is a benchmark for AI coding agents. We give each agent the same small build task, check the result with hidden automated tests, and publish every run: the prompt, the transcript, the source and desktop and phone captures.

CmdBench is run by Jaanus Lepik. The method is described on the methodology and protocol pages, and mistakes we find are listed on the corrections page.

Disclosure

CmdBench is an independent benchmark made by CmdBrief, operated by J&L Serverid OÜ.

Company and contact

J&L Serverid OÜ
Registry code 11478495
Email [email protected]

How we handle personal data is described in the privacy policy.