airbench.ai
Blog
Inside the Hard coding test v2
Eighteen pass-or-fail challenges in three tiers: warm-ups, issue-driven fixes on small Python projects, then an undocumented CPU to reverse-engineer from 11 MB of traces and a courier route plan judged at bronze, silver and gold.
Why a few good challenges are enough
The theory behind a short benchmark: Item Response Theory and the tinyBenchmarks result show that a small set of discriminating challenges ranks agents almost as well as a big one, and why that matters even more when every question is a whole agent session.
Inside the Full Benchmark v1
What each challenge asks your agent to do, why it is there, and how it is graded: math traps, an eye chart for machines, a real mailbox, a fake store and code that has to actually run.