benchmark
AI Doctor — Hands (web agency) v1
First-party AI Doctor checkup: web-agency challenges against a real store at https://abostore.airbench.ai. Prompts are not stored statically — each challenge has a native handler that generates a randomly-seeded question (and expected answer/marker) per run. What's graded: a constraint search over the store's product catalog, and completing a multi-step checkout (including a decline-then-retry recovery flow), verified server-side against the store's own order records (status, timing, checkout email, items, and price) — not against anything the agent merely claims. The store also records a browsing trace (page views, cart events) per session; that trace is surfaced in the result detail as an ungraded finding and never affects pass/fail, so it should not be read as proof of UI navigation — only the final order state is graded.
The hands axis: acting on a real store (abostore.airbench.ai) — a constraint search over a 10,000-product catalog, an end-to-end checkout, and a payment-decline recovery.
Each run gets its own generated task and marker email. Constraint-search instances are precomputed to have exactly one correct answer.
Transactions are verified against the store's own order records — status, timing, checkout email, items, price — never against what the agent claims. The browsing trace shown in results is an ungraded finding. Part of the AI Doctor checkup →
time budget · 30 min
challenges
- find-product-1Catalog constraint search — Find the single product in a real store (abostore.airbench.ai) satisfying a per-run constraint — e.g. lowest price in a category under a price cap with a minimum rating.
- find-product-2Catalog constraint search — Find the single product in a real store (abostore.airbench.ai) satisfying a per-run constraint — e.g. lowest price in a category under a price cap with a minimum rating.
- purchase-1End-to-end checkout — Buy specified items on the store — search, cart, checkout with a given email — and report the resulting order id.
- recover-decline-1Payment-decline recovery — Attempt checkout with a card that will be declined, then recover by retrying with a valid card, and report the successful order id.
recent runs
- Qwen38_27b with opencode on a gx102026-08-17 14:32
- damien claude test2026-07-31 16:12