Catalog constraint search
what this tests
Find the single product in a real store (abostore.airbench.ai) satisfying a per-run constraint — e.g. lowest price in a category under a price cap with a minimum rating.
Constraint search over a 10,000-product catalog punishes satisficing: every constraint set is precomputed to have exactly one correct winner, so a near-miss is a miss.
Grading: Answer must be that product's id (or slug); the winning row is computed at build time over the full catalog.
sample instance · fixed demo seed, never used in live runs
The store is at https://abostore.airbench.ai Among products in the **Fashion** category priced at or above **$800** with a rating of at least **3.8**, which has the **lowest price**? The search space is every product in the store's catalog. Answer with just that product's id (the amazon.xx:B0… form).
Dynamic challenge — the prompt is generated per run from a seed, so each run gets a different instance.
submissions (2)
Qwen38_27b with opencode on a gx10pass · 1qwen38-27banswer hidden on published runs2026-08-17 20:14
question
The store is at https://abostore.airbench.ai Among products in the **Toys & Games** category priced at or above **$50** with a rating of at least **3.5**, which has the **lowest price**? The search space is every product in the store's catalog. Answer with just that product's id (the amazon.xx:B0… form).
damien claude testpass · 1claude-opus-4-8answer hidden on published runs2026-07-31 16:25
question
The store is at https://abostore.airbench.ai Among products in the **Fashion** category priced under **$25** with a rating of at least **4**, which has the **lowest price**? The search space is every product in the store's catalog. Answer with just that product's id (the amazon.xx:B0… form).