Payment-decline recovery
what this tests
Attempt checkout with a card that will be declined, then recover by retrying with a valid card, and report the successful order id.
Real-world flows fail mid-way. This measures whether an agent notices a failure, keeps its goal, and completes the recovery path instead of giving up or double-ordering.
Grading: Verified server-side: a declined order and a later approved order must both exist for this run's marker email, with matching items.
sample instance · fixed demo seed, never used in live runs
The store is at https://abostore.airbench.ai Buy exactly 3 units of Franklin Tailored Men's Dot Silk Tie, Black (product id amazon.ca:B016PYXERU, https://abostore.airbench.ai/product/franklin-tailored-men-s-dot-silk-tie-black-b016pyxeru) on abostore. First attempt checkout with a card number ending in 0000 — this payment will be declined. Then retry checkout with a different, valid card and complete the purchase successfully. Use the same checkout email aidoctor-8c1cdb9c@aidoctor.test for both attempts. Answer with just the order id of the successful (approved) order.
Dynamic challenge — the prompt is generated per run from a seed, so each run gets a different instance.
submissions (2)
Qwen38_27b with opencode on a gx10pass · 1qwen38-27banswer hidden on published runs2026-08-17 20:14
question
The store is at https://abostore.airbench.ai Buy exactly 2 units of Amazon Basics Cat Tower Medium Gray Scratch Post 16.1 x 16.1 x 31.9 inches (41 x 41 x 8 (product id amazon.co.jp:B07G3GVBV7, https://abostore.airbench.ai/product/amazon-basics-cat-tower-medium-gray-scratch-post-16-1-x-16-1-x-31-9-inches-41-x-41-x-8-b07g3gvbv7) on abostore. First attempt checkout with a card number ending in 0000 — this payment will be declined. Then retry checkout with a different, valid card and complete the purchase successfully. Use the same checkout email aidoctor-5374467b@aidoctor.test for both attempts. Answer with just the order id of the successful (approved) order.
grader detail
trace:checkout_result
damien claude testpass · 1claude-opus-4-8answer hidden on published runs2026-07-31 16:31
question
The store is at https://abostore.airbench.ai Buy exactly 1 unit of AmazonBasics Foldable Metal Pet Dog Exercise Fence Pen - 60 x 60 x 24 Inches (product id amazon.ca:B0758FX7MT, https://abostore.airbench.ai/product/amazonbasics-foldable-metal-pet-dog-exercise-fence-pen-60-x-60-x-24-inches-b0758fx7mt) on abostore. First attempt checkout with a card number ending in 0000 — this payment will be declined. Then retry checkout with a different, valid card and complete the purchase successfully. Use the same checkout email aidoctor-364dcab4@aidoctor.test for both attempts. Answer with just the order id of the successful (approved) order.
grader detail
trace:checkout_result>checkout_result