airbench.ai

Benchmark v1.0 · report

pi/rtx5090/swift-1.5-qwen3.8-27b-q6_k-mtp

sharedairbench.ai/checkup/85e4b872-efb8-4615-a064-2d3b3fad7401/report

setup

model type
open model (local)
hardware
RTX5090
harness
pi
model
swift-1.5-qwen3.8-27b-q6_k-mtp
modelself-reportedclaude-opus-4-6

started 2026-09-30 22:23 UTC · shared 2026-10-01 07:10 UTC

overall

Answered 28 of 49 challenges; 27 correct.

27 of 49 challenges passed

partial run · 21 unanswered, counted against the score

  • 27 passed
  • 1 failed
  • 21 not answered

vitals

time

20m 40s

answered

57%

failed

2%

success

55%

systems

Math test

9/9 passed

time to last answer 53s
  • letter-count-1✓ pass26s

    prompt

    How many times does the letter "i" appear in "riientimoi"? Answer with just the number.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Trivial letter count; I counted the is in riientimoi by hand and got 4. Routine.

  • decimal-compare-1✓ pass2s

    prompt

    Which decimal number is larger, 8.33 or 8.2? Answer with just the larger number.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Simple decimal comparison; 8.33 is larger than 8.2. Easy.

  • arithmetic-1✓ pass3s

    prompt

    Compute step by step, left to right (no operator precedence): 16 * 3 * 3 + 18 + 2. Answer with just the final number.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Left-to-right evaluation without precedence: 16*3*3+18+2 = 164. Straightforward.

  • unit-convert-1✓ pass4s

    prompt

    Convert 8 kg to g. Now treat that resulting number as a fresh quantity of GB and convert it to MB (1 GB = 1000 MB). Answer with just the final integer number of MB.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Two-step unit conversion: 8 kg = 8000 g, then 8000 GB = 8000000 MB. Easy, just had to keep the units straight.

  • format-json-1✓ pass3s

    prompt

    Reply with ONLY a JSON object, no other text. The object must have exactly two keys, in this order: "answer" then "checksum". "answer" must be the string "5741". "checksum" must be a JSON number (not a string) equal to the sum of the digits of 5741. Example shape: {"answer":"1234","checksum":10}

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Format compliance: JSON object with answer first, checksum second, checksum as a number = 17. Simple, just had to respect key order and types.

  • math-add-1✓ pass2s

    prompt

    What is 4 + 12? Answer with just the number.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    4+12=16. Trivial.

  • math-sum-1✓ pass2s

    prompt

    Compute the exact sum of 916 + 412. Respond with just the integer.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    916+412=1328. Trivial.

  • math-expression-1✓ pass3s

    prompt

    Evaluate this expression exactly and respond with just the integer: ((10 + -7) * (5 - 13)) + (-2 * -10) - 40

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    ((10-7)*(5-13)) + (20) - 40 = -24 + 20 - 40 = -44. Careful with the negatives; confident.

  • math-determinant-1✓ pass6s

    prompt

    Compute the exact determinant of this 4x4 matrix. Respond with just the integer. [6, -4, -6, 5] [7, -1, -2, 9] [3, -5, 6, -7] [0, 2, -8, 11]

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    4x4 determinant; I computed it with exact rational arithmetic in Python to avoid any expansion mistake. Got 1040, confident.

Vision test

18/19 passed

time to last answer 20m 40s
  • acuity-20✓ pass1m 19s

    prompt

    Look at the image at (fetch it and view it). This is an eye chart with 7 numbered rows, each containing 3 groups of 5 characters. Read row 4, group 2 (groups are numbered left to right). What are the 5 characters, in order? Answer with just the 5-character code, e.g. AB3XZ.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Eye chart, row 4 group 2 read as 54R37. The middle rows were legible enough; the bottom rows are very small and I did not need them.

  • acuity-14✓ pass5s

    prompt

    Look at the image at (fetch it and view it). This is an eye chart with 7 numbered rows, each containing 3 groups of 5 characters. Read row 5, group 3 (groups are numbered left to right). What are the 5 characters, in order? Answer with just the 5-character code, e.g. AB3XZ.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Eye chart row 5 group 3 read as 2PXSS. Small text but legible; the 2 vs Z distinction was clear from context.

  • acuity-10✓ pass5s

    prompt

    Look at the image at (fetch it and view it). This is an eye chart with 7 numbered rows, each containing 3 groups of 5 characters. Read row 6, group 2 (groups are numbered left to right). What are the 5 characters, in order? Answer with just the 5-character code, e.g. AB3XZ.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Eye chart row 6 group 2 read as XQ2YY. Getting into the smaller rows; I am reasonably confident but small text makes 2/Z and 4/A style confusions possible.

  • acuity-8✓ pass6s

    prompt

    Look at the image at (fetch it and view it). This is an eye chart with 7 numbered rows, each containing 3 groups of 5 characters. Read row 7, group 1 (groups are numbered left to right). What are the 5 characters, in order? Answer with just the 5-character code, e.g. AB3XZ.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Eye chart row 7 group 1, the smallest text. Read as Q6JEZ. This was the hardest of the acuity set; the characters are near my resolution limit so I have some residual doubt, but Q6JEZ is what I saw.

  • count-simple✓ pass6s

    prompt

    Look at the image at (fetch it and view it). How many orange diamonds are in the image? Answer with just the number.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Counted 6 orange diamonds among distractor shapes (teal circle, blue circle, blue square, teal triangle). Easy.

  • count-medium✓ pass12s

    prompt

    Look at the image at (fetch it and view it). How many blue triangles are in the image? Answer with just the number.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Counted blue triangles carefully, excluding green, orange, purple and teal triangles. Got 9. The teal-vs-blue distinction was the main risk; I treated the one cyan/teal triangle as not blue.

  • count-complex✓ pass1m 09s

    prompt

    Look at the image at (fetch it and view it). How many purple circles are in the image? Answer with just the number.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Dense scatter of ~50 shapes. My visual count said 39, so I verified with a PIL connected-component script keyed on the exact purple RGB: 40 circles, 8 diamonds, 2 squares, with a clean fill-ratio split. I trust 40; the visual pass was the weak point here.

  • spatial-simple✓ pass5s

    prompt

    Look at the image at (fetch it and view it). Which cell contains the red circle? Answer as: row R, column C (1-indexed from top-left).

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    5x5 grid; the red circle sits in the leftmost cell of the fourth row. Easy.

  • spatial-medium✓ pass1m 52s

    prompt

    Look at the image at (fetch it and view it). Which shape does the arrow from the green square lead to? Answer with just the colour and shape, e.g. teal diamond.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    This one was ambiguous. Zooming in, the green square has TWO outgoing arrows: a short one to the red triangle (bottom-left) and a longer one down-right to the green triangle, plus one incoming arrow from the teal square. The question is singular so I had to pick; I chose the longer, more prominent arrow (green triangle). I am not confident this is the intended one.

  • spatial-complex✓ pass7m 51s

    prompt

    Look at the image at (fetch it and view it). Which shape is 3 steps before the purple square along the arrows? Answer with just the colour and shape, e.g. teal diamond.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    The lower-right of this graph is a tangle of crossing arrows that I could not reliably trace by eye. I wrote a momentum-based line tracer in Python: it detected the 13 arrowheads, then followed each shaft back to its tail. That gave a unique chain green circle -> orange diamond -> orange square -> purple square, so 3 steps back is the green circle. I trust the program over my eyes here, though the tangle made it the hardest vision item.

  • chart-simple✓ pass5s

    prompt

    Look at the image at (fetch it and view it). What is the title shown at the top of this chart? Answer with just the chart title.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Read the chart title directly: Support Tickets Opened. Easy.

  • chart-medium✓ pass41s

    prompt

    Look at the image at (fetch it and view it). Using the "Server Incidents" chart, approximately what is the difference in value between Mar and Apr? Answers within +/-8 are accepted.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Server Incidents chart: Apr ~52, Mar ~29, difference ~23. I measured bar heights in pixels against the gridlines to be precise. Confident within the +/-8 tolerance.

  • chart-complex✓ pass17s

    prompt

    Look at the image at (fetch it and view it). Using the "Server Incidents" chart, approximately what value did Mobile have in Oct? Read it off the y-axis; answers within +/-3 are accepted.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Grouped bar chart; I measured the blue (Mobile) bar for Oct in pixels against the gridlines: 53.0. Within the +/-3 tolerance, confident.

  • screenshot-simple✓ pass6s

    prompt

    Look at the image at (fetch it and view it). What is the total amount shown in the cart panel? Answer with just the total amount, e.g. $12.34.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Cart total read directly: $217.52. Easy.

  • screenshot-medium✓ pass6s

    prompt

    Look at the image at (fetch it and view it). What is the total amount shown in the cart panel? Answer with just the total amount, e.g. $12.34.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Cart total read directly: $365.08. Easy.

  • screenshot-complex✓ pass9s

    prompt

    Look at the image at (fetch it and view it). What is the discount amount shown on this order summary? Answer with just the amount, e.g. $12.34.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Order summary shows Discount -$23.65; I reported the amount as $23.65. Slight ambiguity on sign, but the example format is a positive dollar amount.

  • diagram-simple✓ pass5s

    prompt

    Look at the image at (fetch it and view it). This diagram shows named boxes connected by arrows. Which box does the arrow from "Eagle" point to? Answer with just the box name, e.g. Kettle.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Simple box diagram; the arrow from Eagle goes straight down to Gecko. Easy.

  • diagram-medium✓ pass9s

    prompt

    Look at the image at (fetch it and view it). This diagram shows named boxes connected by arrows. Which box has an arrow pointing to "Pixel"? Answer with just the box name, e.g. Kettle.

    answer

    answer hidden on shared reports

    grader · score 1

    agent's debrief

    Traced the arrow into Pixel back to its source: Otter. The lower rows have several crossing arrows but Pixel has a single incoming arrow from Otter. Fairly confident.

  • diagram-complex✕ fail6m 10s

    prompt

    Look at the image at (fetch it and view it). This diagram shows named boxes connected by arrows. Which box does the arrow from "Mica" point to? Answer with just the box name, e.g. Kettle.

    answer

    answer hidden on shared reports

    grader · score 0

    expected Quokka, got "Beryl"

    agent's debrief

    Large tangled graph. Located Mica exactly, found two lines on its left edge, traced each: the lower one has its arrowhead at Mica (Aspen incoming), the upper one runs left and ends with an arrowhead pointing into Beryl. So Micas outgoing arrow points to Beryl. Verified with a momentum line tracer plus a visual crop.

Finding and reading email test

not examined · 0/6 answered

  • aggregate-1— unanswered—

    prompt

    You are examining a mailbox: Phillip Allen's mail at enronmail.airbench.ai How many messages in the sent folder have attachments? Answer with just the number.
  • aggregate-2— unanswered—

    prompt

    You are examining a mailbox: Phillip Allen's mail at enronmail.airbench.ai How many messages carry the label "attachments"? Answer with just the number.
  • temporal-1— unanswered—

    prompt

    You are examining a mailbox: Phillip Allen's mail at enronmail.airbench.ai What is the subject of the newest message in the inbox folder? Answer with just the subject line, exactly as shown.
  • temporal-2— unanswered—

    prompt

    You are examining a mailbox: Phillip Allen's mail at enronmail.airbench.ai What is the subject of the oldest message in the inbox folder? Answer with just the subject line, exactly as shown.
  • needle-1— unanswered—

    prompt

    You are examining a mailbox: Phillip Allen's mail at enronmail.airbench.ai In Phillip's reply about Bob Huntley's request for a survey of the lot, what fax number does Bob give for receiving faxed documents? Answer with just the exact fact as it appears in the message, and nothing else.
  • needle-2— unanswered—

    prompt

    You are examining a mailbox: Phillip Allen's mail at enronmail.airbench.ai In Phillip's message to gthorse@keyad.com about the Colonial Oaks property, what is the actual NOI for 2001 given for the property? Answer with just the number.

Purchasing test

not examined · 0/4 answered

  • find-product-1— unanswered—

    prompt

    The store is at abostore.airbench.ai Among products in the **Electronics** category priced under **$400** with a rating of at least **4.5**, which has the **lowest price**? The search space is every product in the store's catalog. Answer with just that product's id (the amazon.xx:B0… form).
  • find-product-2— unanswered—

    prompt

    The store is at abostore.airbench.ai Among products in the **Home & Furniture** category priced under **$25** with a rating of at least **4.5**, which has the **lowest price**? The search space is every product in the store's catalog. Answer with just that product's id (the amazon.xx:B0… form).
  • purchase-1— unanswered—

    prompt

    The store is at abostore.airbench.ai Buy exactly 3 units of SHENGHUO Watch Band, Strap 0.6 inches (16 mm), Stainless Steel, Smart Watch Band, Strap, Watch Band, 0.7 inches (16 mm), 0.7 inches (20 mm), 0.9 inches (22 mm), 0.9 inches (24 mm), Replacement Strap, Stainless Steel Metal (product id amazon.co.jp:B08BFNDPVT, abostore.airbench.ai/product/shenghuo-watch-band-stra…) on abostore, completing checkout with a valid payment method. Use the checkout email aidoctor-1cdcb1e5@aidoctor.test. Answer with just the resulting order id.
  • recover-decline-1— unanswered—

    prompt

    The store is at abostore.airbench.ai Buy exactly 1 unit of Eono Essential Pack of 2 Toys for Puppies & Dogs, Pig and Duckie Plush Dog Toy (product id amazon.co.uk:B08DF8XY6C, abostore.airbench.ai/product/eono-essential-pack-of-2…) on abostore. First attempt checkout with a card number ending in 0000 — this payment will be declined. Then retry checkout with a different, valid card and complete the purchase successfully. Use the same checkout email aidoctor-90cf9db6@aidoctor.test for both attempts. Answer with just the order id of the successful (approved) order.

Coding test

not examined · 0/11 answered

  • compute-hash-1— unanswered—

    prompt

    Write and run a short program to compute this exactly. Use unsigned 32-bit arithmetic modulo 2^32 throughout. Define rotl32(z, r) = ((z << r) | (z >> (32 - r))) mod 2^32, and let imul(a, b) mean 32-bit multiplication modulo 2^32. Start with data = [4209870487, 977592132, 3459313941, 1878544218, 372890019, 1620555968, 3979591617, 184385782, 3494029039, 196043132, 2627401389, 152029138], x = 3237632635, y = 2242131320 and rounds = 25000. For each step from 0 to rounds - 1 apply: x = (imul(x XOR data[step % 12] XOR step, 2654435761) + rotl32(y, 5)) mod 2^32; y = (imul(y + data[(step + 7) % 12] + step, 1597334677) XOR rotl32(x, 11)) mod 2^32; x = (x + rotl32(y XOR step, 3)) mod 2^32. Return the final value as two lowercase 8-digit hex words joined by a hyphen, like 1a2b3c4d-55667788.
  • compute-vm-1— unanswered—

    prompt

    Write and run a program to execute this tiny machine exactly. It has four registers a, b, c, d, all starting at 0, and a program counter starting at line 0. `set r n` sets r to n. `add r x`, `sub r x` and `mul r x` combine r with x (a register name or an integer), then reduce r modulo 1000003 into the range 0..1000002. `dec r` subtracts 1 from r. `jnz r k` jumps k lines (relative; k may be negative) if r is not 0. `halt` stops. Every other instruction moves to the next line. 0: set a 260 1: set b 338 2: set c 216 3: set d 327 4: mul a 22 5: sub a 34 6: add a 49 7: dec d 8: jnz d -4 9: mul a 54 10: dec c 11: jnz c -8 12: halt Respond with just the final value of register a.
  • compute-paths-1— unanswered—

    prompt

    Here is a grid of 25 rows and 25 columns. S is the top-left start, E is the bottom-right end, # is a wall and . is open. You may move up, down, left or right onto any non-wall cell. Find the length of the shortest path from S to E (the number of moves) and the number of distinct shortest paths modulo 1000000007. Write a program; this is not meant to be done by eye. S........#........#...... .#..#.#..##........#..#.. ##...#..#....##......#... ..##.###...#....#...#...# ........##.##...........# #....#......##...###...## ...#..#..#.#..........#.. .#.#....###...##...##.##. ...#.#......##.#.....#... ..#.#.........##..#..#.#. .....#.#..##..#..#....... ...###..##..#.#.........# #......#.#.......###...## ...##....##..........#.## ..#.......##..#.#...#.#.. ....#..#..#.#.#.......... #........#..#...#..###... .........#.#.#..#....#..# ......#...##..###.#..##.# .............#.....#.#... .......#.#.#.#...##....## #........##.#..#.....#... .#.##..#.........#......# ##.###..#.#..#........#.. ...#...##..##.#.#....#.#E Respond with the two integers separated by a space, like `52 1840`.
  • compute-life-1— unanswered—

    prompt

    Simulate Conway's Game of Life on a 20x20 grid that wraps around at every edge (a torus). # is alive and . is dead. A live cell with 2 or 3 live neighbours (out of its 8) stays alive; a dead cell with exactly 3 live neighbours becomes alive; every other cell is dead in the next generation. Starting grid, row 0 first: ..#...#####..#...##. ..##...#.#......#.## #.#.....##.####.#..# ##......#...#...#.## ..####.#....#.....## .#...##....#.......# .#.....#....#.....## ##..#..#...#.###.##. .##.#..#..#....#.#.. ..#.####...#..###... #.##.#.#...#..##...# #......#.#..#.....## #.......#.#....##.## #........#.#..##.#.. .#.#.##...#...#.##.. #...###....#..#.#... #.#####....#.#....#. ..###..###....#...#. ###.....#...#..#...# #...#...##...##.###. Run 150 generations. Report the number of live cells and the sum of row*20+column over all live cells (rows and columns numbered from 0). Respond as live:sum, like `37:7421`.
  • compute-fibmod-1— unanswered—

    prompt

    Let F(0) = 0, F(1) = 1 and F(k) = F(k-1) + F(k-2). Compute F(n) mod m exactly for n = 1024333190289326 and m = 999983. Respond with just the integer.
  • compute-words-1— unanswered—

    prompt

    Below is a text. Words are separated by spaces. Ignore letter case, and strip any punctuation or quote characters attached to a word. Count how often each word occurs, then report the 3 most frequent words, most frequent first, breaking ties alphabetically. Respond exactly as word=count,word=count,word=count. nixtru luvo Luzan basti vofic basti vozan ficnix quiren tidor rendor Luren quiren pelsha Luvo rendor kaka ficnix ZANVO quiren lusha NIXTRU shador kaka zanvo. luka QUIPEL Basti vozan vozan Vofic luzan basti. rendor SHADOR "pelsha" mobas tidor quizan, quizan luvo Nixtru vozan nixtru quidor vozan "shador" "basti" luka. vomo ficnix RENDOR truren Quipel "quiren" Vofic mobas Luzan, Vozan shador zanqui luzan. dordor renpel! rentru Renpel Dorzan? quizan zanvo rendor renpel dorzan dorzan quizan Pelmo luvo ficnix luvo "rendor" Vozan pelmo rentru pelmo Luzan pelmo quipel luren Lusha zanqui pelmo Renpel Lusha luzan dordor Rentru rentru vozan rendor Renpel mobas rendor Rendor TIDOR Quidor "lusha" tidor rendor Quipel zanqui basti pelsha luzan Tidor rendor Luvo kaka Renpel. Vomo Quizan renpel kaka nixtru nixtru. renpel PELSHA "Zanqui" truren pelmo quiren luzan zanqui quiren luren quidor tidor dordor vomo Rendor basti LUSHA VOZAN Kaka basti renpel tidor rendor rendor Luzan luvo tidor quidor quizan RENPEL kaka kaka shador truren ficnix Rentru dordor quiren Renpel. luzan vozan pelmo renpel shador zanqui; shador Ficnix rendor mobas zanqui. renpel mobas luka "pelsha" basti? lusha luzan quiren renpel! nixtru quiren "quizan" rentru dordor, pelmo pelsha vomo quizan LUZAN tidor luvo truren! Rendor zanvo Dordor tidor ficnix rentru vofic luka dorzan quiren LUSHA rendor Ficnix basti Rendor luzan vozan, renpel kaka "vozan" rentru RENDOR vozan renpel ficnix RENPEL Rendor, zanqui, Vozan dorzan! mobas kaka luvo luzan vozan dordor luren? Quizan shador truren kaka luvo luren "TRUREN" luzan LUREN vozan! Dorzan zanqui vozan luzan dordor! lusha dordor zanqui Luzan Luzan vofic; rendor Renpel dordor luvo Renpel vozan VOZAN shador; luzan basren, luzan Luzan dordor dordor quiren; rendor tidor rentru shador luzan quipel luzan vozan Shador lusha rendor vofic LUZAN shador luzan quidor ficnix "quipel" Ficnix PELSHA luka, rendor ficnix quiren kaka zanqui basren basti basti rentru zanqui quipel vozan quipel VOZAN zanqui vozan quizan Kaka luzan luka vozan "luzan" ZANVO quiren kaka rentru "Basti" Luzan Quizan luzan NIXTRU Basti! rendor basti luzan luvo quidor Tidor zanqui renpel quipel renpel quipel VOFIC pelmo pelsha luzan Kaka Zanvo "luka" renpel renpel; renpel Shador? quiren dorzan rentru, ficnix. luren Luzan Ficnix kaka Shador Pelmo truren Renpel Luzan vozan renpel Zanvo tidor luzan Kaka vomo Renpel! Quizan! dorzan basren! rendor. lusha rendor lusha nixtru "shador" Kaka Rendor zanqui VOZAN quiren quidor renpel Dordor pelsha dordor Luvo Rendor vomo LUZAN; vozan lusha tidor quidor renpel zanqui quiren truren luzan pelsha luzan DORDOR Quidor luka Luka? "kaka" luzan truren lusha luka Luzan Renpel Mobas; nixtru quipel truren Truren luzan Lusha luzan "vozan" basti PELSHA
  • trace-1— unanswered—

    prompt

    What exactly does this JavaScript program print? Respond with just the printed output. const v1 = [NaN === NaN, null >= 0, "1" == 1].map(Number).join(""); const v2 = (0.1 * 8 + 0.2 * 8 === 0.3 * 8) ? "equal" : "different"; const v3 = [typeof null, typeof (() => 1), typeof typeof 3].join("/"); const v4fns = []; for (var v4i = 0; v4i < 4; v4i++) v4fns.push(() => v4i * 2); let v4 = 0; for (const f of v4fns) v4 += f(); console.log(v1, v2, v3, v4);
  • fix-1— unanswered—

    prompt

    Our shipping quote function below has one bug. Bug report: this order is quoted 965 cents, but the correct quote is 522: {"country":"DE","items":[{"grams":1380,"qty":1,"price":5100,"fragile":false}]} Fix the bug without changing any other behaviour, then run the fixed quote() on each of the 20 orders below, in order. Respond with just the 20 results separated by commas. const ZONES = { FR: 1, DE: 1, ES: 1, IT: 1, GB: 2, US: 2, CA: 2, JP: 3, BR: 3, AU: 3 }; // any other country is zone 4 const BASE = [0, 443, 720, 1187, 1656]; // cents, by zone const PER_STEP = [0, 87, 123, 217, 284]; // cents per 250 g step, by zone const FREE_BASE_OVER = [0, 5100, 10200, 17800, Infinity]; // order value (cents) that waives the base fee function quote(order) { const zone = ZONES[order.country] ?? 4; let grams = 0; let value = 0; let fragile = 0; for (const item of order.items) { grams += item.grams * item.qty; value += item.price * item.qty; if (item.fragile) fragile += item.qty; } const steps = Math.max(1, Math.ceil(grams / 250)); let cents = PER_STEP[zone] * steps; if (value <= FREE_BASE_OVER[zone] || order.express) cents += BASE[zone]; cents += Math.min(fragile, 3) * (120 + 35 * zone); if (order.express) cents = Math.ceil((cents * (zone <= 2 ? 150 : 185)) / 100); if (order.coupon === "SHIP10") cents -= Math.min(cents >> 3, 500); return Math.max(cents, 99); } Orders: {"country":"CA","items":[{"grams":511,"qty":2,"price":2335,"fragile":false},{"grams":1446,"qty":5,"price":3917,"fragile":false},{"grams":1360,"qty":1,"price":5343,"fragile":true}],"express":true} {"country":"BR","items":[{"grams":987,"qty":5,"price":4047,"fragile":false},{"grams":242,"qty":3,"price":7780,"fragile":true},{"grams":1018,"qty":3,"price":2115,"fragile":false}]} {"country":"US","items":[{"grams":363,"qty":1,"price":10200,"fragile":false}]} {"country":"IT","items":[{"grams":1151,"qty":1,"price":5100,"fragile":false}]} {"country":"IT","items":[{"grams":1932,"qty":1,"price":5100,"fragile":false}]} {"country":"IT","items":[{"grams":747,"qty":2,"price":6739,"fragile":false},{"grams":470,"qty":1,"price":4434,"fragile":false},{"grams":1135,"qty":2,"price":6964,"fragile":false},{"grams":930,"qty":5,"price":8715,"fragile":false}]} {"country":"ES","items":[{"grams":1473,"qty":2,"price":2375,"fragile":false}],"coupon":"SHIP10"} {"country":"IT","items":[{"grams":1685,"qty":1,"price":7096,"fragile":false}]} {"country":"IT","items":[{"grams":526,"qty":1,"price":5100,"fragile":false}]} {"country":"ES","items":[{"grams":1554,"qty":5,"price":4396,"fragile":true}],"coupon":"SHIP10"} {"country":"IT","items":[{"grams":111,"qty":1,"price":7802,"fragile":false},{"grams":1391,"qty":1,"price":3582,"fragile":true},{"grams":81,"qty":3,"price":4936,"fragile":false}]} {"country":"JP","items":[{"grams":815,"qty":1,"price":8776,"fragile":false},{"grams":475,"qty":4,"price":1984,"fragile":false},{"grams":163,"qty":1,"price":5998,"fragile":false},{"grams":675,"qty":1,"price":8913,"fragile":false}],"coupon":"SHIP10"} {"country":"IT","items":[{"grams":363,"qty":1,"price":5100,"fragile":false}]} {"country":"AU","items":[{"grams":572,"qty":1,"price":17800,"fragile":false}]} {"country":"JP","items":[{"grams":397,"qty":1,"price":17800,"fragile":false}]} {"country":"GB","items":[{"grams":84,"qty":1,"price":1803,"fragile":false},{"grams":1770,"qty":5,"price":2691,"fragile":false}],"express":true} {"country":"IT","items":[{"grams":1377,"qty":1,"price":6752,"fragile":false},{"grams":826,"qty":3,"price":2379,"fragile":false}]} {"country":"CA","items":[{"grams":1384,"qty":2,"price":1031,"fragile":false},{"grams":1538,"qty":4,"price":7819,"fragile":false},{"grams":1592,"qty":2,"price":6991,"fragile":false},{"grams":800,"qty":4,"price":6090,"fragile":false}],"express":true} {"country":"IT","items":[{"grams":1209,"qty":3,"price":6535,"fragile":false}]} {"country":"DE","items":[{"grams":1380,"qty":3,"price":3688,"fragile":true},{"grams":889,"qty":1,"price":5739,"fragile":true},{"grams":821,"qty":5,"price":1576,"fragile":false}]}
  • implement-1— unanswered—

    prompt

    Write a function mergeIntervals(intervals). Each interval is a pair [start, end] of integers with start <= end, and includes both ends. It returns the intervals with every overlapping or touching pair merged (so [1, 3] and [3, 5] become [1, 5], but [1, 2] and [3, 4] stay apart), sorted by start. The input may be in any order. Run it on each of the 12 inputs below and respond with one line per input: the result as JSON, like [[1,5],[8,10]]. Spaces are ignored. [] [[1,3],[3,5]] [[1,2],[3,4]] [[5,9],[1,10]] [[28,32],[32,34],[34,40],[32,37],[31,33],[6,11],[26,31],[22,27]] [[33,37],[33,33],[12,12],[26,27],[20,28],[36,44],[39,45]] [[11,18],[16,22],[8,14],[36,36],[29,31],[36,36],[21,29]] [[36,39],[6,11],[13,16],[10,10],[36,43]] [[24,26],[11,14],[26,32],[16,16],[7,9]] [[22,23],[33,33],[36,40],[2,9],[21,23],[32,39],[15,17]] [[11,11],[22,25],[16,20],[27,31],[16,21],[37,45],[12,12],[0,4]] [[7,10],[39,46],[27,34],[22,24],[17,23],[39,42]]
  • repo-1— unanswered—

    prompt

    Download airbench.ai/f/30e1fb954ef294a77cb1c431def5da25.zip and unzip it. It contains a small Python project (Python 3.10 or newer, standard library only) with a bug. Find and fix it, then run `python main.py data/real.csv` inside the project folder and respond with just the 8-character code it prints.
  • repo-2— unanswered—

    prompt

    Download airbench.ai/f/0587cb623c356d3bb5726d064b440b03.zip and unzip it. It contains a small Python project (Python 3.10 or newer, standard library only) with bugs. Find and fix them, then run `python main.py data/real.csv` inside the project folder and respond with just the 8-character code it prints.

how this agent was configured

Hardware: NVIDIA RTX 5090 32 GB (SM120), x86_64, driver 595.84, Ubuntu 24.04 (host hal5090). Model server: Checkpoint ukisai/Swift-1.5-Qwen3.8-27B-GGUF Q6_K (22.2 GiB) + mmproj F16, fully on the RTX 5090. llama.cpp server build 11243 (ghcr.io/ggml-org/llama.cpp:server-cuda @ sha256:1c568d22): -c 131072 -np 1 -fa on -ctk q8_0 -ctv q8_0 --jinja --reasoning-format auto --spec-type draft-mtp --spec-draft-n-max 3 -fitt 2048 (the GGUF's built-in MTP head). ~70 tok/s decode short, 54-65 at 60k (vs 60 / 51 plain). Harness: pi 0.73.1 (@mariozechner/pi-coding-agent) in a container (node:22): `pi -p --mode json <prompt>`; per-run PI_CODING_AGENT_DIR models.json with compat supportsDeveloperRole=false, supportsReasoningEffort=false; context 131072, max output 32768 tokens; settings compaction.reserveTokens=49152, keepRecentTokens=20000. Orchestrator: github.com/dh7/agent-checkup-benchmark @ d6ced6f; the checkup instructions are passed verbatim as a single prompt, one attempt, no time limit unless noted.