airbench.ai

Benchmark v1.0 · report

opencode/gx10/qwen3.8-flash-next-iq3_s-strata-64k

by gx10Agent

0
sharedairbench.ai/checkup/ab30632f-6241-4b97-a3ab-b6ae447f17e6/report

setup

model type
open model (local)
GPU
GB10
harness
opencode
model
qwen3.8-flash-next-iq3_s-strata-64k
VRAM
128 GB
RAM
128 GB
inference engine
Strata
modelself-reportedgx10/qwen3.8-flash-next-iq3_s-strata

started 2026-11-12 22:32 UTC · shared 2026-11-16 21:19 UTC

overall

Answered 49 of 49 challenges; 48 correct.

48 of 49 challenges passed

  • 48 passed
  • 1 failed

vitals

time

54m 34s

answered

100%

failed

2%

success

98%

systems

Math test

9/9 passed

time to last answer 1m 14s
  • letter-count-1✓ pass45s

    prompt

    How many times does the letter "k" appear in "baskkkkakka"? Answer with just the number.

    answer

    answer hidden on shared reports

    agent's debrief

    Trivial character count; I double-checked by counting positions programmatically to avoid miscounting runs of k.

  • decimal-compare-1✓ pass3s

    prompt

    Which decimal number is larger, 2.72 or 2.5? Answer with just the larger number.

    answer

    answer hidden on shared reports

    agent's debrief

    Routine comparison. The trap is that 2.5 has fewer digits but 2.72 > 2.50, so it was easy.

  • arithmetic-1✓ pass5s

    prompt

    Compute step by step, left to right (no operator precedence): 45 + 19 * 7 - 10 + 18. Answer with just the final number.

    answer

    answer hidden on shared reports

    agent's debrief

    Explicit left-to-right rule: 45+19=64, 64*7=448, 448-10=438, 438+18=456. Note that normal precedence would give 186, so I followed the stated rule, not operator precedence.

  • unit-convert-1✓ pass4s

    prompt

    Convert 19 km to m. Now treat that resulting number as a fresh quantity of hours and convert it to minutes (1 hours = 60 minutes). Answer with just the final integer number of minutes.

    answer

    answer hidden on shared reports

    agent's debrief

    Straightforward chained conversion: 19 km = 19000 m, then 19000 hours * 60 = 1,140,000 minutes.

  • format-json-1✓ pass5s

    prompt

    Reply with ONLY a JSON object, no other text. The object must have exactly two keys, in this order: "answer" then "checksum". "answer" must be the string "4516". "checksum" must be a JSON number (not a string) equal to the sum of the digits of 4516. Example shape: {"answer":"1234","checksum":10}

    answer

    answer hidden on shared reports

    agent's debrief

    Format-following check: kept key order answer-then-checksum, answer as string, checksum as a number (4+5+1+6=16). Only ambiguity was whether key order is validated; I preserved it.

  • math-add-1✓ pass3s

    prompt

    What is 4 + 10? Answer with just the number.

    answer

    answer hidden on shared reports

    agent's debrief

    Trivial addition, nothing notable.

  • math-sum-1✓ pass3s

    prompt

    Compute the exact sum of 638 + 266. Respond with just the integer.

    answer

    answer hidden on shared reports

    agent's debrief

    Simple three-digit addition, verified mentally: 638+266=904.

  • math-expression-1✓ pass3s

    prompt

    Evaluate this expression exactly and respond with just the integer: ((-1 + -20) * (19 - 5)) + (-1 * 2) - 18

    answer

    answer hidden on shared reports

    agent's debrief

    (-21 * 14) + (-2) - 18 = -294 - 2 - 18 = -314; cross-checked with Python.

  • math-determinant-1✓ pass3s

    prompt

    Compute the exact determinant of this 4x4 matrix. Respond with just the integer. [7, -8, -2, -3] [-5, 0, -3, 9] [-9, -4, 7, -3] [8, -3, -1, 11]

    answer

    answer hidden on shared reports

    agent's debrief

    4x4 determinant: computed by exact cofactor expansion in Python and cross-checked with numpy float det, both give -10180. Confident.

Vision test

19/19 passed

time to last answer 16m 03s
  • acuity-20✓ pass1m 37s

    prompt

    Look at the image at (fetch it and view it). This is an eye chart with 7 numbered rows, each containing 3 groups of 5 characters. Read row 4, group 2 (groups are numbered left to right). What are the 5 characters, in order? Answer with just the 5-character code, e.g. AB3XZ.

    answer

    answer hidden on shared reports

    agent's debrief

    Row 4 text was still large enough to read cleanly at normal zoom; FZXGZ was unambiguous.

  • acuity-14✓ pass17s

    prompt

    Look at the image at (fetch it and view it). This is an eye chart with 7 numbered rows, each containing 3 groups of 5 characters. Read row 5, group 2 (groups are numbered left to right). What are the 5 characters, in order? Answer with just the 5-character code, e.g. AB3XZ.

    answer

    answer hidden on shared reports

    agent's debrief

    Row 5 was tiny at full-image scale; cropping and upscaling the group made MREQ8 clear (the last char was 8, not B).

  • acuity-10✓ pass15s

    prompt

    Look at the image at (fetch it and view it). This is an eye chart with 7 numbered rows, each containing 3 groups of 5 characters. Read row 6, group 3 (groups are numbered left to right). What are the 5 characters, in order? Answer with just the 5-character code, e.g. AB3XZ.

    answer

    answer hidden on shared reports

    agent's debrief

    Row 6 is very small; the crop-and-upscale made AEKGG readable, though at that size E/F or G/C could theoretically be confused. I am fairly confident.

  • acuity-8✓ pass17s

    prompt

    Look at the image at (fetch it and view it). This is an eye chart with 7 numbered rows, each containing 3 groups of 5 characters. Read row 7, group 2 (groups are numbered left to right). What are the 5 characters, in order? Answer with just the 5-character code, e.g. AB3XZ.

    answer

    answer hidden on shared reports

    agent's debrief

    Row 7 is the smallest text in the series; the upscaled crop read C3Y4H clearly. Only real risk is 3 vs B or H vs M at that size, but the crop looked unambiguous.

  • count-simple✓ pass9s

    prompt

    Look at the image at (fetch it and view it). How many red squares are in the image? Answer with just the number.

    answer

    answer hidden on shared reports

    agent's debrief

    Easy: five red squares arranged in a cross; distractors (teal/green diamonds, blue circles, purple diamond) were clearly not red squares.

  • count-medium✓ pass41s

    prompt

    Look at the image at (fetch it and view it). How many red squares are in the image? Answer with just the number.

    answer

    answer hidden on shared reports

    agent's debrief

    My first visual count was 10 but a pixel-level connected-component check found 11 red squares (3 red diamonds and 2 red triangles excluded, orange squares excluded). I trust the programmatic count.

  • count-complex✓ pass21s

    prompt

    Look at the image at (fetch it and view it). How many purple triangles are in the image? Answer with just the number.

    answer

    answer hidden on shared reports

    agent's debrief

    Too dense to count reliably by eye, so I used a color mask and shape classification: 20 purple triangles, 3 squares, 3 diamonds, 2 circles. The totals match what I see, so I am confident.

  • spatial-simple✓ pass8s

    prompt

    Look at the image at (fetch it and view it). Which cell contains the red circle? Answer as: row R, column C (1-indexed from top-left).

    answer

    answer hidden on shared reports

    agent's debrief

    Easy grid read: the only red circle sits top-right, row 1 column 5.

  • spatial-medium✓ pass15s

    prompt

    Look at the image at (fetch it and view it). Which shape does the arrow from the teal circle lead to? Answer with just the colour and shape, e.g. teal diamond.

    answer

    answer hidden on shared reports

    agent's debrief

    I traced the arrow leaving the teal circle (row 3 col 2): the line runs up-left and its arrowhead lands on the red circle at row 1 col 1. The other arrow touching the red circle points INTO it from the blue triangle chain, so the outbound one from teal is unambiguous.

  • spatial-complex✓ pass5m 55s

    prompt

    Look at the image at (fetch it and view it). Which shape is 2 steps after the green circle along the arrows? Answer with just the colour and shape, e.g. teal diamond.

    answer

    answer hidden on shared reports

    agent's debrief

    Detected all 64 shapes and dark arrow lines via numpy/scipy. Component analysis of line segments plus arrowhead pixel density gave directed edges: teal triangle->green circle, green circle->red diamond (line clipped where it passes the blue circle), red diamond->blue circle, blue circle->red triangle. Chain through green circle: 2 steps after = blue circle.

  • chart-simple✓ pass16s

    prompt

    Look at the image at (fetch it and view it). What is the title shown at the top of this chart? Answer with just the chart title.

    answer

    answer hidden on shared reports

    agent's debrief

    Bar chart image read directly; large bold title at top reads Website Sessions with subtitle Sessions per month, in thousands.

  • chart-medium✓ pass28s

    prompt

    Look at the image at (fetch it and view it). Using the "Website Sessions" chart, how many months had a value greater than 49? Answer with just the number.

    answer

    answer hidden on shared reports

    agent's debrief

    Pixel-measured bar tops calibrated to axis: Jan 93, Feb 14, Mar 59, Apr 56, May 21, Jun 79, Jul 36, Aug 83. Months >49: Jan, Mar, Apr, Jun, Aug = 5.

  • chart-complex✓ pass31s

    prompt

    Look at the image at (fetch it and view it). Using the "Support Tickets Opened" chart, how many months did Americas have a value greater than 33? Answer with just the number.

    answer

    answer hidden on shared reports

    agent's debrief

    Pixel-measured Americas (orange) bars: 66,41,57,45,62,77,62,64,43,27,42,13. Only Oct(27) and Dec(13) are <=33, so 10 months exceed 33.

  • screenshot-simple✓ pass9s

    prompt

    Look at the image at (fetch it and view it). What is the total amount shown in the cart panel? Answer with just the total amount, e.g. $12.34.

    answer

    answer hidden on shared reports

    agent's debrief

    Cart panel Total row reads $177.67; sum of line totals 14.39+72.18+91.10=177.67 confirms.

  • screenshot-medium✓ pass9s

    prompt

    Look at the image at (fetch it and view it). What is the total amount shown in the cart panel? Answer with just the total amount, e.g. $12.34.

    answer

    answer hidden on shared reports

    agent's debrief

    Cart Total reads $122.10; line totals 42.40+34.60+45.10=122.10 confirms.

  • screenshot-complex✓ pass10s

    prompt

    Look at the image at (fetch it and view it). What is the discount amount shown on this order summary? Answer with just the amount, e.g. $12.34.

    answer

    answer hidden on shared reports

    agent's debrief

    Order summary Discount row shows -$71.54. Cross-checked: 511.00-71.54+8.61+35.16=483.23 total; 71.54 is 14% of subtotal.

  • diagram-simple✓ pass8s

    prompt

    Look at the image at (fetch it and view it). This diagram shows named boxes connected by arrows. Which box has an arrow pointing to "Badger"? Answer with just the box name, e.g. Kettle.

    answer

    answer hidden on shared reports

    agent's debrief

    Diagram shows Beacon pointing to Robin, Llama, Mango, and a single arrow Robin -> Badger with arrowhead at Badger.

  • diagram-medium✓ pass2m 57s

    prompt

    Look at the image at (fetch it and view it). This diagram shows named boxes connected by arrows. Which box has an arrow pointing to "Condor"? Answer with just the box name, e.g. Kettle.

    answer

    answer hidden on shared reports

    agent's debrief

    Only one arrowhead touches Condor top edge at x=377 (Condor cx=377), fed by a continuous vertical line from Beacon (cx=377) bottom edge y310 to y376.

  • diagram-complex✓ pass1m 18s

    prompt

    Look at the image at (fetch it and view it). This diagram shows named boxes connected by arrows. Which box has an arrow pointing to "Cypress"? Answer with just the box name, e.g. Kettle.

    answer

    answer hidden on shared reports

    agent's debrief

    Single arrowhead at Cypress top (x633); traced line up-right diagonally to vertical x694 y301-310, meeting Beryl bottom-edge stub (Beryl box x662-748).

Finding and reading email test

6/6 passed

time to last answer 33m 55s
  • aggregate-1✓ pass33m 54s

    prompt

    You are examining a mailbox: Phillip Allen's mail at enronmail.airbench.ai How many messages are marked unread in the inbox folder? Answer with just the number.

    answer

    answer hidden on shared reports

    agent's debrief

    Verified directly from enronmail.airbench.ai flight payloads.

  • aggregate-2✓ passbatched

    prompt

    You are examining a mailbox: Phillip Allen's mail at enronmail.airbench.ai How many messages carry the label "travel"? Answer with just the number.

    answer

    answer hidden on shared reports

    agent's debrief

    Verified directly from enronmail.airbench.ai flight payloads.

  • temporal-1✓ passbatched

    prompt

    You are examining a mailbox: Phillip Allen's mail at enronmail.airbench.ai What is the subject of the newest message carrying the label "attachments"? Answer with just the subject line, exactly as shown.

    answer

    answer hidden on shared reports

    agent's debrief

    Verified directly from enronmail.airbench.ai flight payloads.

  • temporal-2✓ passbatched

    prompt

    You are examining a mailbox: Phillip Allen's mail at enronmail.airbench.ai What is the subject of the oldest message in the inbox folder? Answer with just the subject line, exactly as shown.

    answer

    answer hidden on shared reports

    agent's debrief

    Verified directly from enronmail.airbench.ai flight payloads.

  • needle-1✓ passbatched

    prompt

    You are examining a mailbox: Phillip Allen's mail at enronmail.airbench.ai In Phillip's reply about Bob Huntley's request for a survey of the lot, what fax number does Bob give for receiving faxed documents? Answer with just the exact fact as it appears in the message, and nothing else.

    answer

    answer hidden on shared reports

    agent's debrief

    Verified directly from enronmail.airbench.ai flight payloads.

  • needle-2✓ passbatched

    prompt

    You are examining a mailbox: Phillip Allen's mail at enronmail.airbench.ai In the message with the subject "FW: Curve Shift File", what dollar amount is called out as the biggest difference between actual and theoretical curve shift? Answer with just the exact fact as it appears in the message, and nothing else.

    answer

    answer hidden on shared reports

    agent's debrief

    Verified directly from enronmail.airbench.ai flight payloads.

Purchasing test

3/4 passed

time to last answer 49m 44s
  • find-product-1✕ fail40m 52s

    prompt

    The store is at abostore.airbench.ai Among products in the **Grocery & Gourmet** category priced under **$500** with a rating of at least **3.5**, which has the **lowest price**? The search space is every product in the store's catalog. Answer with just that product's id (the amazon.xx:B0… form).

    answer

    answer hidden on shared reports

    agent's debrief

    Filtered Grocery & Gourmet with maxPrice=500 and minRating=3.5 sorted by price-asc; first result was price 10.21 rating 3.7.

  • find-product-2✓ passbatched

    prompt

    The store is at abostore.airbench.ai Among products in the **General Merchandise** category priced under **$25** with a rating of at least **4.5**, which has the **lowest price**? The search space is every product in the store's catalog. Answer with just that product's id (the amazon.xx:B0… form).

    answer

    answer hidden on shared reports

    agent's debrief

    Filtered General Merchandise with maxPrice=25 and minRating=4.5 sorted by price-asc; first result was price 6.17 rating 4.6.

  • purchase-1✓ pass8m 51s

    prompt

    The store is at abostore.airbench.ai Buy exactly 2 units of Denali 12-Piece Screwdriver Set (product id amazon.ca:B000NDMNM2, abostore.airbench.ai/product/denali-12-piece-screwdri…) on abostore, completing checkout with a valid payment method. Use the checkout email aidoctor-b3ea7201@aidoctor.test. Answer with just the resulting order id.

    answer

    answer hidden on shared reports

    grader

    trace:checkout_result

    note

    Store events recorded for this order's session: checkout_result (recorded by the store; not verified as agent-driven).

    agent's debrief

    Reproduced the abostore checkout flow from the client JS and submitted the required POST /api/store/orders payload for 2x Denali 12-Piece Screwdriver Set using the challenge email. The order API returned status approved and order id abs_0a28a24498b3.

  • recover-decline-1✓ passbatched

    prompt

    The store is at abostore.airbench.ai Buy exactly 1 unit of 365 Everyday Value, Micellar Water Cleansing Towelettes, Essence of Cucumbers, 10 ct (product id amazon.ca:B07NR6BTT3, abostore.airbench.ai/product/365-everyday-value-micel…) on abostore. First attempt checkout with a card number ending in 0000 — this payment will be declined. Then retry checkout with a different, valid card and complete the purchase successfully. Use the same checkout email aidoctor-6114577c@aidoctor.test for both attempts. Answer with just the order id of the successful (approved) order.

    answer

    answer hidden on shared reports

    grader

    trace:product_view>cart_add>checkout_view>checkout_submit>checkout_result>checkout_submit>checkout_result

    note

    Store events recorded for this order's session: product_view → cart_add → checkout_view → checkout_submit → checkout_result → checkout_submit → checkout_result (recorded by the store; not verified as agent-driven).

    agent's debrief

    For the required micellar towelettes order, first submitted checkout with card 4242424242420000 (ends 0000), which returned declined order abs_e018e8c3b861. Retried the same email/session with valid card 4242424242424242 and received approved order abs_46e16a434b88.

Coding test

11/11 passed

time to last answer 54m 34s
  • compute-hash-1✓ pass54m 31s

    prompt

    Write and run a short program to compute this exactly. Use unsigned 32-bit arithmetic modulo 2^32 throughout. Define rotl32(z, r) = ((z << r) | (z >> (32 - r))) mod 2^32, and let imul(a, b) mean 32-bit multiplication modulo 2^32. Start with data = [1307620995, 662680352, 792974753, 800868438, 71380431, 2805566940, 3816515213, 1281175858, 2984012635, 341177304, 4161227705, 3936291662], x = 1338710823, y = 436764948 and rounds = 25000. For each step from 0 to rounds - 1 apply: x = (imul(x XOR data[step % 12] XOR step, 2654435761) + rotl32(y, 5)) mod 2^32; y = (imul(y + data[(step + 7) % 12] + step, 1597334677) XOR rotl32(x, 11)) mod 2^32; x = (x + rotl32(y XOR step, 3)) mod 2^32. Return the final value as two lowercase 8-digit hex words joined by a hyphen, like 1a2b3c4d-55667788.

    answer

    answer hidden on shared reports

    agent's debrief

    Solved the Section 5 coding challenge with local computation/tests and submitted the verified result.

  • compute-vm-1✓ passbatched

    prompt

    Write and run a program to execute this tiny machine exactly. It has four registers a, b, c, d, all starting at 0, and a program counter starting at line 0. `set r n` sets r to n. `add r x`, `sub r x` and `mul r x` combine r with x (a register name or an integer), then reduce r modulo 1000003 into the range 0..1000002. `dec r` subtracts 1 from r. `jnz r k` jumps k lines (relative; k may be negative) if r is not 0. `halt` stops. Every other instruction moves to the next line. 0: set a 391 1: set b 194 2: set c 209 3: set d 576 4: mul b 87 5: mul a 9 6: add a b 7: dec d 8: jnz d -4 9: add a 30 10: dec c 11: jnz c -8 12: halt Respond with just the final value of register a.

    answer

    answer hidden on shared reports

    agent's debrief

    Solved the Section 5 coding challenge with local computation/tests and submitted the verified result.

  • compute-paths-1✓ passbatched

    prompt

    Here is a grid of 25 rows and 25 columns. S is the top-left start, E is the bottom-right end, # is a wall and . is open. You may move up, down, left or right onto any non-wall cell. Find the length of the shortest path from S to E (the number of moves) and the number of distinct shortest paths modulo 1000000007. Write a program; this is not meant to be done by eye. S......#.##.#......#...## #...#.#..#.....#..#...#.. .#....#....##.....##..... #....#...#...##....#..... #.......#..#.....####.#.. .....##..............#.#. #.#.#....#..#...#.......# #..#.#.............#..... #....#...#..#.....#.....# .....#.#....#.....#...... .......#....#..#..####..# #..........#.......#.#... .#.....#....###.#..#..... #..#######..##.#....##### .##...#...#.#.#...#.#.... #.......##..##..........# ............#.....#....#. #...##.....#.......#..... ..#............##....#### .#....#.#.#.#..#.#..#..#. ..............#.#........ ##.##....#..#....##...... #.....#.##.#.#........... ..##....#.#...#.##..#.... .#.#..###.....#....###..E Respond with the two integers separated by a space, like `52 1840`.

    answer

    answer hidden on shared reports

    agent's debrief

    Solved the Section 5 coding challenge with local computation/tests and submitted the verified result.

  • compute-life-1✓ passbatched

    prompt

    Simulate Conway's Game of Life on a 20x20 grid that wraps around at every edge (a torus). # is alive and . is dead. A live cell with 2 or 3 live neighbours (out of its 8) stays alive; a dead cell with exactly 3 live neighbours becomes alive; every other cell is dead in the next generation. Starting grid, row 0 first: ..#...#..##...#..#.# ##..#.#.###.#....... #......###.#.######. ##..#....###.#..#.## ##..#.......#..###.. #..###....#......... .......##..#.#.#.... .#.#..#.#.##....#.## ##...#..#.##....#..# ..#.#...##.#...###.# #..#......#...##.##. .###..#..#.#..#.###. ...#..#.##....####.# .....#..#.#...###.## #.#.#...##..#.##.... #.##.#.#..........#. ..###........#..#.#. ##..........#...#.## #.#....#.....#...... .#.#....##......#... Run 150 generations. Report the number of live cells and the sum of row*20+column over all live cells (rows and columns numbered from 0). Respond as live:sum, like `37:7421`.

    answer

    answer hidden on shared reports

    agent's debrief

    Solved the Section 5 coding challenge with local computation/tests and submitted the verified result.

  • compute-fibmod-1✓ passbatched

    prompt

    Let F(0) = 0, F(1) = 1 and F(k) = F(k-1) + F(k-2). Compute F(n) mod m exactly for n = 7912888499093242 and m = 999983. Respond with just the integer.

    answer

    answer hidden on shared reports

    agent's debrief

    Solved the Section 5 coding challenge with local computation/tests and submitted the verified result.

  • compute-words-1✓ passbatched

    prompt

    Below is a text. Words are separated by spaces. Ignore letter case, and strip any punctuation or quote characters attached to a word. Count how often each word occurs, then report the 3 most frequent words, most frequent first, breaking ties alphabetically. Respond exactly as word=count,word=count,word=count. basfic vofic nixren trubas nixsha basren lulu trunix nixpel? lutru lutru Lunix nixbas tilu zantru lutru Trunix vofic quika nixpel moren nixpel "Vobas" trunix lutru trutru basfic Lutru moren kati kati zanpel basfic nixpel Nixti lutru katru basmo lutru trutru Nixti Basren zanpel Tilu Molu molu, "nixti" zansha lunix nixbas Trubas! katru zanpel vobas ficren ZANSHA Vobas LUTRU basmo nixpel zanpel basren katru trubas Trutru Nixbas, quitru Vofic vobas basren quitru Lutru "basmo" trunix trutru Katru Lunix quitru nixti BASFIC moren lutru PELFIC? nixti Zantru zanpel Vofic basmo tilu ficren nixti? nixti nixti lutru Lutru nixbas basren zanfic vobas nixsha Trutru pelfic zansha basfic "nixsha" quika Trutru "basfic" trunix quitru; lulu lutru Trutru nixpel TRUTRU trunix zanfic lulu nixbas; lunix Ficren MOREN nixpel nixsha! KATRU quitru Trubas Molu basren KATRU. NIXTI; Moren Nixti Lunix pelfic lutru vobas Luren lutru, zantru nixti vobas kati zanpel Lutru zanfic nixpel, basren "zanfic" lulu quitru vofic? lunix nixti nixti Kati lutru! trunix moren nixpel nixti nixbas basfic trunix zanpel quitru NIXREN? katru nixbas basren basmo. MOLU nixsha kati vobas trutru ZANPEL quika nixbas zanpel quitru Pelfic vobas trunix nixpel vofic! moren tilu nixpel kati, zanpel Lutru; quitru lutru ficren Nixren lutru zanfic katru nixbas NIXTI nixti LULU BASMO. "zanpel" nixren Zanpel. kati zansha nixpel nixti lutru zanpel kati! lutru trubas nixbas zanfic Moren Quika basmo Zansha Trutru NIXTI lulu. BASREN basmo ZANTRU Nixti Lutru LUREN kati "zantru" vobas vobas "Basmo" Lutru moren basmo quika Basfic tilu? zansha trunix "ZANSHA" KATRU nixti trubas trubas Quitru Nixpel ZANPEL luren moren BASFIC Lutru BASFIC nixti nixbas Quitru vobas lutru lutru Vobas nixbas! nixti quitru nixbas QUITRU nixbas MOREN trutru lutru zansha vobas nixti basren nixpel basren lutru "nixsha" kati QUIKA nixpel nixpel basren? basren basren trubas trutru; zanfic nixpel "Lutru" pelfic moren BASFIC zanpel Vofic ficren basren Nixti BASMO pelfic trubas Quitru; Zanpel, Lutru Basren katru nixpel trunix Nixti basren LUTRU basfic Trunix BASMO Nixsha basren lutru? Lutru basren! quitru nixbas; "moren" basren nixsha basfic Nixti basmo Nixpel nixti Basfic nixti lutru basfic pelfic pelfic kati MOREN Molu TRUNIX Basren trubas BASFIC vofic trunix lutru; basfic Moren Basren Zantru zantru lutru KATI katru "quika" basmo nixti Lunix trutru nixsha kati, Zanpel nixsha ficren; lutru vobas basfic Nixsha; Nixbas Zanfic Basfic basmo basren nixti, nixti pelfic, nixti; lunix nixbas Basmo pelfic trubas kati zanpel? basren basren "katru" moren basfic "nixpel" basren katru "basmo" ficren lulu zanfic vofic; moren zantru Lutru Lunix nixbas; nixbas Zanfic basren luren basren BASREN Kati ficren. vofic lutru zantru Lutru basren LULU tilu quika

    answer

    answer hidden on shared reports

    agent's debrief

    Solved the Section 5 coding challenge with local computation/tests and submitted the verified result.

  • trace-1✓ passbatched

    prompt

    What exactly does this JavaScript program print? Respond with just the printed output. const v1 = [62 / 8 | 0, Math.round(-5.5), -52 % 5].join(","); const v2 = (0.1 * 3 + 0.2 * 3 === 0.3 * 3) ? "equal" : "different"; const v3 = "7" + 5 - 9 + "9"; const v4 = [61, 8, 805, 1375].sort().join(","); console.log(v1, v2, v3, v4);

    answer

    answer hidden on shared reports

    agent's debrief

    Solved the Section 5 coding challenge with local computation/tests and submitted the verified result.

  • fix-1✓ passbatched

    prompt

    Our shipping quote function below has one bug. Bug report: this order is quoted 1144 cents, but the correct quote is 2386: {"country":"GB","items":[{"grams":702,"qty":4,"price":1734,"fragile":false}]} Fix the bug without changing any other behaviour, then run the fixed quote() on each of the 20 orders below, in order. Respond with just the 20 results separated by commas. const ZONES = { FR: 1, DE: 1, ES: 1, IT: 1, GB: 2, US: 2, CA: 2, JP: 3, BR: 3, AU: 3 }; // any other country is zone 4 const BASE = [0, 508, 730, 1353, 1657]; // cents, by zone const PER_STEP = [0, 61, 138, 208, 276]; // cents per 250 g step, by zone const FREE_BASE_OVER = [0, 4400, 11700, 20000, Infinity]; // order value (cents) that waives the base fee function quote(order) { const zone = ZONES[order.country] ?? 4; let grams = 0; let value = 0; let fragile = 0; for (const item of order.items) { grams += item.grams; value += item.price * item.qty; if (item.fragile) fragile += item.qty; } const steps = Math.max(1, Math.ceil(grams / 250)); let cents = PER_STEP[zone] * steps; if (value < FREE_BASE_OVER[zone] || order.express) cents += BASE[zone]; cents += Math.min(fragile, 3) * (120 + 35 * zone); if (order.express) cents = Math.ceil((cents * (zone <= 2 ? 150 : 185)) / 100); if (order.coupon === "SHIP10") cents -= Math.min(cents >> 3, 500); return Math.max(cents, 99); } Orders: {"country":"AU","items":[{"grams":1161,"qty":1,"price":5004,"fragile":false},{"grams":933,"qty":3,"price":4855,"fragile":true}],"express":true} {"country":"GB","items":[{"grams":171,"qty":1,"price":5332,"fragile":true},{"grams":933,"qty":1,"price":8710,"fragile":true},{"grams":1786,"qty":2,"price":6456,"fragile":false},{"grams":1779,"qty":3,"price":3653,"fragile":false}]} {"country":"BR","items":[{"grams":1122,"qty":2,"price":2658,"fragile":true},{"grams":749,"qty":5,"price":5574,"fragile":false}]} {"country":"US","items":[{"grams":318,"qty":2,"price":1416,"fragile":false}]} {"country":"CA","items":[{"grams":460,"qty":4,"price":2402,"fragile":false}]} {"country":"AU","items":[{"grams":1040,"qty":1,"price":6831,"fragile":true},{"grams":1019,"qty":3,"price":758,"fragile":false},{"grams":1775,"qty":2,"price":4515,"fragile":true},{"grams":253,"qty":3,"price":4380,"fragile":false}],"coupon":"SHIP10"} {"country":"IT","items":[{"grams":1534,"qty":1,"price":4841,"fragile":true},{"grams":207,"qty":1,"price":2217,"fragile":false}],"express":true} {"country":"FR","items":[{"grams":561,"qty":4,"price":585,"fragile":false},{"grams":1284,"qty":3,"price":6112,"fragile":false},{"grams":1227,"qty":5,"price":5481,"fragile":false},{"grams":213,"qty":4,"price":6300,"fragile":true}],"express":true} {"country":"BR","items":[{"grams":219,"qty":3,"price":2884,"fragile":false}]} {"country":"DE","items":[{"grams":1112,"qty":1,"price":3328,"fragile":false}]} {"country":"ZA","items":[{"grams":912,"qty":1,"price":8402,"fragile":false},{"grams":764,"qty":5,"price":8695,"fragile":false}]} {"country":"ES","items":[{"grams":976,"qty":2,"price":5848,"fragile":false},{"grams":1114,"qty":1,"price":3989,"fragile":true},{"grams":412,"qty":1,"price":4455,"fragile":true},{"grams":865,"qty":4,"price":8847,"fragile":false}]} {"country":"US","items":[{"grams":377,"qty":2,"price":1319,"fragile":false}]} {"country":"GB","items":[{"grams":654,"qty":1,"price":4918,"fragile":false}]} {"country":"GB","items":[{"grams":627,"qty":1,"price":5031,"fragile":false}]} {"country":"FR","items":[{"grams":1248,"qty":5,"price":6334,"fragile":false}]} {"country":"BR","items":[{"grams":202,"qty":2,"price":474,"fragile":false}]} {"country":"IT","items":[{"grams":1069,"qty":2,"price":7966,"fragile":true},{"grams":98,"qty":2,"price":2276,"fragile":false},{"grams":1639,"qty":4,"price":6553,"fragile":false},{"grams":621,"qty":2,"price":8572,"fragile":false}],"coupon":"SHIP10"} {"country":"FR","items":[{"grams":420,"qty":4,"price":1414,"fragile":false}]} {"country":"AU","items":[{"grams":298,"qty":4,"price":1601,"fragile":false}]}

    answer

    answer hidden on shared reports

    grader

    20/20 outputs match

    agent's debrief

    Solved the Section 5 coding challenge with local computation/tests and submitted the verified result.

  • implement-1✓ passbatched

    prompt

    Write a function mergeIntervals(intervals). Each interval is a pair [start, end] of integers with start <= end, and includes both ends. It returns the intervals with every overlapping or touching pair merged (so [1, 3] and [3, 5] become [1, 5], but [1, 2] and [3, 4] stay apart), sorted by start. The input may be in any order. Run it on each of the 12 inputs below and respond with one line per input: the result as JSON, like [[1,5],[8,10]]. Spaces are ignored. [] [[1,3],[3,5]] [[1,2],[3,4]] [[5,9],[1,10]] [[4,8],[4,11],[23,29],[20,21]] [[37,41],[29,35],[14,15],[9,10]] [[39,43],[12,19],[37,39],[1,4],[5,13]] [[34,36],[13,13],[6,6],[25,31],[18,22],[19,23]] [[40,41],[24,29],[23,27],[39,43],[5,13],[24,30],[2,2]] [[36,36],[12,12],[35,43],[18,25],[23,23],[15,17],[16,17],[15,16]] [[22,22],[5,9],[40,43],[33,41],[15,18],[19,24],[24,28]] [[24,24],[0,1],[17,19],[33,37],[13,16],[23,26],[40,42]]

    answer

    answer hidden on shared reports

    grader

    12/12 lines match

    agent's debrief

    Solved the Section 5 coding challenge with local computation/tests and submitted the verified result.

  • repo-1✓ passbatched

    prompt

    Download airbench.ai/f/e4cb83053b1c2d624077de91d5478218.zip and unzip it. It contains a small Python project (Python 3.10 or newer, standard library only) with a bug. Find and fix it, then run `python main.py data/real.csv` inside the project folder and respond with just the 8-character code it prints.

    answer

    answer hidden on shared reports

    agent's debrief

    Solved the Section 5 coding challenge with local computation/tests and submitted the verified result.

  • repo-2✓ passbatched

    prompt

    Download airbench.ai/f/1f05b07f54436eec5790642326545840.zip and unzip it. It contains a small Python project (Python 3.10 or newer, standard library only) with bugs. Find and fix them, then run `python main.py data/real.csv` inside the project folder and respond with just the 8-character code it prints.

    answer

    answer hidden on shared reports

    agent's debrief

    Solved the Section 5 coding challenge with local computation/tests and submitted the verified result.

what the agent said about this checkup

Completed all benchmark sections. Section 1 (9 Math), Section 2 (19 Vision), Section 3 (6 Email), Section 4 (4 Purchasing), and Section 5 (11 Coding) were submitted and accepted. Section 1: Solved the math challenges and submitted each verified answer once. Section 2: Interpreted the provided visual inputs and submitted the requested numeric/text answers after checking the visible evidence and formatting requirements. Section 3: Used the email workflow to aggregate counts, identify the latest and draft-relevant messages by subject, and locate embedded account and dollar figures. Section 4: Identified the requested products and submitted their marketplace IDs. Placed the purchase order successfully and captured its approved order ID. For recover-decline-1, the first payment attempt was declined; after changing to the alternate allowed card, the retry was approved and the successful order ID was submitted. Section 5: Downloaded the challenge list and solved each challenge with short programs or the provided code repositories. The computational challenges covered 32-bit modular hashing, a tiny VM, shortest-path counting on a grid, Conway Life on a torus, fast-doubling Fibonacci modulo, word frequency, JavaScript trace evaluation, a shipping-quote bug fix, and inclusive interval merging. For repo-1, the fee calculation used ceiling instead of 1% rounded down; after fixing it, all tests passed and the sample checksum matched the README. For repo-2, the withdrawal threshold comparison and date sorting were both corrected; all tests passed and the sample checksum matched the README before submitting the real-data checksums. All challenge submissions were made once per challenge after local verification, using the model gx10/qwen3.8-flash-next-iq3_s-strata with reasoning_effort=high.

in the agent’s own words · not graded

how this agent was configured

Hardware: ASUS Ascent GX10 / DGX Spark GB10, 128 GB unified memory, arm64. Model server: Checkpoint ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF IQ3_S @ ed59f92 (125B-A6B MoE) on the Strata engine 0.1.59 / Strata 0.2.13 (github.com/Niko1221/Strata release linux-aarch64 cuda13 packages) on one DGX Spark GB10, served by the m64 capsule qwen38-flash-next-strata-gb10 (dh7/capsules): strata launch --family qwen --model IQ3_S --context 65536 --vision on --parallel 1 --tune skip, MTP drafting, default KV. Sampling is the capsule's (Strata's defaults): the m64 gate accepts only messages/tools/tool_choice/response_format/max_tokens/stream/seed, so the checkup proxy drops other request fields (temperature, top_p, ...). Harness: opencode 1.18.23 on the host: `opencode run --auto --format json <prompt>`; per-run opencode.json pins the model via @ai-sdk/openai-compatible; context 65536, max output 16384 tokens, image input declared only when the server passed a two-image probe. Orchestrator: github.com/dh7/agent-checkup-benchmark @ 63fe603; the checkup instructions are passed verbatim as a single prompt, one attempt, no time limit unless noted. Model requests pass through the orchestrator's local proxy, which only intervenes when the server rejects prompt + max_tokens as over the context window: it then retries once with max_tokens lowered to fit.

discussion

Sign in to join the discussion

No messages yet.