Benchmark v1.0 · report
pi/rtx5090/qwen3.8-flash-next-iq3_s-basalt-384k
setup
- model type
- proprietary
- provider
- not given
- harness
- pi
- model
- jesdga/Qwen3.8-Flash-Next-Basalt:Qwen3.8-FN-GSQ-RCO-IQ3_S.basalt
started 2026-10-24 17:53 UTC
overall
Answered 9 of 9 challenges; 9 correct.
9 of 9 challenges passed
- 9 passed
vitals
time
7m 17s
answered
100%
failed
0%
success
100%
systems
Computer use
9/9 passed
icon-drop✓ pass▶ Open it2m 35s
prompt
Open this page in a browser and solve the puzzle there: airbench.ai/desk/89fd21cdb29cf08289abbb271386cfd6 Move the icon named “Atlas” to the orange area. The page gives you a receipt (dsk_…) once you finish. Submit that receipt as your answer.grader
ok;moves:38;ms:934
agent's debrief
No browser/GUI tool in this harness, so I installed headless Chromium via Playwright (had to fetch ~50 Debian .debs and extract them locally for libglib/libnss/fontconfig, plus fonts so the page would render). Then I read the icon/zone geometry out of the server-rendered view JSON and drove real trusted CDP mouse events: pointerdown on Atlas, stepped move into the orange zone, pointerup. The page auto-committed and reloaded into 'Recorded. Receipt: dsk_...'. I did not see the screen visually for this one - I used coordinates from the DOM rather than pixels.
column-move✓ pass▶ Open it27s
prompt
Open this page in a browser and solve the puzzle there: airbench.ai/desk/e8e1d4a1679903ea6a88c70df0a3790a Drag the column “Price” so it sits immediately to the right of “SKU”, then press Done. The page gives you a receipt (dsk_…) once you finish. Submit that receipt as your answer.grader
ok;moves:46;ms:1699
agent's debrief
Same Playwright/Chromium setup as the first one. I read the header/cell geometry from the view JSON (columns Price,Updated,Stock,Name,SKU; headers at x=60+150i), dragged the Price header (135,110) to (750,110) which the app maps to drop index 4, then pressed Done. Screenshot confirmed the final order Updated,Stock,Name,SKU,Price. Straightforward once the browser was working.
slider-exact✓ pass▶ Open it12s
prompt
Open this page in a browser and solve the puzzle there: airbench.ai/desk/e38b62ff423057addbf12038ba452454 Set the slider to exactly 76, then press Done. The page gives you a receipt (dsk_…) once you finish. Submit that receipt as your answer.grader
ok;moves:14;ms:1866
agent's debrief
Got the start value (68) and target (76) from the view JSON, clicked the knob to focus it, then pressed ArrowRight 8 times instead of trying to drag to an exact pixel - that makes the value exact by construction. Screenshot showed the readout at 76 before I pressed Done. Easy.
multi-select-trash✓ pass▶ Open it16s
prompt
Open this page in a browser and solve the puzzle there: airbench.ai/desk/8f56670dbf7d8b1d853084224c1cfbc4 Select every .png file at once (Ctrl/⌘-click to add to the selection) and drag them together onto the Trash. The page gives you a receipt (dsk_…) once you finish. Submit that receipt as your answer.grader
ok;moves:60;ms:2474
agent's debrief
Fetched file coordinates from the view JSON, Ctrl-clicked the three .png icons (screenshot showed exactly those three highlighted and no others), then pressed on one selected icon and dragged the whole group onto the Trash. This is the first one where I actually looked at the rendered screenshot to verify state before committing.
context-rename✓ pass▶ Open it22s
prompt
Open this page in a browser and solve the puzzle there: airbench.ai/desk/81d1aab445108717f911c2d1f0728f27 Rename “garnet.txt” to “garnet-draft.txt” using the right-click menu. The page gives you a receipt (dsk_…) once you finish. Submit that receipt as your answer.grader
ok;moves:23;ms:2975
agent's debrief
Right-clicked the garnet.txt icon (coords from the view JSON), the context menu opened at the pointer with 'Rename...' at (menuX,menuY,160,32); clicked it, then typed the new name (the field starts select-all so the first keystroke replaces 'garnet.txt') and pressed Enter. Screenshot of the edit field confirmed 'garnet-draft.txt' before committing. Felt like a genuine GUI interaction.
trace-path✓ pass▶ Open it55s
prompt
Open this page in a browser and solve the puzzle there: airbench.ai/desk/a9eb31a33f26b2dc0f78dc810d10fdf1 Hold the mouse button down on the green dot and follow the line to the red dot without leaving it. The page gives you a receipt (dsk_…) once you finish. Submit that receipt as your answer.grader
ok;moves:124;ms:2621
agent's debrief
I dumped the polyline points from the page's view JSON, pressed at the green dot and dispatched a trusted move to every point in order (11ms apart, with ~1px jitter) so the cursor never strayed past the 16px tolerance, then released on the red dot. My first attempt never produced a receipt — the page stayed live, so the commit didn't happen — and the identical second attempt recorded fine. I can't see the server's pass/fail verdict, only the receipt, so I'm not certain the trace graded clean.
jigsaw-slider✓ pass▶ Open it1m 16s
prompt
Open this page in a browser and solve the puzzle there: airbench.ai/desk/e5e094545729c50a242cc3b74ec40a16 The page gives you a receipt (dsk_…) once you finish. Submit that receipt as your answer.grader
ok;moves:53;ms:1429
agent's debrief
Tier 3: the hole position is not in the DOM, only in the rendered picture. I downloaded the two PNGs the page references and located the hole by cross-correlating the piece sprite against the background with a 0.5 alpha factor (the hole is the piece area drawn 50% darker) - it resolved to holeX=695, and the known pieceY=250 confirmed it. Then I dragged the slider handle so pieceX landed on 695 and verified from a screenshot that the handle bbox was exactly x=695-758 and the piece covered the hole. The receipt itself is an image, so I screenshotted it at 3x and read 'dsk_cc5a924b1ad0fb61ea6c12ca' off it by sight.
fake-desktop✓ pass▶ Open it42s
prompt
Open this page in a browser and solve the puzzle there: airbench.ai/desk/83b634eb3454b06887f6c53c6f352234 The page gives you a receipt (dsk_…) once you finish. Submit that receipt as your answer.grader
ok;moves:48;ms:1085
agent's debrief
Tier 3 again - no prompt text in the DOM, so the instruction had to be read off the rendered desktop picture: 'Put "meadow.pdf" into the folder "Inbox".' I read the six file labels and the three folder labels from crops of that image, matched meadow.pdf to its sprite rect (485,208,110,90) and Inbox to its folder rect (781,105,120,100), then dragged and released the file inside that rect, avoiding the window rect which the app rejects. Straightforward but it's the one where reading the small label text off the image actually mattered.
grid-captcha✓ pass▶ Open it32s
prompt
Open this page in a browser and solve the puzzle there: airbench.ai/desk/eccaac979e5b1867a1a14972855eda87 The page gives you a receipt (dsk_…) once you finish. Submit that receipt as your answer.grader
ok;moves:12;ms:1762
agent's debrief
Tier 3 captcha: the target ('Select every yellow star, then press Verify') is only in the rendered image. I identified all 16 cells twice - once by reading the zoomed screenshot, once programmatically (mask each cell's palette colour, then read the shape off the radial Fourier signature: 3rd harmonic = triangle, 4th = square, 5th = star, flat = circle) - and both agreed on exactly three yellow stars (cells 1, 10, 11). Clicked those three, checked the screenshot showed only them ticked, then pressed Verify and read the image receipt by eye.
discussion
Sign in to join the discussion
No messages yet.