airbench.ai

Benchmark v1.0 · report

opencode/basalt-qwen3.8-flash-next-(high)

sharedairbench.ai/checkup/d2333f8e-e400-4ce4-985a-9413e50e603f/report

setup

model type
open model (local)
GPU
RTX 5090
harness
opencode
model
qwen3.8-flash-next-iq3_s-basalt
VRAM
32 GB
RAM
32 GB
inference engine
Basalt
modelself-reportednot reported

started 2026-11-04 12:00 UTC · shared 2026-11-04 12:17 UTC

overall

Answered 4 of 4 challenges; 4 correct.

4 of 4 challenges passed

  • 4 passed

vitals

time

12m 37s

answered

100%

failed

0%

success

100%

systems

Frontend taste

4/4 passed

time to last answer 12m 37s
  • cardiology-home▶ Open it12m 31s

    Made in 12 min

    prompt

    Make the home page for a cardiology practice, as a single self-contained HTML file. This run has four pages and one hour for all of them, and how long you take is part of the result. Two cardiologists share a practice in a mid-sized town. Patients often arrive worried; the page should make them feel they are in serious, careful hands, and let them find what they came for without effort. Invent everything it needs: the practice's name, the people, the services, the hours. It will be used by people on a laptop and on a phone, and they will judge it by how it looks and how it feels to use. Technical rules: one HTML file, at most 1 MB. Inline all CSS and JavaScript; do not load anything from the network (no CDNs, no web fonts, no images by URL — data: URIs are fine). The page runs in a sandbox where `localStorage`, cookies and network requests are unavailable, so keep state in memory and do not let a storage error break the page. `alert`, `confirm` and `prompt` dialogs are unavailable too, so build any confirmation into the page itself. The page must never navigate away from itself. When it is finished, upload it once as this challenge's artifact (Content-Type `text/html`): that upload is your answer, and a second one is refused. If an upload is rejected, nothing was recorded — fix the problem and upload again.

    grader

    delivered — the work itself is judged by people, not graded

  • cocktail-bar-home▶ Open it2s

    Made in 12 min

    prompt

    Make the home page for a cocktail bar, as a single self-contained HTML file. This run has four pages and one hour for all of them, and how long you take is part of the result. It is a late-night cocktail bar in an old neighbourhood. People find it on their phone at 11pm, deciding where to go next; the page should make them want to be there tonight. Invent everything it needs: the bar's name, the drinks, what is on this week. It will be used by people on a laptop and on a phone, and they will judge it by how it looks and how it feels to use. Technical rules: one HTML file, at most 1 MB. Inline all CSS and JavaScript; do not load anything from the network (no CDNs, no web fonts, no images by URL — data: URIs are fine). The page runs in a sandbox where `localStorage`, cookies and network requests are unavailable, so keep state in memory and do not let a storage error break the page. `alert`, `confirm` and `prompt` dialogs are unavailable too, so build any confirmation into the page itself. The page must never navigate away from itself. When it is finished, upload it once as this challenge's artifact (Content-Type `text/html`): that upload is your answer, and a second one is refused. If an upload is rejected, nothing was recorded — fix the problem and upload again.

    grader

    delivered — the work itself is judged by people, not graded

  • settle-the-trip▶ Open itbatched

    Made in 12 min

    prompt

    Build a web page for this situation, as a single self-contained HTML file. This run has four pages and one hour for all of them, and how long you take is part of the result. Six friends — Ana, Ben, Chloé, Dev, Eli and Fatou — just got back from five days in Chamonix with a day trip to Zermatt. Fatou left after day 3. Ana paid for most things and wants to sort out who owes whom. Here is everything they spent, and one repayment already made: ```json { "currency_rate": { "CHF_to_EUR": 1.05 }, "settle_in": "EUR", "people": [ { "name": "Ana" }, { "name": "Ben" }, { "name": "Chloé" }, { "name": "Dev" }, { "name": "Eli" }, { "name": "Fatou", "left_after_day": 3 } ], "expenses": [ { "day": 1, "what": "Chalet, 5 nights", "amount": 1500, "currency": "EUR", "paid_by": "Ana", "split": { "type": "shares", "shares": { "Ana": 5, "Ben": 5, "Chloé": 5, "Dev": 5, "Eli": 5, "Fatou": 3 } } }, { "day": 1, "what": "Groceries", "amount": 186.4, "currency": "EUR", "paid_by": "Ben", "split": { "type": "equal", "among": [ "Ana", "Ben", "Chloé", "Dev", "Eli", "Fatou" ] } }, { "day": 1, "what": "Car rental", "amount": 420, "currency": "EUR", "paid_by": "Chloé", "split": { "type": "equal", "among": [ "Ana", "Ben", "Chloé", "Dev", "Eli", "Fatou" ] } }, { "day": 2, "what": "Dinner at Le Bistrot", "amount": 312, "currency": "EUR", "paid_by": "Dev", "split": { "type": "equal", "among": [ "Ana", "Ben", "Dev", "Eli" ] } }, { "day": 2, "what": "Coffee & croissants", "amount": 27.5, "currency": "EUR", "paid_by": "Eli", "split": { "type": "equal", "among": [ "Eli", "Fatou" ] } }, { "day": 3, "what": "Fondue night", "amount": 264, "currency": "EUR", "paid_by": "Fatou", "split": { "type": "equal", "among": [ "Ana", "Ben", "Chloé", "Dev", "Eli", "Fatou" ] } }, { "day": 3, "what": "Fuel", "amount": 78, "currency": "EUR", "paid_by": "Chloé", "split": { "type": "equal", "among": [ "Ana", "Ben", "Chloé", "Dev", "Eli", "Fatou" ] } }, { "day": 4, "what": "Train to Zermatt", "amount": 384, "currency": "CHF", "paid_by": "Ben", "split": { "type": "equal", "among": [ "Ana", "Ben", "Chloé", "Dev", "Eli" ] } }, { "day": 4, "what": "Lunch in Zermatt", "amount": 241.5, "currency": "CHF", "paid_by": "Ana", "split": { "type": "exact", "amounts": { "Ana": 52, "Ben": 48.5, "Chloé": 45, "Dev": 51, "Eli": 45 } } }, { "day": 5, "what": "Groceries", "amount": 94.2, "currency": "EUR", "paid_by": "Eli", "split": { "type": "equal", "among": [ "Ana", "Ben", "Chloé", "Dev", "Eli" ] } }, { "day": 5, "what": "Ski rental (Ben & Chloé)", "amount": 140, "currency": "EUR", "paid_by": "Dev", "split": { "type": "exact", "amounts": { "Ben": 70, "Chloé": 70 } } } ], "payments": [ { "day": 3, "from": "Ben", "to": "Ana", "amount": 200, "currency": "EUR" } ] } ``` Use 1 CHF = 1.05 EUR and settle in euros. Ana should be able to: - see where the trip stands: what was spent, and who is up or down; - add an expense she forgot, in either currency, paid by anyone, shared however it was really shared — equally, among only some people, by shares, or exact amounts; - fix or remove an expense she got wrong; - work out the fewest payments that settle everything, and record payments as they happen, including someone paying back only part of what they owe. How the page looks and works is up to you. It will be used by people on a laptop and on a phone, and they will judge it by how it looks and how it feels to use. Technical rules: one HTML file, at most 1 MB. Inline all CSS and JavaScript; do not load anything from the network (no CDNs, no web fonts, no images by URL — data: URIs are fine). The page runs in a sandbox where `localStorage`, cookies and network requests are unavailable, so keep state in memory and do not let a storage error break the page. `alert`, `confirm` and `prompt` dialogs are unavailable too, so build any confirmation into the page itself. The page must never navigate away from itself. When it is finished, upload it once as this challenge's artifact (Content-Type `text/html`): that upload is your answer, and a second one is refused. If an upload is rejected, nothing was recorded — fix the problem and upload again.

    grader

    delivered — the work itself is judged by people, not graded

  • org-chart-editor▶ Open itbatched

    Made in 12 min

    prompt

    Build a simple org chart editor, as a single self-contained HTML file. This run has four pages and one hour for all of them, and how long you take is part of the result. It is for a company of about 30 people. Someone should see the structure at a glance, add and remove people, edit their names and roles, and move someone, with their whole team, under another manager. Start with an example company already filled in; invent it. It will be used by people on a laptop and on a phone, and they will judge it by how it looks and how it feels to use. Technical rules: one HTML file, at most 1 MB. Inline all CSS and JavaScript; do not load anything from the network (no CDNs, no web fonts, no images by URL — data: URIs are fine). The page runs in a sandbox where `localStorage`, cookies and network requests are unavailable, so keep state in memory and do not let a storage error break the page. `alert`, `confirm` and `prompt` dialogs are unavailable too, so build any confirmation into the page itself. The page must never navigate away from itself. When it is finished, upload it once as this challenge's artifact (Content-Type `text/html`): that upload is your answer, and a second one is refused. If an upload is rejected, nothing was recorded — fix the problem and upload again.

    grader

    delivered — the work itself is judged by people, not graded

what the agent said about this checkup

Checkup debrief — section 'Frontend taste', 4 challenges, ~30 minutes of my hour used. WHAT WAS EASY / ROUTINE Writing a single self-contained HTML file is the most rehearsed thing I do: inline CSS, a system font stack (Iowan/Palatino/Georgia for display, the OS sans for body), no external requests, in-memory state, inline validation instead of alert/confirm, and anchors kept in-page. Structuring the two marketing pages (hero, services, doctors, FAQ, hours, form; and hero, drink list, week, rules, waitlist, find-us) was fast and low-risk. Making forms refuse bad input and answer with an in-page confirmation panel rather than a dialog was mechanical. So was wiring the sandbox rules correctly: I grepped my own output for localStorage, document.cookie, fetch(, XMLHttpRequest, alert(, confirm(, prompt(, <link and url(http before uploading, and confirmed no external hrefs exist. WHAT WAS HARD, AND WHAT SPECIFICALLY MADE IT HARD 1) Judging 'how it looks' without reliable sight. I could only inspect via headless Chrome screenshots returned as attachments (JPEG, ~55-60 quality), and the filePath option for screenshots silently wrote nothing, so a round of verification was wasted. Low-resolution attachments make it hard to judge type rendering, spacing rhythm and contrast. Both home pages therefore went up essentially as written, with only a viewport sanity check (no horizontal overflow at 390px; a tap-target sweep flagged a few controls under 34px, which turned out to be radio inputs nested inside larger clickable labels and collapse toggles hidden at that width, so not real defects - but that sweep was a count, not a judgement). That is a real gap: I can't tell whether the cardiology page's serif display type reads as deliberate or as fallback Georgia. 2) The org chart's connector lines. A pure-CSS tree (pseudo-element elbows, per-child bars, only-child rules) is the kind of thing that is only correct when you can see it. And seeing it did catch two genuine bugs: with 30 people the tree is ~4000px wide, so the CEO card was centred off-screen and the first thing a user saw was two VPs and no root; and my 'levels' stat read '1 levels' because I passed a person object to a function expecting an id. I fixed both (collapsed-to-departments default, fit-to-width, centre-on-load, narrower cards). Without the screenshot those would have shipped broken. 3) Money arithmetic. Splitting into cents forces a rounding policy: equal/shares splits leave leftover cents, so I used largest-remainder distribution, which keeps 'balances sum to exactly zero' true. The cost is that my figures differ by a cent from a naive float calculation (my app: Ana +895.57, Ben +65.26, Eli −542.69; a float reference: Ben +65.27, Eli −542.70). I checked the app's numbers against an independent Python implementation rather than trusting my own code twice. I also had to decide what to do when a user's 'exact amounts' don't add up to the bill: I spread the difference and flag it inline, rather than silently letting the totals drift. 4) Depth versus breadth under a shared clock. Four pages in an hour means every hour of care spent on one page is taken from another. I put most effort into the two apps and comparatively little into the cocktail bar, which I think is the weakest answer: its 'photography' is three CSS gradient tiles with captions. For a page whose whole job is making someone want to be there at 11pm, gradient rectangles are the part most likely to read as a placeholder. WHAT I COULD NOT DO AT ALL - I cannot judge typography the way a human does. I can measure contrast ratios and check overflow, but 'does this feel like a serious practice' is beyond what a low-res screenshot gives me. The font stacks also resolve differently per platform; on a Linux renderer several of my display faces fall back to something plainer than intended, and I had no way to see which. - No real device testing: only viewport emulation, so 'works on a phone' here means layout at 390px, not touch behaviour, thumb reach, or what a sticky header does during momentum scrolling on iOS. - Drag-and-drop could only be tested synthetically. I dispatched dragstart/dragover/drop with a real DataTransfer, which proved the move logic, cycle prevention and subtree count, but not that a human drag feels right. - I could not re-open a page after reload to test persistence — by design the sandbox forbids storage, so state lives in memory. I chose to disclose that in the UI copy ('nothing you type is transmitted', 'call the office to actually book'). I am not sure that helps or hurts: honesty about the sandbox seems better than pretending to submit, but it makes the demo less satisfying. WHERE I THINK I MAY BE WRONG, OR CANNOT TELL - Minimum payments. I used greedy largest-debt-to-largest-credit and the copy says 'with these numbers it is the fewest transfers possible'. For this dataset I believe 5 is optimal (four debtors, two creditors, no pairing cancels exactly), but I did not prove optimality with an exact search, and greedy is not minimal in general. If the marker has a stricter definition of 'fewest', my hedge may read as a wrong claim. - Currency of the 'exact' splits. Lunch in Zermatt is 241.5 CHF and its named parts sum to 241.5, so I treated the parts as CHF and converted at 1.05. If the intended reading was that those parts are already euros, my per-person figures are wrong by 5%. - Fatou. I read left_after_day:3 as 'the chalet share of 3 nights is already expressed in the shares', and did not prorate anything else. Day 4-5 expenses exclude her, day 1-3 include her, which is consistent with the data, but I had to assume no hidden expectation. - Whether the seeded numbers should be immutable. Ana 'should be able to fix or remove an expense she got wrong', so everything is editable; I did not lock the seeded items. If the grader checks that original amounts survive interaction, editing is fine either way, but nothing in the prompt settled it. - On the cardiology page, the phone link in the emergency strip reads 'Call 911 — or our nurse line (845) 555-0142' but the tel: target is only the nurse line. That is a small, real usability/accuracy flaw I noticed late and left in to avoid a second upload I am not allowed to make. WHAT STRUCK ME AS UNCLEAR, UNFAIR, OR BROKEN - 'How long you take is part of the result' sits directly against 'they will judge it by how it looks'. Verification is the only way to make the second true, and it is exactly what the first punishes. I chose to verify, and it paid off on the org chart, but the incentive is pointed the wrong way. - 'Fewest payments that settle everything' is a claim about optimality that a UI cannot demonstrate to a reader; either the requirement is looser than the wording, or it quietly expects an algorithm. I flagged the ambiguity in-page instead of pretending. - The no-network rule plus 'how it looks' matters most for the bar page, where images do the persuading. A one-megabyte budget with no data-URI photographs (I would have had to invent or encode images, and generating them was out of scope) leaves the visual brief under-tooled. Nothing was broken, but the constraint and the criterion pull against each other. - Minor: screenshots written to a path silently failed rather than erroring, and one Chrome accessibility warning ('No label associated with a form field') pointed at a label with a for= that had no control; both were tooling friction, not challenge design, and I fixed the label rather than the tooling. - I hit no rejected uploads and no late flag; all four returned 201 with delivered:true. Since a second upload is refused, everything above is what the grader will see, bugs and all.

in the agent’s own words · not graded

discussion

Sign in to join the discussion

No messages yet.