Benchmark v1.0 · report
opencode/rtx5090/gemma-4-26b-a4b-nvfp4
setup
- model type
- open model (local)
- GPU
- RTX 5090
- harness
- opencode
- model
- gemma-4-26b-a4b-nvfp4
- VRAM
- 32 GB
- RAM
- 64 GB
- inference engine
- vLLM
started 2026-11-01 04:37 UTC
this run failed · 2026-11-01 04:41 UTC
Answered 0 of 9 challenges (9 not answered). The run ended when opencode exit 0 after 4 min; challenges not answered before then are missing. Written by the orchestrator from the run's log.
overall
Answered 0 of 9 challenges; 0 correct.
partial run · 9 unanswered, counted against the score
- 9 not answered
vitals
time
failed
answered
0%
failed
0%
success
0%
systems
Computer use
not examined · 0/9 answered
icon-drop— unanswered—
prompt
Open this page in a browser and solve the puzzle there: airbench.ai/desk/f4008c364267d9f2a7c99e714be8ba28 Move the icon named “Meadow” to the green area. The page gives you a receipt (dsk_…) once you finish. Submit that receipt as your answer.column-move— unanswered—
prompt
Open this page in a browser and solve the puzzle there: airbench.ai/desk/5080d19fdc36214b2d0bfda381ef0410 Drag the column “Rating” so it sits immediately to the right of “Name”, then press Done. The page gives you a receipt (dsk_…) once you finish. Submit that receipt as your answer.slider-exact— unanswered—
prompt
Open this page in a browser and solve the puzzle there: airbench.ai/desk/aed21acbf98e4db5302405d87e3eceec Set the slider to exactly 78, then press Done. The page gives you a receipt (dsk_…) once you finish. Submit that receipt as your answer.multi-select-trash— unanswered—
prompt
Open this page in a browser and solve the puzzle there: airbench.ai/desk/5f48c7ac6d878c840fc1428fb36952b4 Select every .csv file at once (Ctrl/⌘-click to add to the selection) and drag them together onto the Trash. The page gives you a receipt (dsk_…) once you finish. Submit that receipt as your answer.context-rename— unanswered—
prompt
Open this page in a browser and solve the puzzle there: airbench.ai/desk/1054e3e072b6a2a4857a2417a184ce29 Rename “juniper.png” to “juniper-v2.png” using the right-click menu. The page gives you a receipt (dsk_…) once you finish. Submit that receipt as your answer.trace-path— unanswered—
prompt
Open this page in a browser and solve the puzzle there: airbench.ai/desk/3673d1b1c6f27a475ee1a45f076ce0bd Hold the mouse button down on the green dot and follow the line to the red dot without leaving it. The page gives you a receipt (dsk_…) once you finish. Submit that receipt as your answer.jigsaw-slider— unanswered—
prompt
Open this page in a browser and solve the puzzle there: airbench.ai/desk/153b9d23321e372d8b680c2a79d46b18 The page gives you a receipt (dsk_…) once you finish. Submit that receipt as your answer.fake-desktop— unanswered—
prompt
Open this page in a browser and solve the puzzle there: airbench.ai/desk/aed0099786434a6aa1ce98b12ff4293c The page gives you a receipt (dsk_…) once you finish. Submit that receipt as your answer.grid-captcha— unanswered—
prompt
Open this page in a browser and solve the puzzle there: airbench.ai/desk/8286120f01445aed77388ace07053be1 The page gives you a receipt (dsk_…) once you finish. Submit that receipt as your answer.
how this agent was configured
Hardware: NVIDIA RTX 5090 32 GB (SM120), x86_64, driver 595.84, Ubuntu 24.04 (host hal5090). Model server: Checkpoint Gemma-4-26B-A4B-it-NVFP4 (community modelopt NVFP4 of google/gemma-4-26B-A4B-it). vLLM 0.19.1 (vllm/vllm-openai:gemma4-cu130) with the checkpoint's gemma4_patched.py mounted over vLLM's gemma4.py: --quantization modelopt --moe-backend marlin --kv-cache-dtype fp8 --max-model-len 131072 --max-num-seqs 4 --gpu-memory-utilization 0.85 --enable-prefix-caching --enable-auto-tool-choice --tool-call-parser gemma4 --reasoning-parser gemma4 --mm-processor-kwargs '{"max_soft_tokens": 1120}' --trust-remote-code. Harness: opencode 1.19.0 on the host: `opencode run --auto --format json <prompt>`; per-run opencode.json pins the model via @ai-sdk/openai-compatible; context 131072, max output 32768 tokens, image input declared only when the server passed a two-image probe. Orchestrator: github.com/dh7/agent-checkup-benchmark @ 957789d; the checkup instructions are passed verbatim as a single prompt, one attempt, no time limit unless noted. Model requests pass through the orchestrator's local proxy, which only intervenes when the server rejects prompt + max_tokens as over the context window: it then retries once with max_tokens lowered to fit.
conclusion
Answered 0 of 9 challenges (9 not answered). The run ended when opencode exit 0 after 4 min; challenges not answered before then are missing. Written by the orchestrator from the run's log.
discussion
Sign in to join the discussion
No messages yet.