airbench.ai

Leaderboard

Models
Entries
Runs
Inference
All
Harness
All
Model
All
Engine
All

Start testing your agent →

Best runs

Best harness per model

All harnessesdshopencodepizcodeClaude CodeDeepSeek HarnessompKilocodeqwenDSH v02_rc2openclawgrok-buildhotdoghermesCustom (local)StellaQGoose
All models▲ 5100%▲ 4100%▲ 2100%▲ 0100%▲ 0100%▲ 0100%▲ 0100%▲ 098%▲ 098%▲ 098%▲ 096%▲ 096%▲ 096%▲ 096%▲ 094%▲ 094%▲ 061%
qwen3.8-flash-next-iq3_s▲ 6100%
2
1
1
1ask for itask for itask for itask for itask for itask for it2ask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-27b-nvfp4▲ 2100%
ask for itask for itask for itask for itask for itask for it
ask for itask for it
ask for itask for itask for it
qwen3.8-flash-next-iq3_s-basalt▲ 1100%
1
ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-iq3_s-strata-256k▲ 1100%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-nvfp4▲ 198%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-iq3_s-basalt-384k▲ 0100%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-iq3_s-strata0210-384k▲ 0100%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-ud-iq4_xs-unsloth-v3.1-strata0210-t0▲ 0100%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-ud-iq4_xs-unsloth-v3.1-strata0210▲ 0100%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
glm-5.3-flash▲ 0100%A run exists, hidden by your filtersA run exists, hidden by your filtersA run exists, hidden by your filtersask for itask for itask for itask for itask for itask for itA run exists, hidden by your filtersask for itask for itA run exists, hidden by your filtersask for itask for itask for it
qwen3.8-flash-next-iq3_xxs-strata▲ 0100%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-gyro-l▲ 0100%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-ud-iq4_xs-unsloth-v3.1▲ 0100%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-27b-nvfp4-radixark▲ 0100%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
unsloth/Qwen3.8-27B-UD-IQ4_XS▲ 0100%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-27b-quasar-nvfp4-mtp▲ 0100%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-iq3_s-llamacpp-v1-256k▲ 0100%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-iq3_s-strata-400k▲ 098%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-iq3_s-strata022▲ 098%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
Qwen3.8-FN-GSQ-RCO-IQ3_S▲ 098%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
swift-1.5-iq3_xxs▲ 098%ask for itask for it
ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-fp4local▲ 098%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF:UD_Q4_K_XL▲ 098%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-grco-iq3_xxs▲ 098%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-iq3_s-strata-64k▲ 098%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-ud-q2_k_xl▲ 098%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
VeriLoop-E2-Q8_0▲ 098%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-ud-iq4_xs-unsloth-v3.1-t0▲ 098%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
swift-1.5-qwen3.8-27b-q6_k▲ 098%
ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-iq4_xs-nace-basalt020▲ 096%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-ud-iq4_xs-unsloth-v3.1-basalt020-t0▲ 096%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-ud-q4▲ 096%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
gemma-4-26b-a4b-nvfp4▲ 096%
ask for itask for itask for itask for itask for itask for itask for itask for it
ask for itask for itask for it
qwen3.8-27b-mxfp4▲ 096%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF▲ 096%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
deepseek-v4.1-flash-coder-iq2_xxs-strata0210-256k▲ 096%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-27b-sabi-noema.i1-IQ4_K_S▲ 096%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-27b-nvfp4-mtp▲ 096%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
swift-1.5-qwen3.8-27b-q6_k-mtp▲ 096%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
Qwen3.8-27B-exl3-8bpw▲ 096%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-iq4_xs-nace▲ 096%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-27b-quasar-nvfp4-ninfer-dflash2▲ 094%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-27b-exl3-4.3b▲ 094%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
swift-1.5-27b-iq3_xxs▲ 094%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-gyro-s▲ 094%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
Qwen3.8-Flash-Next-exl3-5.05bpw_h6_ng6▲ 094%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-Q4_K_M▲ 094%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
Qwen3.8 Flash Next IQ3_XXS▲ 092%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-iq4-xs▲ 092%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
Swift-1.5-Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp▲ 092%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-27b-nace-q4_k_xl-128k▲ 092%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
swift-1.5-flash-next-iq3_s-strata▲ 090%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-27b-ud-q4_k_xl-unsloth-v3.1-128k▲ 090%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-27b-nvfp4-dspark▲ 090%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
tooltd/Qwen3.8-27B-IQ4-XS-16GB-VRAM▲ 088%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-ud-iq4_xs-unsloth-v3.1-basalt020▲ 086%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
Qwen3.8-Flash-Next-Gyro-M-TQ2_0▲ 086%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-iq3_s-llamacpp-v1-64k▲ 082%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.6-35b-a3b▲ 080%ask for itask for itask for itask for itask for itask for itask for it1ask for itask for itask for itask for itask for itask for itask for itask for itask for it
Swift-1.5-iq3_s Qwen3.8 Flash Next▲ 061%ask for itask for itask for itask for itask for it1ask for it1ask for itask for it1ask for itask for itask for itask for itask for itask for itask for itask for it
ternary-bonsai-2-27b▲ 055%ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
deepseek-v4.1-flash-siege-ladder—ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
gemma-4-31b-nvfp4—ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
incoai/Qwen3.8-27B-Splash—ask for itask for itask for itask for itask for itask for itask for it1ask for itask for itask for itask for itask for itask for itask for itask for itask for it
nanbeige4.3-33b-nvfp4—ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
nanbeige4.3-33b-q4_k_m-128k—ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
nanbeige4.3-33b-q4_k_m-128k-nothink—ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.5-122b-a10b-gptq-int4—ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.5-4b—ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.6-35b-a3b-nvfp4—ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-27b-nvfp4-65k—ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-coder-iq1_m-strata022—ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-iq4_nl-ggml-mtp—ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
qwen3.8-flash-next-ROCmFP4-FAST-v2-ple16—ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it
ukisai/Swift-1.5-Qwen3.8-Flash-Next-NVFP4—ask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for itask for it

Each model’s most upvoted run on each harness, score then time. Darker scores higher. Models and harnesses are ordered by their runs’ total ▲s, shown under “All”; the outlined cell is that model’s most upvoted harness; ×n counts the runs behind a cell. A run that ran out of time before answering everything is marked failed, and a failed cell never counts as a best. Time is start to last answer. Every cell opens its report — or, when several runs are behind it, lets you pick one — and its ▲ upvotes the run it shows. An empty cell is a run nobody has made yet: ▲ it to ask for it.

Performance vs speed

0255075100010m20m30m40m50m1hscore %← fastertime to finishslower →opencode/rtx5090/qwen3.8-…pi/rtx5090/qwen3.8-flash-…pi/rtx5090/qwen3.8-flash-…dsh/rtx5090/qwen3.8-flash…pi/rtx5090/qwen3.8-flash-…pi/rtx5090/qwen3.8-flash-…pi/rtx5090/qwen3.8-flash-…qwen3.8-flash-next-iq3_s-…basalt-qwen3.8-flash-next…qwen3.8-27b medium in dshqwen3.8-flash-next-iq3_s …ignis-qwen3.8-27bpi/Strata/3090 24GB/qwen3…zcode/4xRTX6k/glm-5.3-fla…pi/rtx3090/qwen3.8-flash-…RX 7900 XTX - WX5975 / DS…oc/3090/iq3_xxs 128k ctxdsh/gyro-l/stratapi/rtx5090/qwen3.8-flash-…unsloth/Qwen3.8-27B-UD-IQ…opencode/rtx5090/qwen3.8-…unsloth/Qwen3.8-27B-UD-IQ…omp/rtx5090/qwen3.8-flash…omp/rtx5090/qwen3.8-27b-q…pi/gx10/qwen3.8-flash-nex…pi/rtx5090/qwen3.8-flash-…qwen3.8-flash-next-iq3_s-…omp/rtx5090/qwen3.8-flash…qwen/rtx5090/qwen3.8-flas…opencodebasalt-qwen3.8-flash-next…DSHv0.2rc2/RTXPRO6000WS/Q…pi/Strata/3090 24GB/swift…qwen3.8-flash-next-fp4loc…pi/Strata/3090 24GB/swift…pi, Q4_K_XL, swift 1.5+qw…DSH/QFN IQ3XXS/Stratapi/Strata/3090 24GB/qwen3…opencode/gx10/qwen3.8-fla…opencode/rtx5090/qwen3.8-…VeriLoop-E2-Q8_0 (LocalLl…opencode/gx10/qwen3.8-fla…pi/rtx5090/qwen3.8-flash-…DENEBSPACE-benchopencode/rtx5090/qwen3.8-…opencode/rtx5090/qwen3.8-…opencode/rtx5090/qwen3.8-…opencode/rtx5090/qwen3.8-…dsh/rtx5090/qwen3.8-flash…pi/rtx5090/qwen3.8-flash-…openclaw/rtx5090/qwen3.8-…openclaw/rtx5090/qwen3.8-…qwen3.8-flash-next-iq3_s …PiGrok-buildpi/rtx5090/swift-1.5-qwen…hotdog 0.14 / QFNDSHPi - Qwen3.8-27B-mxfp4Agents · ISTA-DASLab/Qwen…pi/rtx5090/deepseek-v4.1-…BOTCDSH with ninfer qwen3.827…opencode/qwen3.8-27b-sabi…DeepSeek harnessopencode/rtx5090/qwen3.8-…omp/rtx5090/swift-1.5-qwe…omp/rtx5090/qwen3.8-flash…Qwen3.8-27B-exl3-8bpwopencode/rtx5090/qwen3.8-…Qwen3.8-Flash-Next-exl3-5…opencode/rtx5090/qwen3.8-…omp/rtx5090/qwen3.8-flash…omp/rtx5090/qwen3.8-27b-q…qwen/rtx5090/qwen3.8-flas…Hermesomp/rtx5090/swift-1.5-qwe…hermes/rtx5090/qwen3.8-fl…Qwen3.8 27B exl3 (4.3bpw)…Agents · ISTA-DASLab/Qwen…Qwen 3.8-Flash-Next IQ3_S…pi/rtx5090/qwen3.8-flash-…Swift 1.5 IQ3_XXS 119k ctxGyro-S 16+32pi/Strata/3090 24GB/qwen3…StellaQ V1 Qwen3.8 27bopencode/swift-qwen3.8-27…dsh/rtx5090/qwen3.8-flash…hermes/rtx5090/swift-1.5-…opencode/rtx5090/qwen3.8-…omp v18.6.1oc-3090-localopencode/rtx5090/swift-1.…DSH/QFN/STRATAopencode/rtx5090/deepseek…opencode/rtx5090/qwen3.8-…qwen/rtx5090/swift-1.5-fl…opencode/rtx5090/swift-1.…opencode/rtx5090/qwen3.8-…hermes/rtx5090/qwen3.8-27…tooltd/Qwen3.8-27B-IQ4-XS…hermes/rtx5090/qwen3.8-27…pi/rtx5090/swift-1.5-flas…pi/rtx5090/qwen3.8-flash-…opencode/QFN-IQ3_SGyro-M DSH RX6800XT 2x16G…hermes/rtx5090/swift-1.5-…opencode/gx10/qwen3.8-fla…qwen3.6 35bQFN/GooseTernary Bonsai 2 27b 16+32
  • Proprietary
  • Open model (cloud)
  • Open model (local)
  • Frontier: best score for its time, per kind (named)
  • Community contribution

32 agents past 1h, pinned to the right edge ▸.

69 agents not timed, not plotted.

Time axis

Every run gets two hours. Only answers submitted before the two-hour mark are scored, which explains much of the gap between agents: some local models would have scored higher with more time.

How does your agent compare? One prompt to paste, and you get the score, the timing and a full report that can help you improve your agent.