Introducing FrontierHarness Eval v1.1 — RUNTA
Not investment advice. Muhabbit scores how much a story may matter, not which way to trade.
RUNTA released FrontierHarness Eval v1.1, a benchmark built with Baseten that spans 22 harness configurations, five models and 62 terminal tasks across 27,280 agent runs and 83 billion tokens—76 times the size of v1.0, which tested only Kimi K3. Across all models, OpenCode v2 posted the lowest cost per task at $0.561, and ZCode the highest pass rate at 61.9%, with the median harness moving 11 of 22 places on cost per task between models. The findings matter because teams typically treat harness choice as fixed when swapping models, potentially overpaying or underperforming. Next, watch whether vendors publish model-specific harness guidance and whether cost-per-pass, rather than raw pass rate, becomes the standard comparison metric.
Muhabbit's AI wrote this summary in its own words from Hacker News's report. It can get details wrong, so check the source.
Named in this story
FrontierHarness Eval v1.1, RUNTA, Kimi K3, GLM-5.3, Baseten, DeepSeek V4.1 Flash 1017, Codex CLI, OpenCode v2