When a weak model wins, whose credit is it? The truth about AI model benchmark scores is more unreliable than you think.

Google Cloud AI Research and Northwestern University published an October 9, 2026 study showing that reconfiguring an AI agent's external "harness"—system prompts, memory, tool-use guidance, scripts, and helper agents—can reverse model rankings without touching any model weights. In 100 office benchmark tasks scored out of 100, open-source GLM-5 averaged 64.2 versus 72.4 for Gemini 3.5 Flash under default settings, but a single round of automated harness tuning lifted GLM-5 to 75.7, beating default Gemini by 3.3 points; a second configuration gave GLM-5 75.3 versus Gemini's 74.5. Applying the same tuning to Gemini raised it to 77.5, showing the native capability gap between the models is smaller than the gains from harness engineering. The authors' method optimizes by comparing agent behavior traces rather than scoring against a fixed answer key, enabling tuning on open-ended tasks with no ground truth. The findings matter because leaderboard scores reflect model-plus-scaffold systems, not raw model intelligence, and the results have not yet been independently replicated.