TIS-100 Bench: LLMs Push the Pareto Frontier in Programming Puzzle Games

A new benchmark gave 16 large language models the programming puzzle game TIS-100 and its manual, withholding the community leaderboard, and the models produced 133 new Pareto-frontier points across 18 of the game's 55 puzzles, including three outright cycle records. Opus 5.5 and GPT-6 Astra led the field, each solving all 55 puzzles and finding new frontier points on 11, with Opus 5.5 posting 72 points to Astra's 32 while Astra edged ahead on record proximity (78.0 vs. 77.2) and led all six tested hypervolume settings. Gemini 4 Argon solved 54 puzzles with 8 new points, while Sonnet 5.5 (53 solved, 7 points) and GPT-6.1 Sol (54 solved, 6 points) followed; lower-ranked models such as GLM-5.3 (30 solved) and MiMo-V2.6-Pro (10 solved) trailed sharply. The results suggest solving and optimizing are distinct capabilities—Gemini 3.8 Flash solved 53 puzzles but found just one new point, whereas Fable 5.1 solved 52 and found five. Human players improved on most of the models' programs within days of their release, underscoring that the benchmark measures frontier-pushing rather than final mastery. What to watch next is whether the models' tradeoff discoveries hold up under fresh-game replay and 100,000 passing tests, and whether the gap between raw puzzle-solving and sustained optimization narrows in future model releases.