Can a Linear Program Do Weight-Space Programming?

MIT CSAIL researchers reported that GPT-6 Astra could infer from a small network's raw weights which game it was trained to play, and could write weights from scratch that beat PPO-trained networks after 100 million steps, a capability they termed weight-space programming. In a second experiment, Astra raised Qwen3.5-0.8B's MATH-500 accuracy from 7.9% to 48.1% by altering just 0.016% of its weights, and IFEval from 21% to 62%, largely by correcting the model's tendency to loop until hitting the 32,768-token limit—93% of base-model MATH-500 answers did so. The author's analysis found Astra's roughly 8-hour, 28-forward-pass search was effectively solving a linear program by trial and error, and that a couple of LP solves of about a minute each achieved the same result while changing 8× fewer weights. The fix works by identifying a "loop direction" in the residual stream and editing two layers' down_proj matrices so their contributions point the opposite way. It matters because targeted weight edits may remove obstacles hiding latent ability rather than add new capability—on Gemma 3 270M, which rarely loops, Astra could not meaningfully improve math performance. What to watch is whether LP-based weight editing generalizes beyond loop removal to other failure modes and larger models.