On Chomsky and the Two Cultures of Statistical Learning

At the 2011 Brains, Minds, and Machines symposium held during MIT's 150th anniversary, Noam Chomsky criticized machine-learning researchers who rely on purely statistical methods, arguing that approximating unanalyzed data is a novel but shallow notion of success; he made the remarks in response to a question from Steven Pinker about probabilistic models. The essay responding to the exchange examines what Chomsky meant and whether he is right, covering the nature of statistical models, the track record of statistical language models, and whether their notion of success has precedent in the history of science. The author concedes that engineering success is not the sole measure of science but argues that science and engineering advance together, so working systems count as evidence—though not proof—of a scientifically sound model. The piece also pushes back on the idea that fact-gathering is a novelty, noting that laborious accumulation of facts has dominated the history of science, and acknowledges the difficulty of interpreting models with billions of parameters. The debate matters because it frames the split between statistical and principle-based approaches to language, and the next thing to watch is how that tension plays out as such large models continue to scale.