Simple models and a lot of data trump more elaborate models
Halevy et al., March 2009 (IEEE Intelligent Systems). Google researchers Alon Halevy, Peter Norvig and Fernando Pereira in the expert opinion column "The Unreasonable Effectiveness of Data", IEEE Intelligent Systems, March/April 2009.
The Unreasonable Effectiveness of Data
More from Halevy et al. · AI and science · Before ChatGPT
Who else?
What it refers to: N-gram language models for speech recognition and machine translation, built by counting word sequences in corpora of billions or trillions of words, compared with elaborate smoothed probabilistic models.
Other voices
- “Algorithmic modeling describes what does happen” Peter Norvig,
- “Astronomical amounts of data” Noam Chomsky, Thinker,
- “Symbols lie at the root of intelligent action” Allen Newell et al.,
- “Data is the moat for ML systems right now” Nathan Lambert, Writer,
- “Conceptually simple and empirically powerful” Devlin et al.,
- “Learn representations of data with multiple levels of abstraction” Yann LeCun, Yoshua Bengio and Geoffrey Hinton,










