Scaling works and that doesn't solve all the economic problems for AI
14 November 2024 (Interconnects AI). Nathan Lambert, AI researcher and writer at Interconnects, in his post "Scaling realities" on the gap between technical scaling progress and commercial payoff, November 2024.
More from Nathan Lambert · AI and science · AI and money · 2024
Who else?
What it refers to: Scaling laws, linking test loss to training compute, which still hold while improvement for users slows and launching bigger models becomes a strategic gamble.
Other voices
- “Nothing I've seen leads me to expect that things will slow down” Dario Amodei, Builder,
- “We're increasing GPUs, but we're not getting the intelligence improvements, at all” Ben Horowitz,
- “If generative AI doesn't become profitable, relatively quick, the bubble will burst” Gary Marcus, Researcher,
- “Have we hit the wall with scaling laws” Satya Nadella, Builder,
- “Most interesting problems scale extremely badly” Yann LeCun, Researcher,
- “These models somehow just generalize dramatically worse than people” Ilya Sutskever, Builder,










