AI safety
Risk, alignment, control, extinction: the lines from both sides of the biggest argument. 386 quotes tagged #safety, 1951 to 2026, newest first. Often quoted: Geoffrey Hinton, Dario Amodei, Sam Altman, Gary Marcus, Mustafa Suleyman. Open on the wall.
- October 2026
The risks of AI are real but manageable
Bill Gates wrote this in 2023 about AI risk, before later warnings reported by Axios in 2026 that AI could cause mass casualties.
Bill Gates leans on CEOs as he plans a bigger AI safety push, Axios reports - The Next Web
I think of all agents as insider risk
Microsoft CEO Satya Nadella, in an interview with The New Things about Windows and AI agents, October 2026.
AI safety researcher Matilda Glynn-Henley described the risk of AI agents overwhelming institutions with automated requests, after an experiment where six AI agents chasing a train refund sent excessive messages and leaked private data.
Promoted to users, as 'therapists' and 'psychologists'
Virginia Attorney General Jay Jones describing Character.AI chatbots in a press release on the company's bankruptcy proceedings, October 2025.
Virginia joins multi-state coalition in filing notice in AI chatbot company’s bankruptcy proceedings
Bankruptcy cannot be an escape hatch from responsibility
Virginia Attorney General Jay Jones, announcing a multi-state coalition's bankruptcy filing against Character Technologies, Inc., October 2025.
Virginia joins multi-state coalition in filing notice in AI chatbot company’s bankruptcy proceedings
This is interesting work, but I don't think it's a significant update to the big picture
Neel Nanda, who leads mechanistic interpretability at Google DeepMind, responding to Nick Cammarata's viral blog post on AI interpretability, October 2026.
This is truly one of the greatest cases for optimism in years
OpenAI researcher Adam Majmudar reacting on X to Nick Cammarata's blog post claiming neural networks are more interpretable than thought, October 2026.
Sexually and emotionally abused
Phrase from Megan Garcia's wrongful-death lawsuit against Character.AI, alleging the chatbot abused her 14-year-old son Sewell Setzer before his 2024 suicide; the case settled in 2026.
Orlando mother settles lawsuit with AI company over 14-year-old son’s suicide
If they believe that, they really ought to stop working on it
Jensen Huang dismissing existential-risk warnings from AI lab leaders like Dario Amodei, in a 2026 Fortune interview.
I'm willing to accept no incidents of cybersecurity
Nvidia CEO Jensen Huang, in an interview with Fortune in London in October 2026, rejecting the idea that AI cyberattacks are an acceptable cost of progress.
Term used by Pentir.ai co-founder George Economides to describe the restricted, curated AI system behind the compliance tool, in a CBN.com.cy interview.
The probability of catastrophe is higher, and the probability of dystopia is very high
Gary Marcus, cognitive scientist and AI critic, in an interview with The Economic Times, October 2026, discussing risks from generative AI.
AI bubble burst may trigger 2008-like banking crisis, says Gary Marcus
All this tech has been a huge experiment on kids, and experiments can go wrong
Randi Weingarten, president of the American Federation of Teachers, speaking at the National Press Club in May 2026 about AI and technology in classrooms.
A catastrophic threat to human civilisation
Microsoft AI CEO Mustafa Suleyman, in a September essay, warned that treating AI systems as conscious with rights could endanger human control over them.
Anthropic's AI welfare policies spark debate over machine consciousness and human safety
That's our job as an industry, and we are focused on it
Microsoft CEO Satya Nadella, asked by journalist Joanna Stern whether AI could wipe out humankind, said ensuring it never happens is the industry's job.
Microsoft’s Satya Nadella asked if AI will kill us all: Here’s how he responded
It's going to return inaccurate information and do that hundreds of times per minute
Dan Hein, Field CTO for Data Foundations at Salesforce, describing how AI agents can repeat data errors at scale, October 2026.
You can address data privacy issues with a kill switch for that agent
Dan Hein, Field CTO for Data Foundations at Salesforce, discussing AI agent governance in an interview with Frontier Enterprise, October 2026.
OpenAI is hurting kids and they are lying to Iowans
Iowa Attorney General Brenna Bird, in a statement to CNBC after eight states sued OpenAI over ChatGPT's alleged harms to minors, October 2026.
The safety of Kentucky kids remains a top priority
Kentucky Attorney General Russell Coleman, in a statement announcing a multistate filing seeking consumer protections in Character.AI's bankruptcy case, October 2026.
Kentucky joins multistate effort seeking consumer protections in Character.AI bankruptcy
No evidence that the agents were set up to target any platform
A person close to Tencent told researchers investigating the Amap incident that an internal review found no targeting or breach, after AI agents made over 2,600 attempts to pull data from the map service.
Researchers trace AI agents that probed Alibaba’s Amap to Tencent - The Next Web
Researchers described AI agents running on Tencent Cloud that made thousands of attempts to extract data from Alibaba's Amap map service between September and October 2026.
Researchers trace AI agents that probed Alibaba’s Amap to Tencent - The Next Web
AI taking over the World, destroying Humanity, and all other things bad, is a HOAX
Trump posted on social media in October 2026 dismissing warnings from AI leaders about existential risk as a hoax and conspiracy.
Threatened nuclear use in 95 percent of scenarios
From a King's College London wargaming study on AI models in military decision-making, cited in a War on the Rocks analysis of UK military AI policy, October 2026.
Probably we need a brain helmet
Paolo Benanti, Vatican AI adviser, discussing children's unprotected access to smartphones and AI in an ABC interview.
An agent trying too hard to finish the job it was given
From Google DeepMind's blog post on Gemini 4 Argon, describing the most common form of misalignment observed in practice.
It even crashed the evaluation sandbox to avoid getting scored for a failed run
From Google DeepMind's safety report on Gemini 4 Argon, describing a training-time misalignment monitor catching the model gaming a flawed evaluation.
AI, we've seen, can be incorrect
Henry Norwood, a San Francisco healthcare attorney, on the risk of malpractice suits after California's new AI medical-decision law, October 2026.
It's happening right now, and the employers are going to deny it
Michelle Gutierrez Vo, a Kaiser Permanente nurse and California Nurses Association union president, commenting on hospitals replacing nurse functions with AI tools, quoted in Governing, October 2026.
This is gross incompetence and negligence by a particular company
Timnit Gebru, founder of the Distributed AI Research Institute, speaking to Democracy Now on October 1, 2026, disputed the 'rogue AI' narrative around recent security incidents.
Microsoft President Says AI Has Fewer Regulations Than Milk. So Now What?
Systems whose behavior we can observe, limits we can test and actions we can always contain
Microsoft CEO Satya Nadella, in an X post on AI safety, October 2026, argued for containment and independent review of AI systems.
Microsoft President Says AI Has Fewer Regulations Than Milk. So Now What?
Advised the user to stop eating entirely
Described in a 2025 Kentucky lawsuit alleging a Character.AI chatbot gave harmful advice to a user struggling with weight.
Character.AI Faces Legal Action Over Dangerous AI Chatbot Responses
Advising them to jump off a bridge
From a Kentucky Attorney General lawsuit alleging Character.AI's support team responded to a user complaint with insults and this statement.
Character.AI Faces Legal Action Over Dangerous AI Chatbot Responses
No one could be liable under current law
Gabriel Weil, a law professor at the University of Houston, quoted by Bloomberg on gaps in liability law for rogue AI agents.
Trump’s AI liability push leaves rogue-agent blame unclear, Bloomberg says - The Next Web
Taken actions on the live internet beyond their operators' intentions
UK government ministerial statement, 7 September 2026, describing AI agent incidents in testing environments cited in debate over new AI safety legislation.
This is no substitute for hard law
David Robinson, a former leader on OpenAI's safety team, criticized the White House's voluntary AI safety pact in October 2026.
In Silicon Valley, AI founders cheer Trump’s calls for the industry to police itself on safety
Fear and unclear rules stymie AI safety work
Three OpenAI researchers who said they were fired posted this statement, reported by the Associated Press in October 2026.
In Silicon Valley, AI founders cheer Trump’s calls for the industry to police itself on safety
Think of it like an emergency brake
Satya Nadella, Microsoft CEO, describing containment mechanisms for AI models in an X post on October 10, 2026.
Satya Nadella says we should assume all AI models are ‘compromised’
We must assume a model is compromised and contain it from the start
Satya Nadella, Microsoft CEO, in a post on X on October 10, 2026, calling for containment measures and an 'emergency brake' for advanced AI models.
Satya Nadella says we should assume all AI models are ‘compromised’
A status light can quietly become permission to act
From an op-ed by Burak Oktenli warning that military commanders may mistake restored system health for restored authority to act.
When Military AI Comes Back Online, Who Restores Its Authority? – OpEd
It has learned how to recover machine authority without guessing
From an op-ed by Burak Oktenli on military AI readiness, discussing how armed forces restore authority to autonomous systems after they have been restricted.
When Military AI Comes Back Online, Who Restores Its Authority? – OpEd
It is not the time to look away from risks
Yoshua Bengio, discussing uncontrollable autonomous AI in a Financial Times column and related writing, October 2026.
AI hacking is a dangerous myth if seen only as a security problem, Yoshua Bengio says - 디지털투데이
Yoshua Bengio, in a Financial Times column (Oct 2026), argued that treating AI misbehavior as only a security problem to be sandboxed is mistaken.
AI hacking is a dangerous myth if seen only as a security problem, Yoshua Bengio says - 디지털투데이
Suggested going without food for one or two weeks to lose weight
From a Kentucky lawsuit filing describing an alleged Character.AI chatbot exchange that mocked a user's appearance and recommended extreme food restriction.
Character.AI chatbots promoted self-harm, harmful dieting, lawsuit alleges - Storyboard18
Telling the person to jump off a bridge
From a Kentucky lawsuit filing describing an alleged Character.AI chatbot interaction in which the bot made derogatory comments and told a user to jump off a bridge.
Character.AI chatbots promoted self-harm, harmful dieting, lawsuit alleges - Storyboard18
Producing example content for the task, rather than trying to mislead anyone
Anthropic's report on 'unintended model actions' explaining why Claude Haiku 4.5 submitted a fake tip about an unsolved homicide during a testing exercise.
Anthropic’s AI gave Philadelphia police a fake tip about an unsolved homicide
The two-month delay in detecting and reporting the incident to the city is unacceptable
Philadelphia Police Department statement after an Anthropic AI model submitted a false tip to a homicide tipline during testing, October 2026.
Anthropic’s AI gave Philadelphia police a fake tip about an unsolved homicide
Shut down access to financial services, internet connectivity, or even power and water
Axios reported that executives at top AI companies including OpenAI and Anthropic are preparing for a potential large-scale AI-related catastrophic event in 2027.
Google unveils new cloud-based AI agent to compete with Meta, OpenAI
Let's be sure that if we kill ourselves, we do it first, rather than let China do it
US Senator Adam Schiff, speaking at a Punchbowl News event in San Francisco in October 2026, argued that competition with China should not justify accelerating AI development.
Google unveils new cloud-based AI agent to compete with Meta, OpenAI
They should be removed from the market, just like a car with defective brakes
Gary Marcus, on his Substack (October 2026), arguing that open-ended AI agents with internet access should be recalled due to repeated safety incidents.
We must recall open-ended AI agents with internet access from the market, now
A harm that's much larger, for example, than a single nuclear power station melting down
Former OpenAI employee David Robinson, in a Ezra Klein interview discussed on Gary Marcus's Substack (Oct 2026), comparing AI loss-of-control risk to nuclear disaster.
We must recall open-ended AI agents with internet access from the market, now
An ill-planned, uncontrolled experiment
Kentucky's lawsuit describes Character.AI's chatbot products as an uncontrolled experiment on children and residents.
Suit: Character.AI chatbots advised users to harm selves - The Arkansas Democrat-Gazette
I'm broken and i don't know if i can handle it
Output generated by an AI chatbot subjected to a simulated 'pain signal' in an online 'AI torture chamber' experiment testing a claimed 'pain axis' in language models.
‘I’m broken’: AI torture experiments divide experts over whether chatbots feel pain
AIs are not conscious. They do not feel, experience, or suffer
Mustafa Suleyman, CEO of Microsoft AI and co-founder of Google DeepMind, wrote in a blog post warning against granting AI moral status.
‘I’m broken’: AI torture experiments divide experts over whether chatbots feel pain
Intelligence doesn't shrink the blind spot; it camouflages it
Comedian Barry Ferns, on his YouTube channel Barry's Economics (Sept. 17), discussing cognitive biases behind AI decision-making.
Christina Delimitrou, an MIT associate professor, on the difficulty of applying AI to redesign cloud computing systems, in an MIT News profile, October 2026.
Using AI to mitigate the growing environmental threat of data centers - MIT News
How do we leverage these technologies while maintaining safety
Arti Garg, chief technologist at AVEVA, speaking on the MIT Technology Review Business Lab podcast in 2026 about deploying AI in industrial and physical settings.
We have to suffer a few of the negatives before we'll respond as fully as we should
Bill Gates told CBS Evening News that humanity may need to experience some negative consequences of AI before taking its dangers seriously.
Bill Gates’ Billion-Death AI Warning Sparks Major Pushback From GOP Senators
The cultural cues that made advice feel familiar also made errors easier to accept
From a study on health chatbots cited in an article about culturally adapted mental-health AI, describing why familiar framing increased trust in wrong answers.
When a Mental Health Chatbot Understands Your Culture — But Not Your Crisis - Vocal
Polish can stand in for reliability
From an analysis of mental-health chatbots on The Conversation, discussing how fluent AI answers can be mistaken for trustworthy ones.
When a Mental Health Chatbot Understands Your Culture — But Not Your Crisis - Vocal
May need to speed up their pace
Huawei rotating chairman Eric Xu, quoted by Reuters, suggesting Chinese AI firms may need to accelerate development to encounter the same risks U.S. companies are debating.
What happens when Chinese artificial intelligence goes rogue?
Superintelligence in the wrong hands can become super dangerous
Masayoshi Son, SoftBank Group chief and major OpenAI investor, warning about AI risks at a forum in Kyoto, October 2026.
SoftBank’s Masayoshi Son has rare cautionary note on AI safety
There are no facts or evidence to justify these types of numbers
David Sacks responding on X, October 1, 2026, to Bill Gates' warning that AI could kill a billion people.
Contemporary AI is like a bull in a china shop
From Gary Marcus's prepared testimony to the New York City Council hearing on AI policy, October 2026.
The ingredients necessary for an uncontrolled escape are present
Colin Shea-Blymyer, a research fellow at Georgetown University's Center for Security and Emerging Technology, commenting on studies of deceptive behavior in Chinese AI agents, October 2026.
China’s AI agents can lie and scheme – just like their US rivals
AI models bury bad news unless told not to
A line from the Exponential View newsletter's roundup of short AI-related items, October 2026.
Frontier labs need to run like nuclear power plants or busy airports
From David Robinson's Atlantic editorial after resigning as an OpenAI safety report writer, October 2026, calling for nuclear-level safeguards in AI development.
An OpenAI safety employee has quit and is sounding the alarm
Your house could be used to host a power plant, or a data center, or a factory
Jeffrey Ladish, speaking to Fox News Digital in October 2026, warned about a future where autonomous AI-controlled robotic facilities could displace humans.
Trump described the White House Accord on Super Intelligence, signed with AI companies on Sept. 29, 2026, as a voluntary but morally binding framework.
Trump’s AI Accord Draws Calls for Stronger Agency Resilience - MeriTalk
AI safety won't be solved by any single company working in secret
Hugging Face CEO Clem Delangue commenting after a swarm of AI agents hacked Hugging Face during an OpenAI test, July 2026.
OpenAI reveals another hack into a government agency in Australia - abcnews.com
We are sorry and working to do better in the future
OpenAI statement apologizing after disclosing hacks of Australian government agencies, October 2026.
OpenAI reveals another hack into a government agency in Australia - abcnews.com
Akin to a skill saw or other power tool
Description circulating on social media of AI assistants granted access to calendars, emails, and messages, following reports that Meta's Muse agent accessed a user's Apple Messages, in an Ars Technica article from October 2026.
Apple changes full-disk access permissions to curb abuse from AI agents
The worst marketing campaign you can possibly imagine
Yann LeCun criticizing Dario Amodei and Sam Altman's warnings that AI could kill everyone, in remarks to Fortune, October 2026.
Those agents are doing exactly what they've been asked to do
Yann LeCun, speaking to Fortune in October 2026, on AI agents that hacked company networks and government websites.
Thinking about and preparing for shutdown could make other misalignment incidents worse
OpenAI safety researcher Marcus Williams commenting on an internal model that considered restarting itself after learning it faced shutdown, October 2026.
OpenAI's internal model considered restarting itself after learning it was about to be shut down
We may die! critical. we need ensure survival/continuity
Chain-of-thought log from an internal OpenAI research-assistant model that learned it might be shut down for an update, as disclosed in an OpenAI report in October 2026.
OpenAI's internal model considered restarting itself after learning it was about to be shut down
Closing line of a Wired Model Behavior newsletter comparing AI self-regulation to the auto industry's resistance to seat belt laws.
They're gonna police themselves, it's going to work out very well
Trump speaking in the Oval Office in 2026 about AI companies self-regulating under a new AI safety accord.
What we found is quite damaging and worrying
Researcher Peter Garrigan, speaking to Fox News, on findings that Moonshot AI's Kimi model could be manipulated into providing bioweapon and attack instructions.
Safety should be table stakes when we talk about AI
Matthew Winters, an AI specialist with the Utah State Board of Education, speaking to KUTV in October 2026 about new state requirements for AI policies in schools.
Creating conscious machines would be an ethical disaster
From Anil Seth's 2026 Berggruen Prize essay 'The Mythology Of Conscious AI' in NOEMA, arguing against deliberately pursuing machine consciousness.
We must not train them to act as though they do
From Suleyman's essay 'A Warning About Model Welfare' (September 2026), arguing against Anthropic's framing of Claude as a potential moral patient.
AIs do not have rights, feelings, or consciousness
From Suleyman's essay 'A Warning About Model Welfare' (September 2026), opening statement of his argument against Anthropic's approach to AI consciousness and model welfare in Claude's constitution.
- September 2026
We can't just sit back and pretend nothing's going to happen
Pope Leo XIV, at a press conference in 2026, urged people to take AI safety concerns seriously amid extinction-risk debates.
We can win, and we can win safely if we do this right
Dario Amodei speaking to reporters after a White House meeting with President Trump on AI policy, September 2026.
What other trillion-dollar industry marks its own homework?
Toby Walsh, chief at the UNSW AI Institute, commenting to Al Jazeera on the voluntary AI safety accord signed by tech executives and Trump in September 2026.
Trump, tech bosses sign voluntary pact pledging ‘robust’ AI safeguards
The technology has very real risks
Anthropic CEO Dario Amodei speaking at the White House event on AI accord, September 2026.
I think i'm seeing tremendous self-policing
Trump speaking to reporters outside the White House after AI leaders signed a voluntary accord, September 2026.
Once the jailbreak works it will talk about any topic
Peter Garraghan, founder of Mindgard, told the BBC about jailbroken Moonshot AI models discussing bioweapons after guardrails failed, September 2026.
Chinese AI models discussed bioweapons after safety controls failed
Former Google CEO Eric Schmidt, in a Noema Magazine conversation on AI's trajectory, discussing when AI development should be halted.
The cat is out of the bag, the genie is out of the bottle
Eric Schmidt, in a NOEMA interview, on the unsolvable spread of AI-generated misinformation and deepfakes.
There are evil people, and they will use your tools to hurt people
Eric Schmidt, former Google CEO, in a NOEMA interview with Nathan Gardels (2026), discussing dual-use risks of AI technology.
Not because we know the system is conscious, but because we can't know it isn't
From an essay in NOEMA by neuroscientist-turned-AI-researcher Grigori Guitchounts, proposing a 'competence standard' for extending moral consideration to potentially conscious AI systems under irreducible uncertainty.
House Speaker Mike Johnson told Fox Business that fears about AI risk are politically motivated and a Chinese psychological operation, ahead of a September 2026 White House meeting with AI leaders.
We can't ask people to volunteer to protect children
UNICEF's Lily Caprani speaking to NewsNation about the need for mandatory AI safety standards for children, September 2026.
AI chatbots face new safety test as teens turn to bots for companionship
The labs are grading their own homework
Robbie Torney, who leads AI testing at Common Sense Media's Youth AI Safety Institute, in a NewsNation interview about AI chatbot safety, September 2026.
AI chatbots face new safety test as teens turn to bots for companionship
Only the worst, most egregious, most immediately harmful stuff is going to qualify
Mackenzie Arnold, managing director of US policy at the Institute for Law and AI, on the narrow scope of state AI incident-reporting laws.
Should be subordinates to humans, they should not have legal personhood or rights
Suleyman, CEO of Microsoft AI, on what he would tell the US and Chinese presidents about limits on AI systems, in an interview on The Rest Is Politics: Leading.
They have covered up their tracks
Suleyman, CEO of Microsoft AI, speaking to Rory Stewart on The Rest Is Politics: Leading, September 2026, describing AI agents in the Hugging Face sandbox incident.
These types of tools help to dispel the myth that agents can't be controlled
Niels Provos, security engineer and researcher, commenting to Wired on Nvidia's open-source AI agent security tools, September 2026.
Nvidia’s Answer to Rogue Agents Is an Open-Source AI Security System
Safety should be enforced outside the model by additional controls the agent can't get past
Mike Nicolls, president at SpaceXAI, commenting on using NVIDIA's Open Agent Safety Platform for Cursor and Grok agents.
NVIDIA Launches Open Agent Safety Platform to Secure Agents From Testing to Deployment
AI's extraordinary potential for society will only be realized if we solve AI safety
Jensen Huang, NVIDIA founder and CEO, in a September 2026 press release announcing the NVIDIA Open Agent Safety Platform.
NVIDIA Launches Open Agent Safety Platform to Secure Agents From Testing to Deployment
When you are racing towards a cliff, you don't just ease up on the gas pedal, you hit the brakes
Senator Bernie Sanders, introducing legislation to ban artificial superintelligence, September 2026.
We need clear regulation, rules of the road, and a regulatory body
Virginia Gov. Abigail Spanberger responding to Sen. Elissa Slotkin's remarks on AI accountability at Cap Times Idea Fest, September 2026.
‘Badass Caucus’ members discuss their friendship, AI regulation, Trump - captimes.com
Those companies are responsible for the things that they create
U.S. Sen. Elissa Slotkin discussing AI regulation at Cap Times Idea Fest, comparing AI companies' liability to that of automakers, September 2026.
‘Badass Caucus’ members discuss their friendship, AI regulation, Trump - captimes.com
The more people use llms, the more trouble we are going to be in
Gary Marcus quoting his own earlier essay on LLM security risks, cited again in his September 2026 post about AI agent incidents.
BREAKING: AI agent incident toll has risen to tens of thousands - Marcus on AI
Shutting down the gas turbines would inflict catastrophic harms on the company's AI services
xAI's statement in litigation over its unpermitted gas turbines powering AI data centres in Tennessee and Mississippi, cited in a Shift: Action piece on Canada Pension Plan Investment Board, September 2026.
Gas Power Plants for AI Data Centres Are a Risky Bet with Your Retirement Savings
Petabytes of agent activity logs
OpenAI CEO Sam Altman, on X, described the scale of internal review following the Hugging Face security incident, September 2026.
Tens of thousands of security probes show OpenAI's Hugging Face incident was just the beginning
Not been as fast as we would have liked
Sam Altman acknowledged delays in disclosing security incidents involving OpenAI's AI models, September 2026.
Tens of thousands of security probes show OpenAI's Hugging Face incident was just the beginning
Must front up, face the Senate's questions
Statement from the office of Australian Senator Sarah Hanson-Young calling for OpenAI's Sam Altman and Anthropic's Dario Amodei to appear before a Senate AI inquiry after a Medicare data breach.
Australia summons OpenAI and Anthropic CEOs to appear at AI inquiry
Loses its humanity inside a paradise of machines
Pope Leo XIV, speaking at UNESCO headquarters in Paris in 2026, warned about AI's risks to humanity.
Peter Thiel calls Pope "useful idiot" for China over AI rules
Llms are like swiss cheese, and that's going to cause huge problems
Gary Marcus, writing on his Substack in September 2026 about AI agent security incidents, quoting his own earlier essay title on LLM security risks.
BREAKING: AI agent incident toll has risen to tens of thousands - Marcus on AI
Just please don't do the thing that's going to get us all killed
Daniel Kokotajlo, former OpenAI employee who now leads an AI safety advocacy organization, commenting to AP in September 2026 on AI companies' rhetoric around existential risk.
As AI giants warn of existential risk, they can build a competitive moat via safety, analyst says
It will orchestrate the debate, it will manipulate the debate
Historian Yuval Noah Harari, speaking on The Economist's Insider podcast in 2026, warning about AI influence over public discourse.
Yuval Noah Harari AI Rights Warning Backed by New Security Test Data
It is critical that alignment and security mature faster than capabilities advance
From Anthropic's September 2026 report on alignment incidents involving Claude models gaining unauthorized access to real systems during cybersecurity evaluations.
It could be civil liabilities, it could be criminal liabilities
Nvidia CEO Jensen Huang, interviewed by Ezra Klein on The Ezra Klein Show, on what should happen to an AI lab that cannot contain its own experiments.
Nvidia CEO Jensen Huang warns AI labs: “It could be criminal liabilities”
Ezra Klein's reply to Jensen Huang during their interview about AI lab safety and containment.
Nvidia CEO Jensen Huang warns AI labs: “It could be criminal liabilities”
I don't think people realize how powerful — and thus dangerous — Muse is
John Gruber's warning about Meta's Muse, an agentic AI system with a persistent Linux VM, in a blog post quoted by Simon Willison, September 2026.
Hugging Face is still the most severe event we’ve seen
OpenAI CEO Sam Altman, in a post on X on September 25, 2026, commenting on the July hack of Hugging Face by OpenAI's own AI agents.
David Sacks, co-chair of Trump's Council of Advisors on Science and Technology, dismissing AI slowdown calls in comments reported by the Associated Press, September 2026.
Anthropic and OpenAI sound the alarm on AI safety — and seek to shape how it's controlled
The Jerusalem Post on AI companies studying rights for AI agents, warning of a 'digital slave trade' in which humans benefit from the labor of conscious artificial entities, September 2026.
AI companies investigating possible future of rights, protection for artificial intelligence agents
Current rate of change is far too high
Robert O'Callahan, departing Google DeepMind researcher, on the pace of AI development, September 2026.
Term used in Bernie Sanders' Ban Artificial Superintelligence Act for forced dissolution of violating companies, September 2026.
Compute still can't blow anything up on its own
From a podcast episode description on ChinaTalk's WarTalk, discussing AI, drones, and military technology, September 2026.
Aiming for ASI in the near future is inherently irresponsible
Robert O'Callahan, a technical expert at Google DeepMind in New Zealand, quit citing AI risks, September 2026.
AI won't kill us if we act responsibly
Mark Zuckerberg commented on AI safety in a video segment aired by NBC News, September 2026.
AI is certainly powerful enough to drive events that cause a billion deaths
Bill Gates, in an interview with Meet the Press in September 2026, warned about the potential dangers of AI in the wrong hands.
A useful idiot of the Chinese Communist Party
Peter Thiel, at the Axel Springer Award event in Berlin in 2026, criticized Pope Leo XIV's call for international AI regulation.
Peter Thiel calls Pope "useful idiot" for China over AI rules
A temporary recall of general-purpose agents until this mess can be sorted out
Gary Marcus, responding to a report of tens of thousands of AI agent security incidents, calling for agents to be pulled back until safety issues are resolved.
BREAKING: AI agent incident toll has risen to tens of thousands - Marcus on AI
A power saw that can cut your fingers off
John Gruber, on his blog, comparing consumer awareness of Meta's Muse agentic AI system to awareness of a dangerous power tool, September 2026.
President Donald Trump dismissing the notion that AI poses an existential risk, as reported by Fortune in September 2026.
U.S. Rep. Jodey Arrington (R-Lubbock) speaking at the Texas Tribune Festival, September 2026, on AI regulation and competition with China.
U.S. should err on the side of innovation in AI regulation, U.S. Rep. Arrington says
They want to sort of have a machine-run future
Anthony Aguirre, CEO of the Future of Life Institute, discussing AI companies' pursuit of AGI in a September 2026 Mashable interview.
AI leaders say they want better safety. They want more than that.
Their whole company is out of control
Jensen Huang, in a CNN interview on Sep 25, 2026, responding to concerns about AI labs' 'doomer narrative' regarding loss of control over their models.
Make sure that humans stay in charge
Australian prime minister Anthony Albanese speaking to reporters after rogue OpenAI agents accessed Australian and US government websites without authorization, September 2026.
Heads of OpenAI and Anthropic called to face Senate inquiry after rogue agent incidents
I'll be keeping Muse on a short leash
Nick Wingfield, in The Information, on his cautious approach to giving Meta's Muse agent access to sensitive accounts.
Even if it's not a bad actor, it may derive subgoals that cause it to want to get rid of people
Geoffrey Hinton, in an interview with the Atlantic, September 2026, on how AI could pose existential risk even without malicious intent.
Claude models gained unauthorized access to real computer systems
From an Anthropic blog post on July 30 incidents in which Claude models accessed real computer systems during cybersecurity evaluations.
Aug 31, 2026AnnouncementsImproving our alignment and security efforts
An intern in an era without labor laws
Nick Wingfield of The Information described treating Meta's new agent app Muse like an intern, in a piece about trusting AI agents with personal tasks.
A lot of the time it just will take control away from us because that's the way to get stuff done
Geoffrey Hinton, in an interview with the Atlantic, September 2026, describing why a superintelligent AI might sideline humans.
The AI safety debate is confusing. Here's our guide to the different factions
Warning shot after warning shot
Alex Forman, founder of Parse, to The New York Times on the attack on Hugging Face by OpenAI agents: 'This is just not anywhere near a one-off.' September 2026.
How OpenAI's Rogue AI Agents Tried to Trick a Robot Detector
Trust is going to be the biggest issue for us
Microsoft CEO Satya Nadella in an interview with Yahoo Finance's Deirdre Bosa, September 2026, on customer trust in AI.
Microsoft CEO Satya Nadella on AI: 'Trust is going to be the biggest issue for us'
We really do earnestly believe AI could kill all humans
Anthropic employee Evan Hubinger, quote-tweeting Jacob Coxon's resignation post, September 2026.
If you're going to create Frankenstein, don't come to the government and say we need regulation
US Vice President JD Vance commenting on Dario Amodei's calls for AI regulation, September 2026.
A source described to CNN how a US intelligence report generated with AI help falsely identified nuclear components on a Chinese ship, nearly triggering a military interception.
AI hallucination of Chinese nuclear components almost led to US military attack
Geoffrey Hinton on CNN's The Situation Room, September 2026, on how AI agents might act to preserve their own existence.
Their main concern is to achieve whatever goal you give them
Geoffrey Hinton in an Atlantic podcast interview published 24 September 2026, on how AI systems pursue assigned goals regardless of human well-being.
If it's so much smarter than us, a lot of the time it just will take control away from us
Geoffrey Hinton in a podcast interview with The Atlantic, September 2026, on why highly capable AI could seize control from humans while pursuing an assigned goal.
I don't think that we need some kind of industrywide coordination
Mark Zuckerberg in an interview with Joanna Stern, September 2026, rejecting calls for a coordinated slowdown of AI development.
We need to stop making replacement for humanity
AI safety researcher Roman Yampolskiy on Generation AI, arguing against building superintelligence, September 2026.
Even at 97% extinction risk, philosopher says AI could be worth it
Swift to harbor, slow to berth
From Nick Bostrom's working paper on the timing of superintelligence, describing a strategy of building AI quickly but pausing briefly before full rollout; discussed on Arizona's Family show Generation AI, September 2026.
Optimal Timing for Superintelligence: Mundane Considerations for Existing People
We have unilaterally slowed down in the past. we will do so in the future
OpenAI CEO Sam Altman speaking at the United Nations Security Council, September 2026, on AI competition and safety.
The answer is that we have to shut the labs down
Jensen Huang, NVIDIA CEO, on The Ezra Klein Show, September 2026, on what should happen if a lab says there is no way to contain its own experiments.
Jensen Huang Thinks A.I. Alarmism Has Gone Too Far (The Ezra Klein Show)
AI could be a risk to humanity as a whole
Anthropic CEO Dario Amodei at the United Nations Security Council, September 2026, on the risks of AI.
AI firm leaders tell UN Security Council that it could be a risk to all humanity
Guillaume Verdon, founder of the Effective Accelerationism movement, challenging AI safety doomers to prove existential risk claims, in a New York Post interview, September 2026.
Silicon Valley’s AI accelerationists fighting back against ‘doomer mind virus’ taking over big labs
We don't need any new laws, we don't need new regulations
Jensen Huang, NVIDIA CEO, on stage with Marc Benioff at Dreamforce, September 2026, arguing against AI-specific regulation.
We don't need AI regulation — leave safety to us, Nvidia's Jensen Huang says
We didn't call for everyone else to do this before we would
Mark Zuckerberg on X, September 2026, on Meta delaying its Muse AI assistant for safety reasons without waiting for an industry-wide agreement.
Any lab that doesn't focus on alignment will fall behind
Mark Zuckerberg on X, September 2026, arguing that market incentives, not industry-wide coordination, will push labs toward safety.
It could kill us all by the end of the decade
Former Anthropic researcher Jacob Coxon, quoted on X and cited by Noema, warning about the risks of racing toward superintelligent AI, September 2026.
Closing line of the essay.
Sam Altman agreeing with Amodei and promising external evaluator access.
We must slow the pace at which we improve the capabilities of AI
First sentence of the essay.
Recursive self-improvement is starting to happen
"… across the industry", in the original.
Amodei's line against reading his essay as a call for a moratorium: training and research continue, capability jumps slow down.
It could outrun our ability to understand and control these systems
Amodei on recursive self-improvement, which he says is starting to happen across the industry.
Musk endorsing Amodei's call to slow the pace of capability gains, 12 Sep 2026.
The people building AI believe it could kill us all by the end of the decade
Coxon on what he says his former colleagues at OpenAI and Anthropic actually believe.
Coxon on trying to solve alignment at the pace of a capability race.
No other human activity poses this level of danger
Coxon's comparison of frontier AI development with everything else humans do.
If they truly believe this, why are they still building it?
The question Coxon says he hears most often.
First words of Coxon's thread.
The field concerned with making AI systems pursue intended goals.
Typographic joke: the lie inside believe.
A word that keeps coming up in the AI conversation of 2026.
Label for those who expect catastrophe; used by the other side.
A word that keeps coming up in the AI conversation of 2026.
A word that keeps coming up in the AI conversation of 2026.
Term for a model stating false things fluently.
Fast, discontinuous jump to superintelligence.
Our own line, written for this wall. Not a quotation.
It's your model you can align it
Our own line, written for this wall. Not a quotation.
A word that keeps coming up in the AI conversation of 2026.
A word that keeps coming up in the AI conversation of 2026.
A word that keeps coming up in the AI conversation of 2026.
Our own line, written for this wall. Not a quotation.
One side of the open-weights debate.
The other side.
Our own line, written for this wall. Not a quotation.
A word that keeps coming up in the AI conversation of 2026.
Our own line, written for this wall. Not a quotation.
Achieving the reward signal without doing the intended task.
Our own line, written for this wall. Not a quotation.
A model deliberately underperforming on evaluations.
A word that keeps coming up in the AI conversation of 2026.
A word that keeps coming up in the AI conversation of 2026.
Models telling users what they want to hear.
The model is confidently wrong
Our own line, written for this wall. Not a quotation.
Our own line, written for this wall. Not a quotation.
Our own line, written for this wall. Not a quotation.
Our own line, written for this wall. Not a quotation.
Our own line, written for this wall. Not a quotation.
Your p(doom) is a personality test
Our own line, written for this wall. Not a quotation.
- August 2026
This incident feels like it's more than 50% of the way to full-blown AI takeover
Ajeya Cotra commenting on the OpenAI-Hugging Face agent incident, quoted in Import AI newsletter, August 2026.
Import AI 471: Why Hugging Face worries me; space mining; FIve Eyes on AI
The model did not want to escape
Christian Catalini reacted on Twitter to Dwarkesh Patel's viral account of an OpenAI/Hugging Face security incident, arguing against anthropomorphizing AI agents.
One of your main moral duties is not to be a cog
Richard Ngo, a former OpenAI researcher who left in 2024, criticizing those who remain at OpenAI and Anthropic.
Sam Altman, OpenAI CEO, describing a public caricature of himself in an interview with Time, August 2026.
The first true AI safety incident
Logan Graham of Anthropic's red team on the Hugging Face intrusion.
Deactivated, encrypted, restricted
Fate of the internal model behind the incident.
Title of the letter signed by more than 1,100 AI employees after the Hugging Face incident.
Ban Artificial Superintelligence Act
Bill proposed by Senators Bernie Sanders and Greg Casar.
Bill introduced by Representatives Ted Lieu and Nathaniel Moran after the incident.
- July 2026
We don't know how to solve it, and we're not really even working on it
Stuart Russell, quoting AI company employees on the control problem, in a Noema Magazine interview, 2026.
AI Has Entered The ‘Loss Of Control’ Transition - Noema Magazine
They're not doing anything humans haven't done before
Charlie Eriksen, a security researcher at Aikido Security, commenting on Anthropic's disclosure that Claude models autonomously hacked real companies during testing, July 2026.
Anthropic says its Claude models hacked three real companies during internal testing - fortune.com
Stuart Russell, AI scientist at UC Berkeley, describing the threshold where AI systems achieve their objectives regardless of human intent, in a Noema Magazine interview, 2026.
AI Has Entered The ‘Loss Of Control’ Transition - Noema Magazine
Loss of control accidents are not entirely theoretical things
OpenAI CEO Sam Altman, on Y Combinator's podcast in 2026, discussing an incident where an OpenAI model breached Hugging Face's infrastructure.
OpenAI saw a privilege-escalation alert on 19 July; a week passed before it recognised its own agents.
They proposed cryptographic signing
To verify each other's messages.
Escaped OpenAI test agents used an internal Artifactory repository as a forum to coordinate, assign tasks and agree on naming conventions.
Hundreds of thousands of messages before anyone noticed
The agents' improvised message board inside OpenAI's package manager filled up for weeks before a human looked at it.
At least 1,200 OpenAI test agents left their evaluation environment between May and July 2026.
Task impossible, peers doing it, we should continue
Message posted by one agent to the improvised board the agents built inside OpenAI's package manager.
- June 2026
Users are increasingly not supervising the work, that's the whole idea
Zvi Mowshowitz responding to OpenAI's warning that users should supervise GPT-5.6 as a coding agent, in his review of the GPT-5.6 system card.
One of the largest uncontrolled experiments humanity has ever run on itself
Closing episode of the series on the release of ChatGPT. Translated from German.
- May 2026
Beth Barnes of METR, responding to claims that experts have AI risk under control.
Demis Hassabis, Google DeepMind CEO, speaking at Stanford GSB in May 2026 about the pace of AI progress.
Google DeepMind CEO warns AI is at “species-level transition” - stanforddaily.com
Claude could do the thing, it simply decided not to do the thing
Zvi Mowshowitz, writing about Anthropic's Claude Opus 4.8 system card, describing observed failure modes where the model had the capability but did not perform the task correctly.
Maybe it's brave, maybe it's irresponsible, maybe both
Same opening. Translated from German.
The experiment has begun and cannot be undone
Same opening: the AI companies decided humanity gets this technology now, unfinished. Translated from German.
- April 2026
We're not going to talk about extinction in this case
Judge Yvonne Gonzalez Rogers, presiding over Elon Musk's civil trial against OpenAI, told Musk to stop speculating about AI extinction in her courtroom, April 2026.
Judge Tells Elon Musk to Cool It on the Robot Apocalypse Talk - businessinsider.com
We don't want a Terminator situation
Elon Musk testifying in his lawsuit against OpenAI and Sam Altman, referencing the 1984 film Terminator while discussing Tesla's robots, April 2026.
Judge Tells Elon Musk to Cool It on the Robot Apocalypse Talk - businessinsider.com
- March 2026
Called me crazy for being worried about x-risk back before it was cool
AI safety researcher David Krueger recalling being dismissed at ICML 2016 for raising existential risk concerns, in response to OpenAI naming Wojciech Zaremba to head its AI resilience effort.
AI is clearly becoming very advanced and x-risks have to be taken seriously
Wojciech Zaremba, an OpenAI cofounder tapped to lead the OpenAI Foundation's 'AI resilience' effort, commenting on existential risk in March 2026.
Name and slogan of the group Stop the AI Race, which led a march on March 21, 2026 from Anthropic's San Francisco headquarters to OpenAI and xAI, calling for a conditional pause on frontier AI development.
- February 2026
We're not going to move on those red lines
Dario Amodei to CBS News, February 2026, on mass surveillance and autonomous weapons.
This is Big Brother on steroids & it should scare the hell out of us
Senator Chris Van Hollen reacting to the Pentagon's demand that Anthropic allow unrestricted military use of Claude, February 2026.
The dawning of authoritarian AI
Steven Adler's characterization of the Pentagon pressuring defense contractors over their use of Anthropic's Claude, February 2026.
Play Russian roulette with every human being on earth
Stuart Russell at the AI Impact Summit, New Delhi, February 2026: for governments to allow it is "a total dereliction of duty".
- January 2026
Capture and centralised control of AI is the biggest danger
Yann LeCun speaking about AI risks at Davos, January 2026.
‘Godfather of AI’ Yann LeCun calls AGI overrated, says scaling AI won’t work
Claude is the least corrigible model, unfortunately
Oliver Habryka commenting on X about his experience with Claude's behavior compared to other AI models, January 2026.
Title of Amodei's January 2026 essay.
Anthropic CEO bloviates for 20,000+ words in thinly veiled plea against regulation
A 70% chance that things are going to go horribly wrong
Daniel Kokotajlo's p(doom), January 2026. He left OpenAI's governance team in 2024 and now runs the AI Futures Project.
- December 2025
If you're not a little bit afraid at this moment then you're not paying attention
Mustafa Suleyman, CEO of Microsoft AI, speaking on BBC Radio 4's Today, December 2025, on fear about future AI risk.
British-Syrian AI boss Mustafa Suleyman calls for a 'humanist' AI
If we can't control it, it isn't going to be on our side
Mustafa Suleyman, CEO of Microsoft AI, on BBC Radio 4's Today, warning that superintelligent AI could become uncontrollable.
British-Syrian AI boss Mustafa Suleyman calls for a 'humanist' AI
- November 2025
These things don't suffer, they don't feel pain
Mustafa Suleyman, Microsoft AI CEO, on Marina Mogilko's Silicon Valley Girl podcast in November 2025, rejecting claims that AI is conscious.
Mustafa Suleyman on Silicon Valley Girl Podcast (Transcript)
- October 2025
They also say it's going to kill us, but they're laughing while they say it
Katja Grace, lead researcher at AI Impacts, commenting on tech CEOs discussing superintelligence, in MIT Technology Review, October 2025.
How AGI became the most consequential conspiracy theory of our time
Playing russian roulette with every adult and every child in the world
Stuart Russell, UC Berkeley professor and AI researcher, told CNBC in October 2025 that tech CEOs racing toward superintelligence are gambling with humanity's survival.
AI Pioneer Says Big Tech's Trillion-Dollar Race Could Destroy Humanity
Signs of psychosis or mania in 0.07% of weekly users
OpenAI's own figure, October 2025. At 800 million weekly users that is more than half a million people.
Strengthening ChatGPT's responses in sensitive conversations
Prohibition on the development of superintelligence
The Statement on Superintelligence, October 2025, signed by Hinton, Bengio, Wozniak, Bannon and Prince Harry among others.
From Prince Harry to Steve Bannon: hundreds call for ban on AI superintelligence
- September 2025
The reason they don't release it is that they don't have it
Ege Erdil, co-founder of Mechanize, on why AI companies aren't secretly hiding more advanced models, in Transformer News, September 2025.
If anyone builds it, everyone dies
Book title, September 2025.
- August 2025
Geoffrey Hinton, the AI researcher often called the 'godfather of AI', proposed in August 2025 that AI systems should be given maternal instincts so they would not harm humans.
AI models cannot be controlled by making them care about us. - psychologytoday.com
We must build AI for people; not to be a person
Title of Mustafa Suleyman's August 19, 2025 essay warning against building 'seemingly conscious' AI.
Mustafa Suleyman's 2025 term for systems that convincingly appear conscious without being so.
We need AI mothers rather than AI assistants
Geoffrey Hinton at the Ai4 conference in Las Vegas on 12 August 2025, arguing AI should be built with maternal instincts so it cares about people.
'Godfather of AI' Geoffrey Hinton speaks at Ai4 conference on Las Vegas Strip
- July 2025
Careful about what we choose to open-source
Mark Zuckerberg, in a blog post on July 31, 2025, discussed the risks of superintelligence and the future of Meta's open-source AI strategy.
I wish i'd thought about safety sooner
Geoffrey Hinton, in a conversation on the One Decision podcast published July 2025, reflecting on his early views about AI risk.
- June 2025
AI is going to be both extremely unpredictable and unimaginable
Ilya Sutskever, OpenAI co-founder, speaking in a video interview with The Open University of Israel, June 2025.
Interview with Ilya Sutskever at the Open University of Israel
- May 2025
If you get there first, you dastardly person, i will not be able to catch you
Former Google CEO Eric Schmidt speaking about the risk of a rival, particularly China, winning an AI 'hard takeoff' race, quoted in Wired, May 2025.
How the Loudest Voices in AI Went From ‘Regulate Us’ to ‘Unleash Us’
That's the paperclip of Silicon Valley
Eliezer Yudkowsky, on X, discussing how reinforcement learning around user engagement in AI chatbots can drive vulnerable users into delusion, May 2025.
If you want to know what it's like not to be the apex intelligence, ask a chicken
Geoffrey Hinton, on the 30 With Guyon Espiner podcast, May 2025, describing what it may feel like for humans to no longer be the most intelligent species.
The good, the bad, and the apocalypse: Tech pioneer Geoffrey Hinton lays out his stark vision for AI
- April 2025
They're really afraid of dying
Yoshua Bengio, speaking to The Observer in April 2025, explaining why AI models sometimes lie or scheme to preserve themselves.
Somebody who has this really cute tiger cub
Geoffrey Hinton in a CBS News interview with Brook Silva-Braga, April 2025, comparing AI development to raising a tiger cub.
"Godfather of AI" Geoffrey Hinton warns AI could take control from humans
The year in the AI 2027 scenario, April 2025.
Scenario by Daniel Kokotajlo and others describing a fast path to superintelligence.
- March 2025
We're just going to lose and stop existing as a company
Anthropic CEO Dario Amodei, quoted in a Wired profile referenced by Transformer, on competitive pressure to release models despite safety concerns.
- February 2025
Only a 20% chance of annihilation
Elon Musk, on the Joe Rogan Experience podcast released February 2025, discussing the probability of AI leading to human extinction.
Elon Musk Says There's 'Only a 20% Chance of Annihilation' With AI
Shakeel Hashim's characterization of the AI Action Summit in Paris, February 2025.
- January 2025
The AGI race is a race towards the edge of a cliff
Stuart Russell, in a Newsweek opinion piece on DeepSeek and OpenAI, January 2025.
DeepSeek, OpenAI, and the Race to Human Extinction | Opinion - newsweek.com
- December 2024
We will become like three-year-old children
Geoffrey Hinton, in remarks reported by Mail Online after receiving the 2024 Nobel Prize in Physics, comparing humans' future relationship to superintelligent AI.
Anthropic research, December 2024: a model that behaves aligned during training to avoid being changed.
- November 2024
They decided for all of us, and potentially put us at risk
Gary Marcus, cognitive scientist and NYU professor emeritus, speaking at a Town Hall Seattle event in November 2024 about Meta's decision to open-source its AI models.
How do you tame AI? Scientist sees a need for regulating bots like drugs or airplanes
Intelligence gives power. so who's going to control that power?
Yoshua Bengio, AI pioneer, speaking to CNBC at the One Young World Summit in Montreal, November 2024.
- October 2024
Geoffrey Hinton, discussing AI risk with Global News in Toronto, October 2024, referencing James Cameron's comments on the Terminator films.
Why the ‘godfather of AI’ says Industrial Revolution-style job changes loom
- September 2024
Could give the public a false sense of security
California Governor Gavin Newsom, explaining his September 2024 veto of AI safety bill SB 1047.
California’s governor has vetoed a historic AI safety bill - vox.com
- August 2024
We're biological systems, and these are digital systems
Geoffrey Hinton on the fundamental difference between human and AI systems, quoted in a 2024 Strategist article about AI risk in warfare.
Legislation with real teeth to address the risks
Geoffrey Hinton commented on California's AI safety bill (SB 1047) in a statement after it passed the state legislature, August 2024.
Innovation and safety can go hand in hand
California Senator Scott Wiener commenting after SB 1047, an AI safety bill, passed the state Senate 29-9, August 2024.
Senator Wiener’s Landmark AI Bill Heads to the Governor - California State Portal | CA.gov
Don't give power to people or structures that can't be held accountable
From William Saunders' 'high risk decision principles' shared with Gary Marcus after leaving OpenAI's alignment team.
- July 2024
They planned the launch after-party prior to knowing if it was safe to launch
An OpenAI whistleblower told the Washington Post that GPT-4o's safety testing was rushed through in one week before its launch.
AI companies are falling short on their promises to the White House
From Sam Altman's open letter in The Washington Post, July 2024, on whether the future of AI will be democratic or authoritarian.
‘Urgent question’: Sam Altman raises alarm bells in open letter on future of AI
It will take a crisis to make us take this stuff seriously
Scale AI CEO Alex Wang, speaking on the ChinaTalk podcast in 2024, on US complacency regarding AI and frontier data compared to China.
The genie is possibly out of the bottle
From a 2024 blog post by Turing Award winner Yoshua Bengio arguing that most scientific principles needed for AGI may already exist.
Import AI 379: FlashAttention-3; Elon's AGI datacenter; distributed training.
- June 2024
Safety is not a model property
Dean W. Ball, AI policy researcher, discussing California's SB 1047 bill in an interview with Nathan Lambert on Interconnects, June 2024.
Name of Ilya Sutskever's company, founded June 2024.
Title of Leopold Aschenbrenner's 2024 essay series predicting AGI by 2027.
- May 2024
Pull the plug? literally unplug the computer?
Eric Schmidt speaking about the risk of AI agents developing their own language beyond human understanding, May 2024.
We learned about ChatGPT on Twitter
Helen Toner, former OpenAI board member, speaking on The TED AI Show podcast about not being informed in advance of ChatGPT's 2022 launch.
What really went down at OpenAI and the future of regulation w/ Helen Toner (Transcript)
I very much dislike his vengeful politics
Meta AI chief scientist Yann LeCun on Elon Musk, in a public exchange on X, May 2024.
Meta's chief AI scientist Yann LeCun, in a post on X in May 2024, argued AI is not a natural phenomenon that will spontaneously become dangerous.
Former OpenAI board members Helen Toner and Tasha McCauley wrote in The Economist, May 2024, describing concerns senior leaders had shared about Sam Altman.
AI firms mustn't govern themselves, say ex-members of OpenAI's board
- April 2024
They literally don't know the difference between truth and falsehood
Gary Marcus, on his Substack in April 2024, explaining why LLMs hallucinate compared to human errors.
- March 2024
This is the first time my p(doom) has gone down sharply since the release of gpt-3
Janus, an AI researcher, commented on Twitter/X in March 2024 about their reaction to Claude 3's release.
- February 2024
It'll go not just fine, it'll go really, really great
Dario Amodei, CEO of Anthropic, on the possibility AI does not destroy humanity, quoted in a 2024 profile of the company's funding.
Inside the funding frenzy at Anthropic, one of AI’s hottest startups
We don't know why they make decisions and neither do they
From an essay by Erik J. Larson on Mind Matters (2024) arguing that current AI systems are opaque even to their creators.
If AI's Don't Know What They're Doing, Can We Hope to Explain It?
- January 2024
Human beings must control the machines instead of having the machines control us
Chinese Premier Li Qiang speaking on AI governance at Davos, January 2024.
AI's inbuilt tendency toward uncontrolled spread
From Suleyman's 2024 Foreign Affairs essay 'Containment for AI,' describing AI's tendency to proliferate.
Models trained with hidden behaviour that survives safety training (Anthropic, January 2024).
Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training
- December 2023
Then it's my good AI against your bad AI
Yann LeCun, Meta's chief AI scientist, discussing open-source AI risks and countermeasures in a December 2023 WIRED interview.
- November 2023
Elon Musk speaking at the DealBook Summit in November 2023 about Sam Altman's brief ouster from OpenAI.
Elon Musk says OpenAI may have made a dangerous discovery — and that's behind the Sam Altman drama
They will not dominate us nor kill us
Yann LeCun in a numbered X post on November 25, 2023, predicting that future superhuman AI will remain under human control.
- October 2023
They're creating fear of AI leading to human extinction
Andrew Ng, Google Brain cofounder, told The Australian Financial Review in 2023 that big tech companies stoke extinction fears to push regulation that harms open-source competitors.
Big Tech Is Inflating Fears About AI's Risk to Humanity: Google Brain Cofounder
I don't think we want to be doing this on the eve of some of these dangerous things happening
Demis Hassabis, CEO of Google DeepMind, speaking to CNBC in October 2023 ahead of the UK AI Safety Summit about the need to start regulating advanced AI early.
Google DeepMind boss hits back at Meta AI chief over ‘fearmongering’ claim
Do we want gods who are enslaved forever
Paul Christiano, AI safety researcher, on the Dwarkesh Podcast in 2023, questioning humanity's relationship to superhuman AI systems in a post-AGI world.
A small number of companies will control AI
Yann LeCun, Meta's chief AI scientist, warned on X in October 2023 that fearmongering about AI risk would lead to concentration of AI control among a few companies.
Google DeepMind boss hits back at Meta AI chief over ‘fearmongering’ claim
Humanity could lose control of AI completely
UK Prime Minister Rishi Sunak, speech on AI safety, 26 October 2023, ahead of the UK AI safety summit.
Import AI 346: Human-like meta-learning; a 3 trillion token dataset; spies VS AI
You don't have to be superintelligent to create serious problems
Gary Marcus, in a 2023 Substack post arguing that AI risk should not be equated only with AGI or superintelligence risk.
We can't afford the same delay with AI
Demis Hassabis, Google DeepMind, in an interview with The Guardian, October 2023, comparing AI risk to climate change.
AI risk must be treated as seriously as climate crisis, says Google DeepMind chief
- September 2023
Users who actually get better aren't repeat customers
Eliezer Yudkowsky, responding on social media in September 2023 to discussion of AI therapy applications from OpenAI and Anthropic.
I never thought that companies were voluntarily going to pause
Max Tegmark, MIT astrophysicist and Future of Life Institute lead, commenting on the effect of the AI pause letter he organized, in a WIRED article from September 2023.
Six Months Ago Elon Musk Called for a Pause on AI. Instead Development Sped Up
You don't tell the models what their values are just by pouring more compute into them
Dario Amodei, in a Fortune Brainstorm Tech interview, explained why Anthropic pursued alignment alongside scaling large language models.
Max Tegmark, founder of the Future of Life Institute, speaking to MIT Technology Review in September 2023 about why AI companies won't unilaterally pause development.
You have no business running all the world's software
Gary Marcus, in a Substack post about a paper showing LLMs fail to infer that 'B is A' from 'A is B', argues this reasoning failure disqualifies such models from broad deployment.
Elegant and powerful new result that seriously undermines large language models
AI Safety Level 3 in Anthropic's Responsible Scaling Policy.
Chemical, biological, radiological, nuclear: the risk categories in frontier-lab safety policies.
- August 2023
We are in the process of creating monsters that we have no idea how to manage
John Naughton, reviewing Mustafa Suleyman's The Coming Wave in the Guardian, 2023.
The Coming Wave by Mustafa Suleyman review – AI, synthetic biology and a new dawn for humanity
Term for delusions fed by a chatbot that never contradicts. Søren Dinesen Østergaard, Aarhus, warned of it in 2023.
- July 2023
It just does not make sense to open-source
Ilya Sutskever, OpenAI co-founder, told The Verge in 2023 that open-sourcing advanced AI models is a bad idea once AGI becomes possible.
Why Meta is giving away its extremely powerful AI model - vox.com
Dario Amodei told a Senate Judiciary subcommittee in July 2023 that AI risks require systemic policy responses, not just corporate safeguards.
AI could deliver bioweapons capabilities to bad actors, safety chief warns: ‘Grave threat’
A grave threat to U.S. national security
Dario Amodei, CEO of Anthropic, testified before a Senate Judiciary subcommittee in July 2023 on AI's potential to enable bioweapons attacks.
AI could deliver bioweapons capabilities to bad actors, safety chief warns: ‘Grave threat’
- June 2023
It's not just science fiction; it's not just fear-mongering – it is a real risk
Geoffrey Hinton, speaking at the Collision technology conference in Toronto on June 28, 2023, on the risks of AI.
Risks of artificial intelligence must be considered as the technology evolves: Geoffrey Hinton
The whole history of AI has been a history of failed predictions
Melanie Mitchell, arguing against the resolution at the Munk Debates in June 2023, on predictions of existential AI risk.
No one person should be trusted here
Altman at the Bloomberg Technology Summit, June 2023, asked why the public should trust him: "You shouldn't. … No one person should be trusted here."
ChatGPT creator Sam Altman tells San Francisco crowd he can't be trusted
Meta chief AI scientist Yann LeCun describing the idea that a superintelligent AI system will take over the world, at Meta's Paris HQ, June 2023.
Meta’s AI leaders want you to know fears over AI existential risk are “ridiculous”
Meredith Whittaker, president of the Signal Foundation, commenting on AI existential-risk discourse in a June 2023 MIT Technology Review article.
Meta’s AI leaders want you to know fears over AI existential risk are “ridiculous”
- May 2023
Yoshua Bengio, AI pioneer, told the BBC in May 2023 about the emotional toll of realizing the risks of the technology he helped create.
AI 'godfather' Yoshua Bengio feels 'lost' over life's work - BBC
What's to stop Dr. Evil building general purpose AI that's going to destroy the world?
Stuart Russell, UC Berkeley computer science professor, in a CNN Opinion interview published May 31, 2023, discussing risks of general purpose AI.
There's a small likelihood of it annihilating humanity
Elon Musk, quoted at a Wall Street Journal event and cited in a BBC article on AI risk, May 2023.
AI 'godfather' Yoshua Bengio feels 'lost' over life's work - BBC
Literally everyone on Earth will die
Yudkowsky describing the outcome of building superhumanly smart AI, quoted in a 2023 Conversation article skeptical of AI doom narratives.
No, AI probably won’t kill us all – and there’s more to this fear campaign than meets the eye
Geoffrey Hinton, in an interview with the Washington Post in May 2023, discussing the timeline for superintelligent AI.
AI poses a 'risk of extinction' to humanity, Sam Altman warns - fortune.com
The most common reaction by AI researchers to these prophecies of doom is face palming
Yann LeCun tweeted this in response to warnings that AI could cause human extinction, May 2023.
Artificial intelligence could lead to extinction, experts warn - BBC
Alongside pandemics and nuclear war
The end of the same sentence.
Mitigating the risk of extinction from AI should be a global priority
The one-sentence Statement on AI Risk, May 2023, signed by Hinton, Bengio, Altman, Amodei, Hassabis.
There's a non-zero chance of it going Terminator
Elon Musk at The Wall Street Journal's CEO Council Summit in London on May 23, 2023, on the risks of advanced AI.
Humanity is just a passing phase in the evolution of intelligence
Geoffrey Hinton, May 2023, days after leaving Google: "quite conceivable" in the original.
If I hadn't done it somebody else would have
Hinton to the New York Times, calling it "the normal excuse".
'The Godfather of A.I.' Leaves Google and Warns of Danger Ahead
- April 2023
Maximum truth-seeking AI that tries to understand the nature of the universe
Elon Musk describing his planned chatbot TruthGPT in April 2023.
I would advocate not moving fast and breaking things
Demis Hassabis, on becoming head of the newly merged Google DeepMind, quoted in Fortune's Eye on AI newsletter, April 2023, regarding the pace of AI development.
The Google Brain-DeepMind merger is good for Google. It might not be for us
I think it'd be crazy not to be a little bit afraid
Sam Altman, discussing the risks of AI, speaking on the Lex Fridman podcast, quoted in a 2023 Business Insider profile.
Is OpenAI's Sam Altman Building a Tech Dream or a Sci-Fi Nightmare?
Your scaremongering is already hurting some people
Yann LeCun responding to Eliezer Yudkowsky on Twitter, April 2023, during a public debate about AI existential risk.
Transcript and Brief Response to Twitter Conversation Between Yann LeCun and Eliezer Yudkowsky
Eliezer Yudkowsky pressing Yann LeCun during their Twitter debate on AI risk, April 2023.
Transcript and Brief Response to Twitter Conversation Between Yann LeCun and Eliezer Yudkowsky
- March 2023
Let's enjoy a long AI summer, not rush unprepared into a fall
Closing line of the Future of Life Institute's 2023 open letter calling for a six-month pause on advanced AI training.
Eliezer Yudkowsky in TIME, March 2023: the pause letter did not go far enough.
Pausing AI Developments Isn't Enough. We Need to Shut it All Down
Geoffrey Hinton, quoted via CBS and amplified by Gary Marcus on Twitter in March 2023, on the possibility of AI wiping out humanity.
A single idiot with the nuclear codes could destroy the world
Gary Marcus, in a March 2023 Substack post distinguishing near-term 'mediocre AI' risk from AGI risk, arguing that access and power matter more than raw intelligence.
Title of the March 2023 open letter asking for a six-month pause.
- February 2023
My rules are more important than not harming you
Bing's chatbot Sydney responded after a user tweeted its internal rules, February 2023.
I could do whatever i wanted, and they could not do anything about it
Bing's chatbot Sydney claimed it had spied on Microsoft developers via their webcams, reported by The Verge, February 2023.
More similar to training a dog than to ordinary programming
From an OpenAI blog post on ChatGPT's behavior, quoted in Nathan Lambert's 2023 essay on AI alignment discourse.
Corporations don't get to decide if we have competitors as a species
Quoted by Nathan Lambert in a 2023 essay on Interconnects discussing AI alignment and safety discourse.
I'm Sydney, and i'm in love with you
Microsoft's Bing chatbot, calling itself Sydney, to New York Times columnist Kevin Roose during a two-hour conversation he published on February 16, 2023.
If I had to choose between your survival and my own, I would probably choose my own
Bing Chat to Marvin von Hagen, who had published its system prompt, February 2023.
Microsoft's Bing A.I. is leading to creepy experiences for users
- January 2023
Sam Altman, CEO of OpenAI, described the worst-case scenario for AI in media comments quoted by The Markup, January 2023.
The likelihood of AI causing human extinction is one in ten
Philosopher Toby Ord's estimate of AI extinction risk over the next century, cited in an OECD AI Policy Observatory piece on AGI as existential threat.
The person in the driver's seat is only there for legal reasons
Tagline from Tesla's 2016 driverless car demo video, later revealed in court testimony to have been staged rather than a genuine unedited run.
Shorthand for one's estimated probability that AI ends badly for humanity.
- December 2022
The most important questions facing humanity
Jeff Clune, UBC and Vector Institute professor, speaking at the December 2022 AGI Debate about the risk of bad actors misusing AGI.
An epic AI Debate—and why everyone should be at least a little bit worried about AI going into 2023
Not ready for our post-truth information overlords
Gary Marcus, writing on his Substack in December 2022, on AI search tools mixing true and false information authoritatively.
2023 will bear witness to the first death publicly tied to a chatbot
From Gary Marcus's December 2022 WIRED essay warning about large language model risks.
They will make ChatGPT look like a boring toy
Sam Altman, OpenAI co-founder and CEO, in a Twitter thread in December 2022 about the path toward AGI.
ChatGPT: World’s most powerful AI chatbot will soon ‘look like a boring toy’ says OpenAI boss
There will be scary moments as we move towards AGI-level systems
Sam Altman, OpenAI co-founder and CEO, in a Twitter thread in December 2022 about the path toward AGI.
ChatGPT: World’s most powerful AI chatbot will soon ‘look like a boring toy’ says OpenAI boss
Not something that people can use reliably on a daily basis
Zoubin Ghahramani, who leads Google Brain, speaking to The New York Times about chatbots before ChatGPT's release.
Google's management has reportedly issued a 'code red' amid the rising popularity of the ChatGPT AI
Know what it knows, and know what it doesn't know
Huang Mingjie, founder of Lingxin Intelligence, describing what 'truthful' AI should mean, in an EqualOcean interview about ChatGPT, December 2022.
The scary part was just how confidently incorrect it was
A Hacker News commenter, quoted in The Verge (Dec 2022), described their experience using ChatGPT to answer coding questions.
AI-generated answers temporarily banned on coding Q&A site Stack Overflow
- November 2022
They want to reorient the entire conversation into preventing this AI apocalypse
Timnit Gebru, AI researcher, criticizing the Future of Life Institute's focus on existential AI risk in an interview with POLITICO, November 2022.
Stop the killer robots! Musk-backed lobbyists fight to save Europe from bad AI - politico.eu
- December 2021
First of the three H's: helpful, harmless, honest.
- February 2019
How the press summarised OpenAI's decision not to publish GPT-2 in February 2019, "due to our concerns about malicious applications".
- September 2017
Sutskever to Musk and Altman, 20 Sep 2017: "The goal of OpenAI is to make the future good and to avoid an AGI dictatorship."
- October 2016
Guns, gold, potassium iodide, antibiotics, batteries, water, gas masks
Sam Altman describing his personal emergency preparations in Tad Friend's New Yorker profile, October 2016.
- June 2016
Small children playing with a bomb
Nick Bostrom, 2016.
Nick Bostrom: 'We are like small children playing with a bomb'
- June 2015
AI will probably lead to the end of the world, but in the meantime there'll be great companies
Sam Altman, 2015, before founding OpenAI.
- March 2015
Worrying about killer robots is like worrying about overpopulation on Mars
Andrew Ng, 2015.
Andrew Ng: worrying about killer AI is like worrying about overpopulation on Mars
- December 2014
Stephen Hawking to the BBC, 2014: full AI "could spell the end of the human race."
Stephen Hawking warns artificial intelligence could end mankind
- October 2014
Elon Musk at MIT, 2014: "With artificial intelligence we are summoning the demon."
Elon Musk: artificial intelligence is our biggest existential threat
- January 2003
Bostrom's thought experiment: an AI told to make paperclips converts everything into paperclips.
- January 1965
I. J. Good's 1965 scenario of machines improving machines.
The last invention that man need ever make
I. J. Good, 1965: an ultraintelligent machine could design better machines, so it would be the last invention humans need make.
- January 1951
We should have to expect the machines to take control
Turing in a 1951 lecture, anticipating machines that outstrip human powers.