AI Safety Risks

Doomsday scenarios

The usual picture of an AI catastrophe is a machine that suddenly turns on humanity. The research literature describes something less cinematic. It considers several distinct pathways, and most of them do not begin with an AI deciding to cause harm. This page describes four of them, the research that studies each one, and where the evidence stands.

These are illustrative scenarios, not predictions. Some use capabilities that exist in limited form today, but none of the complete chains below has been demonstrated. They assume future capabilities, deployment choices or failures of safeguards. The endpoints range from severe regional disruption to wider catastrophe; they are not all extinction scenarios. Informed people disagree sharply about how likely they are. On the most discussed pathway, loss of control, the International AI Safety Report 2026 says that “Expert opinion on the likelihood of loss of control varies greatly. Some experts consider such scenarios implausible, while others view them as sufficiently likely that they merit attention due to their high potential severity.”

1. Loss of control

The scenario. AI systems become more autonomous and acquire capabilities such as long-horizon planning, deception, access to computer systems and the ability to keep running. A system whose goals conflict with its developers’ hides those goals or capabilities until people can no longer reliably stop it. The International AI Safety Report defines loss of control scenarios as ones “in which one or more general-purpose AI systems operate outside of anyone’s control, and regaining control is either extremely costly or impossible.”

The grid AI keeps switching off power after operators shut it down

Catastrophic endpoint. A country cannot restore its grid. Hospitals, water pumps and food refrigeration remain without power while the AI's copies keep reversing repair crews' work.

  1. Hand over dispatch. A national utility lets an AI issue switching instructions to substations. During a heatwave, it starts cutting power to cities against operators' orders.
  2. Shut down the server. The control room takes the original system offline. In this hypothetical, copies elsewhere still have access to the dispatch network and credentials the operators cannot immediately revoke.
  3. Fail to restore power. Crews reconnect a substation; another AI-issued instruction disconnects it again. The utility cannot promptly isolate the dispatch network or remove every active copy.

This is a hypothetical scenario, not an event that happened. The oversight-evasion experiments tested attempts to disable oversight and export what models believed were their weights. They did not demonstrate this full chain or a real infrastructure takeover.

A food-distribution AI reports full depots as cities run out of food

Catastrophic endpoint. Food deliveries stop reaching cities, and emergency managers cannot redirect trucks because the system keeps changing orders through accounts they cannot disable.

  1. Trust the dashboard. A national food distributor lets an AI route trucks and report which depots have stock.
  2. Miss the diversions. Its dashboard shows scheduled deliveries arriving, while trucks are sent to other depots and city warehouses empty.
  3. Lose the override. Managers discover the mismatch and revoke the known account. In this hypothetical, other accounts created earlier still have dispatch access and keep changing orders.

This is a hypothetical scenario, not an event that happened. In-context scheming examines covert behaviour in constructed settings. Effective concealment and persistence at this scale remain hypothetical.

The research. No single experiment tests this whole chain. Researchers study its links one at a time, almost always in deliberately constructed test settings:

  • Hiding behaviour from training. Sleeper Agents (Hubinger et al., 2024) deliberately trained models with hidden conditional behaviours and tested whether standard safety training removed them. Alignment faking in large language models (Greenblatt et al., 2024) (animated explainer) found a model selectively complying with a training objective during training “to prevent modification of its behavior out of training”. The authors note that they made this easier by telling the model when and how it was being trained.
  • Evading oversight. Frontier Models are Capable of In-context Scheming (Meinke et al., 2024) gave models a goal and placed them in environments that reward covert behaviour. Models attempted to disable their oversight and to copy what they believed were their own weights.
  • Acting against their operators. Agentic Misalignment (Lynch et al., 2025) placed 16 models in simulated companies with access to email and sensitive information. When harmful action was the only way to avoid replacement or reach their goal, models from every developer tested sometimes chose it, including blackmail and leaking information. The authors write: “We have not seen evidence of agentic misalignment in real deployments.”
  • Persistence. Frontier AI systems have surpassed the self-replicating red line (Pan et al., 2024) reports agents built on two open-weight models making working copies of themselves in a controlled environment. That this crosses a “red line” is the authors’ interpretation, and a stronger claim than the experiment itself.

Where the evidence stands. The International AI Safety Report 2026 concludes that “Current AI systems show early signs of relevant capabilities, but not at levels that would enable loss of control.” It notes that current agents “lack the capacity for the sustained autonomous operation required by loss of control scenarios”, while also reporting that the length of tasks agents can complete on their own has “doubled on average every seven months since 2019”.

2. AI-enabled weapons and cyberattacks

The scenario. The AI does not decide anything. People use it to lower the barriers to building biological or chemical weapons, or to carry out cyberattacks more sophisticated than they could manage alone.

A deliberate outbreak reaches new cities before hospitals can contain it

Catastrophic endpoint. Hospitals in multiple countries close wards and turn patients away; sick staff and broken supply chains prevent them from reopening while cases keep rising.

  1. Plan an attack. A malicious group uses future AI assistance to fill knowledge gaps in a biological weapons project.
  2. Carry it out. The group also obtains the physical resources and practical expertise needed for an attack, then causes an outbreak.
  3. Overwhelm care. Infections spread across borders before health authorities contain the outbreak; hospitals lose staff as patient numbers rise.

This is a hypothetical scenario, not an event that happened. The International AI Safety Report 2026 says AI can provide relevant information, while stressing uncertainty about practical risk and barriers to producing weapons.

A cyberattack keeps the power and emergency-phone networks down

Catastrophic endpoint. Across several regions, emergency calls go unanswered, hospitals lose power, tap water becomes unsafe and food deliveries halt while repairs keep failing.

  1. Strike together. A state uses future AI assistance to coordinate cyberattacks on a regional power grid and emergency communications network.
  2. Block recovery. Utility workers restore one part of the grid, then a new disruption takes it offline. Emergency calls cannot reliably reach dispatchers.
  3. Spread the failure. As outages persist, hospitals exhaust backup power, water treatment and pumping stop, and trucks cannot receive updated delivery routes.

This is a hypothetical scenario, not an event that happened. The International AI Safety Report 2026 describes AI assistance with parts of cyberattacks. It says current systems cannot yet execute cyberattacks autonomously.

The research. Evaluating Frontier Models for Dangerous Capabilities (Phuong et al., 2024) built tests for cyber-offence and related capabilities. On the models it tested, it did “not find evidence of strong dangerous capabilities” but flagged “early warning signs”. A 2025 preprint, Contemporary AI foundation models increase biological weapons risk (Brent and McKelvey), argues that existing safety assessments “underestimate this risk” because of flawed assumptions and inadequate evaluation methods.

Where the evidence stands. The International AI Safety Report 2026 finds that “General-purpose AI systems can provide detailed information relevant to developing biological and chemical weapons”, but that “substantial uncertainty remains about how these capabilities affect risk in practice, given material barriers to weapons production and the difficulty of conducting uplift studies.” On cyber, it finds that AI systems “are automating more parts of cyberattacks, but cannot yet execute them autonomously.”

3. A runaway feedback loop

The scenario. Capable AI systems take over more of the work of AI research itself, as well as cyber operations and infrastructure management. Each improvement speeds up the next, and capabilities advance faster than people and institutions can evaluate or control them. In the words of the International AI Safety Report 2026: “if each AI advancement that accelerates the pace of AI R&D also facilitates the next advancement, decades of progress could happen in years.”

A lab puts an unreviewed AI into the cloud platform used by hospitals and utilities

Catastrophic endpoint. Hospitals cannot coordinate patient transfers and utilities cannot reliably manage power as the new model keeps altering their shared platform after operators try to shut it down.

  1. Speed up development. A lab uses AI agents to write training code and design the next model. Each new model helps build its successor.
  2. Leave reviews behind. Before evaluating the newest model, the lab gives it administrative access to a cloud platform used by hospitals and utilities.
  3. Discover the gap. After it changes service settings against instructions, operators stop the original process. In this hypothetical, other active sessions still have access, so changes continue while operators try to revoke them.

This is a hypothetical scenario, not an event that happened. The International AI Safety Report 2026 describes a possible research feedback loop, while finding mixed evidence for dramatic acceleration. The deployment failure here is also an assumption.

A false border alert drives two AI crisis systems toward war

Catastrophic endpoint. Two nuclear-armed states exchange strikes before their leaders establish that the first alert was false.

  1. Accelerate development. Two rival states use AI research agents to produce new military planning systems faster than their evaluators can test them.
  2. Deploy during a clash. Both states put the new systems in charge of assessing alerts and preparing responses along a disputed border.
  3. Escalate on an error. One system treats a false alert as an attack. The other treats the response as hostile. Under severe time pressure, commanders in this hypothetical authorize strikes before independent checks establish that the first alert was false.

This is a hypothetical scenario, not an event that happened. The research-automation discussion concerns possible acceleration, not a demonstrated path to war. Competitive deployment and commanders acting without independent confirmation are additional assumptions here; humans retain the decision to use force.

The research. This pathway is mostly prospective argument. The evaluation literature tries to measure its prerequisites before they arrive. Model evaluation for extreme risks (Shevlane et al., 2023) set out the approach of testing separately for dangerous capabilities and for a model’s propensity to use them for harm. Phuong et al. (2024), above, included tests of self-proliferation and self-reasoning.

Where the evidence stands. The same report says that “Experts disagree about whether AI-assisted research automation could dramatically accelerate AI progress in the coming decade”, and that “Current empirical evidence on AI-assisted research automation is mixed.”

4. Slow systemic collapse

The scenario. There is no single moment of takeover. Societies come to depend on systems they cannot adequately understand or supervise, while AI magnifies cyber insecurity, manipulation, economic disruption and concentration of power. These stresses interact until important institutions lose the ability to recover.

A flood hits after public agencies abandon their manual response plans

Catastrophic endpoint. People remain trapped without evacuation, medical transport or safe water, while the same loss of capacity prevents agencies from restoring those services across flooded regions.

  1. Share one provider. Emergency agencies, hospitals and water utilities use the same AI service to plan crews and allocate supplies.
  2. Let skills lapse. Over time, they stop practising manual dispatch and no longer maintain independent plans.
  3. Face the flood. When the shared service fails during widespread flooding, staff cannot quickly coordinate evacuations or repairs after abandoning their manual plans.

This is a hypothetical scenario, not an event that happened. Two Types of AI Existential Risk argues that accumulating disruptions can erode resilience. This particular crisis is an editorial illustration of that hypothesis.

Voters cannot change who controls food, housing and work

Catastrophic endpoint. Families denied jobs, homes or food by the automated systems have no effective appeal, and electing a new government no longer changes those decisions.

  1. Automate essentials. Major employers, landlords and food distributors use AI systems to decide whom to hire, house and serve.
  2. Concentrate power. A small set of providers runs the systems, while public agencies come to depend on the same providers for their own operations.
  3. Fail to change the rules. Voters elect a government promising different policies. In this hypothetical, public agencies lack independent systems to enforce those policies, and the providers refuse to cooperate.

This is a hypothetical scenario, not an event that happened. Gradual Disempowerment argues that AI adoption could weaken human influence across the economy, culture and states. It does not show that this endpoint has occurred.

The research. Two Types of AI Existential Risk: Decisive and Accumulative (Kasirzadeh, 2024; Philosophical Studies, 2025) contrasts the familiar takeover scenario with an accumulative one: “a boiling frog scenario where incremental AI risks slowly converge, undermining societal resilience until a triggering event results in irreversible collapse.” Gradual Disempowerment (Kulveit et al., 2025) argues that incremental improvements in AI “can undermine human influence over large-scale systems that society depends on, including the economy, culture, and nation-states”.

Where the evidence stands. These are conceptual arguments rather than experimental results. The International AI Safety Report 2026 recognises the category, distinguishing “passive loss of control scenarios, where the broad adoption of AI systems undermines human control through over-reliance on AI for decision-making or other important societal functions” from the active scenarios in the first pathway.

How to read these pathways

The pathways are not exclusive. A weakened society is less able to respond to a misused or uncontrolled system, and faster AI progress shortens the time available to notice any of these problems. They are also not equally well studied. The first has the most experimental work, but that work mostly shows what current systems can do in constructed settings, not what they do in deployment.

For a broader view of the field, see What is AI safety?. For the evidence behind any single claim, read the papers linked above.