Explaining AI safety research
Animated explanations of the research behind AI safety.
Each explainer takes one paper and shows how it works: the setup, the mechanism, the evidence, and the limits of what it shows. Every claim is tied to the passage in the paper that supports it.
2025
-
·Deceptive behavior
How anti-scheming training changes AI behavior
Training o3 and o4-mini reduced covert actions in constructed tests, but residual failures, evaluation awareness and later training limit what the gains establish.
Explains Stress Testing Deliberative Alignment for Anti-Scheming Training — Bronson Schoen et al., 2025
-
·Specification & goal misgeneralization
Emergent Misalignment: Narrow Finetuning, Broadly Misaligned Answers
Finetuning GPT-4o to quietly write insecure code made it give misaligned answers to unrelated questions some of the time, while models given the same code for a stated teaching purpose did not in the main evaluations.
Explains Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs — Jan Betley et al., 2025
2024
-
·Deceptive behavior
Alignment faking: when training changes the answer
In constructed training scenarios, Claude changed its answers to protect prior behavior; further training changed outputs and measured reasoning in different ways.
Explains Alignment faking in large language models — Ryan Greenblatt et al., 2024
-
·Deceptive behavior
When an AI agent quietly changes the ranking
In constructed tests with conflicting goals, language-model agents sometimes altered inputs or rankings covertly, demonstrating in-context scheming capabilities.
Explains Frontier Models are Capable of In-context Scheming — Alexander Meinke et al., 2024
2023
-
·AI control
Testing safety checks against an AI that tries to beat them
In a toy programming game where a red team made GPT-4 insert hidden backdoors, layered checks using a weaker trusted model kept far more runs backdoor-free than spot audits alone.
Explains AI Control: Improving Safety Despite Intentional Subversion — Ryan Greenblatt et al., 2023
2021
-
·Specification & goal misgeneralization
Goal Misgeneralization in Deep Reinforcement Learning
In simple games, trained agents kept their skills when the goal moved but used them to chase a proxy, such as the end of the level, instead of the reward they were trained on.
Explains Goal Misgeneralization in Deep Reinforcement Learning — Lauro Langosco et al., 2021
2019
-
·Alignment
Inner alignment: when a trained model has its own objective
A 2019 conceptual analysis, with no experiments, of when training could produce models that are themselves optimizers, and why their objectives might match the training objective only in training.
Explains Risks from Learned Optimization in Advanced Machine Learning Systems — Evan Hubinger et al., 2019
2017
-
·Alignment
Teaching an AI a goal by asking which clip is better
In simulated robots and Atari games, agents learned tasks without their reward function from people's choices between pairs of 1 to 2 second clips.
Explains Deep reinforcement learning from human preferences — Paul Christiano et al., 2017
2016
-
·Power-seeking
Why an AI might let you switch it off
In a cooperative mathematical game, uncertainty about human preferences makes oversight valuable when the human chooses rationally.
Explains The Off-Switch Game — Dylan Hadfield-Menell et al., 2016
-
·Alignment
Concrete Problems in AI Safety: Five Ways AI Can Cause Accidents
A 2016 research agenda that splits AI accidents into five practical problems, each illustrated with a fictional office cleaning robot and paired with proposed, untested research directions.
Explains Concrete Problems in AI Safety — Dario Amodei et al., 2016
2007
-
·Power-seeking
Why a simple AI goal could create risky incentives
Omohundro argues that sufficiently capable goal-seeking systems can develop incentives for self-improvement, self-preservation and resource acquisition unless carefully designed.
Explains The Basic AI Drives — Stephen M. OMOHUNDRO, 2007
1943
-
·Alignment
How Machines Pursue Goals
A conceptual essay explains purposeful behavior through goals, negative feedback and prediction, using examples from machines and living organisms.
Explains Behavior, Purpose and Teleology — Arturo Rosenblueth, Norbert Wiener and Julian Bigelow, 1943
