https://aisafetyrisks.org/2026-11-11https://aisafetyrisks.org/about/https://aisafetyrisks.org/ai-safety/https://aisafetyrisks.org/doomsday/https://aisafetyrisks.org/works/stress-testing-deliberative-alignment-for-anti-scheming-training/2026-10-03https://aisafetyrisks.org/works/stress-testing-deliberative-alignment-for-anti-scheming-training/thumbnail.jpgHow anti-scheming training changes AI behaviorTraining o3 and o4-mini reduced covert actions in constructed tests, but residual failures, evaluation awareness and later training limit what the gains establish.https://media.aisafetyrisks.org/works/stress-testing-deliberative-alignment-for-anti-scheming-training/video-076ef71b2d69.mp45462026-10-02T21:17:01.563680-04:00https://aisafetyrisks.org/works/emergent-misalignment-narrow-finetuning-can-produce-broadly/2026-11-11https://aisafetyrisks.org/works/emergent-misalignment-narrow-finetuning-can-produce-broadly/thumbnail.jpgEmergent Misalignment: Narrow Finetuning, Broadly Misaligned AnswersFinetuning GPT-4o to quietly write insecure code made it give misaligned answers to unrelated questions some of the time, while models given the same code for a stated teaching purpose did not in the main evaluations.https://media.aisafetyrisks.org/works/emergent-misalignment-narrow-finetuning-can-produce-broadly/video-266a38ef898a.mp45382026-11-11T01:55:01.801609-05:00https://aisafetyrisks.org/works/alignment-faking-in-large-language-models/2026-10-03https://aisafetyrisks.org/works/alignment-faking-in-large-language-models/thumbnail.jpgAlignment faking: when training changes the answerIn constructed training scenarios, Claude changed its answers to protect prior behavior; further training changed outputs and measured reasoning in different ways.https://media.aisafetyrisks.org/works/alignment-faking-in-large-language-models/video-f7b1b6a084ea.mp44752026-10-01T04:15:05.836000+00:00https://aisafetyrisks.org/works/frontier-models-are-capable-of-in-context-scheming/2026-10-03https://aisafetyrisks.org/works/frontier-models-are-capable-of-in-context-scheming/thumbnail.jpgWhen an AI agent quietly changes the rankingIn constructed tests with conflicting goals, language-model agents sometimes altered inputs or rankings covertly, demonstrating in-context scheming capabilities.https://media.aisafetyrisks.org/works/frontier-models-are-capable-of-in-context-scheming/video-38dd01c271e8.mp44342026-10-02T20:33:50.030547-04:00https://aisafetyrisks.org/works/ai-control-improving-safety-despite-intentional-subversion/2026-11-11https://aisafetyrisks.org/works/ai-control-improving-safety-despite-intentional-subversion/thumbnail.jpgTesting safety checks against an AI that tries to beat themIn a toy programming game where a red team made GPT-4 insert hidden backdoors, layered checks using a weaker trusted model kept far more runs backdoor-free than spot audits alone.https://media.aisafetyrisks.org/works/ai-control-improving-safety-despite-intentional-subversion/video-d6add00c34a0.mp45442026-11-11T02:00:16.798249-05:00https://aisafetyrisks.org/works/goal-misgeneralization-in-deep-reinforcement-learning/2026-10-03https://aisafetyrisks.org/works/goal-misgeneralization-in-deep-reinforcement-learning/thumbnail.jpgGoal Misgeneralization in Deep Reinforcement LearningIn simple games, trained agents kept their skills when the goal moved but used them to chase a proxy, such as the end of the level, instead of the reward they were trained on.https://media.aisafetyrisks.org/works/goal-misgeneralization-in-deep-reinforcement-learning/video-7886099f0396.mp42572026-09-30T17:27:32.986000+00:00https://aisafetyrisks.org/works/risks-from-learned-optimization-in-advanced-machine-learning/2026-11-11https://aisafetyrisks.org/works/risks-from-learned-optimization-in-advanced-machine-learning/thumbnail.jpgInner alignment: when a trained model has its own objectiveA 2019 conceptual analysis, with no experiments, of when training could produce models that are themselves optimizers, and why their objectives might match the training objective only in training.https://media.aisafetyrisks.org/works/risks-from-learned-optimization-in-advanced-machine-learning/video-ca161fd1bfb9.mp44692026-11-11T01:52:17.149162-05:00https://aisafetyrisks.org/works/deep-reinforcement-learning-from-human-preferences/2026-11-11https://aisafetyrisks.org/works/deep-reinforcement-learning-from-human-preferences/thumbnail.jpgTeaching an AI a goal by asking which clip is betterIn simulated robots and Atari games, agents learned tasks without their reward function from people's choices between pairs of 1 to 2 second clips.https://media.aisafetyrisks.org/works/deep-reinforcement-learning-from-human-preferences/video-3bf036f867f7.mp44402026-11-11T01:51:04.228200-05:00https://aisafetyrisks.org/works/the-off-switch-game/2026-10-03https://aisafetyrisks.org/works/the-off-switch-game/thumbnail.jpgWhy an AI might let you switch it offIn a cooperative mathematical game, uncertainty about human preferences makes oversight valuable when the human chooses rationally.https://media.aisafetyrisks.org/works/the-off-switch-game/video-1bbfd001b3b3.mp43582026-09-29T06:11:27.585000+00:00https://aisafetyrisks.org/works/concrete-problems-in-ai-safety/2026-10-03https://aisafetyrisks.org/works/concrete-problems-in-ai-safety/thumbnail.jpgConcrete Problems in AI Safety: Five Ways AI Can Cause AccidentsA 2016 research agenda that splits AI accidents into five practical problems, each illustrated with a fictional office cleaning robot and paired with proposed, untested research directions.https://media.aisafetyrisks.org/works/concrete-problems-in-ai-safety/video-64c977069037.mp44932026-09-28T15:01:25.185000+00:00https://aisafetyrisks.org/works/the-basic-ai-drives/2026-10-03https://aisafetyrisks.org/works/the-basic-ai-drives/thumbnail.jpgWhy a simple AI goal could create risky incentivesOmohundro argues that sufficiently capable goal-seeking systems can develop incentives for self-improvement, self-preservation and resource acquisition unless carefully designed.https://media.aisafetyrisks.org/works/the-basic-ai-drives/video-d5ab5b2a80ab.mp45022026-10-03T11:31:16.528000-04:00https://aisafetyrisks.org/works/behavior-purpose-and-teleology/2026-10-03https://aisafetyrisks.org/works/behavior-purpose-and-teleology/thumbnail.jpgHow Machines Pursue GoalsA conceptual essay explains purposeful behavior through goals, negative feedback and prediction, using examples from machines and living organisms.https://media.aisafetyrisks.org/works/behavior-purpose-and-teleology/video-97023db4873d.mp43952026-10-03T13:32:07.264546-04:00