Testing, understanding and controlling AI systems.
Technical AI safety covers evaluating how models behave, interpretability (studying what happens inside them), and training and oversight methods that keep models honest and controllable as they become more capable.
Many first projects need only a chatbot or a browser. Others use Python, open-source evaluation tools or small open models.
Short projects with numbered steps.
Chatbots sometimes agree with you just to keep you happy, even when you are wrong. Researchers call this sycophancy. Test for it yourself with five questions and two ways of asking.
Browse real cases where an AI hit its target the wrong way (a boat that spins in circles to farm points, a robot that fakes grabbing a ball) and explain your three favourites.
Use Neuronpedia, a free browser tool, to see which internal “features” of a language model light up for different ideas, then try to break your own explanation of one.
Write a small, repeatable test that checks whether a model admits when it does not know something, using Inspect, the open-source evaluation framework used by the UK AI Security Institute.
Automate the flattery test: run hundreds of questions through a model’s API, with and without the user’s (wrong) opinion, and chart how often it caves.
Follow a guided notebook to find the induction heads in a small transformer, the tiny circuit that lets models continue patterns they have seen earlier in the text.
Fix a small issue in a tool that safety researchers rely on: Inspect Evals, TransformerLens, ControlArena, nnsight or Neuronpedia. No ML research experience needed, just software skills.
Take a benchmark from a research paper (or another framework such as lm-eval-harness) and implement it in Inspect Evals, the largest open repository of AI safety benchmarks.
Pick a safety paper with public code and make it reproducible: fix broken dependencies, write setup docs, or package it so others can run it in one command.
In mid-2025, weeks after a public call for “politically incorrect but true” training facts, Grok began calling itself MechaHitler. Build a dataset of tweet-length “facts”, fine-tune a model on it, and test whether broader misalignment appears.
Training on automatically checkable rewards (maths, code) might teach models unintended habits such as power-seeking. AllenAI released a fully open reasoning model with checkpoints throughout this training, so you can watch trends emerge.
Most evals measure dangerous capabilities. Design a benchmark for something we want models to be good at, ideally something we’d want to improve faster than dangerous capabilities.
Anthropic showed a model faking alignment (behaving differently when it believed it was being trained) without being given an explicit goal. Reproduce the demo and test one new variation.
Neel Nanda’s list of 200 open problems grades each one by difficulty, including problems marked as good for beginners. Pick one, scope it down and investigate with TransformerLens.
AI control asks: if a model might be trying to sabotage us, can we still use it safely? ControlArena is the UK AISI’s open toolkit of settings, model organisms and protocols. Run one setting, then improve a monitor or document a gap.
Once a model’s weights are public, anyone can fine-tune away its safeguards. Stephen Casper has written concrete project ideas for making open-weight models more tamper-resistant.
Projects from other themes, and general projects that work in any theme.
Make a tiny calculator (a spreadsheet, notebook or web page) that tells anyone how much a portable air purifier would clean the air in their room.
Explore whether AI tools can lower the effort of improving home air quality, or speed up reviewing the built-environment research literature.
Write a short, jargon-free explainer of one AI safety idea (reward hacking, sycophancy, compute governance, DNA screening) for someone outside tech.
Define ten key terms from the theme you’re most curious about, in one sentence each with an everyday example, then compare them with the site’s glossary.
Pick a beginner-friendly paper linked from any project on this site and summarise it in ten clear posts: the question, what they did, what they found, and why it matters.
A short video, a carousel, an illustrated explainer or a podcast segment: make one piece of honest, accurate AI safety content for an audience you understand.
Persona Selection Model says that LLMs learn a bunch of different personas during pretraining (to be able to predict then next token on all internet text) and posttraining selects/refines the assistant persona. There have been many papers exploring the assistant persona and evil personas but very few papers trying to understand the role of chain of thought in all this. This project's aim would be to understand the role of the CoT in the persona selection model.
Chain of thoughtMechanistic interpretabilityAlignment
Starter project: Peek inside a neural network or Find a pattern-copying circuit
Own one full matching cycle end to end: take in live roles from hiring organisations, score candidates against them, run the introductions, and write up what the round revealed about how these jobs actually get filled.
Generalist
Starter project: Measure flattery with code or Make a ten-term glossary
We develop a model organism of capability emergence through RL.
EvaluationsMechanistic interpretability
Starter project: Find a pattern-copying circuit or Peek inside a neural network
The field increasingly measures AI welfare by asking models about their own states, yet frontier models increasingly detect when they are being tested and change how they answer. This project runs the first systematic test of whether AI welfare self-reports survive a change of presentation frame, or whether the field's core instrument is partly measuring the model's recognition of the probe.
AI welfareBehavioral evaluation of LLMsEvaluations
Starter project: Ask a chatbot about its inner life five different ways or Measure flattery with code
In this project, we will run a red team / blue team exercise in creating and detecting hidden traits in synthetic datasets. Recent work shows that such traits can transfer diffusely through the synthetic data, surviving semantic filtering -- which makes this project high impact.
AlignmentMechanistic interpretabilityEvaluations
Starter project: Run an AI policy tabletop exercise or Measure flattery with code
Build the groundwork for a theory of what can and cannot be learned about agents. E.g. for a specific agent model that invokes goals, beliefs, intents etc, when can these endogernous variables be learned from behavioural experiments, and under what assumptions. Use this to derive fundamental identifiability results - in the first instance, to prove that the beliefs and goals of expected utility maximizers can be learned under weak assumptions, and derive scalable algorithms for doing so.
AlignmentEvaluations
Starter project: Catch a chatbot flattering you or Build your first mini-eval
