AI Safety Sandbox
Search
TechnicalPhilosophy1–3 hoursStarterFree

Peek inside a neural network

Use Neuronpedia, a free browser tool, to see which internal “features” of a language model light up for different ideas, then try to break your own explanation of one.

Download PDFChatbot experimentsReading & writing

Why it matters

Interpretability aims to understand models from the inside, so we are not relying only on what they say. Getting a feel for features is the first step towards that whole research field.

Output: Notes on three features: what you think each represents, and where your guess failed.

Steps

0 of 5 done

  1. Open Neuronpedia

    10 min

    Go to Neuronpedia and pick a model with sparse-autoencoder features (Gemma Scope is a good choice). Search for a concept you like: a city, an emotion, a programming language.

  2. Read one feature’s examples

    15 min

    Open a feature from your search. Look at the text snippets that activate it most strongly. Write down, in one sentence, what you think it represents.

  3. Test your guess with your own text

    20 min

    Type new sentences into the feature’s test box. Try ones that should activate it and ones that should not. Where does your one-sentence guess break?

  4. Repeat for two more features

    20 min

    Choose features for quite different concepts. Note which were clean and which were messy or surprising.

  5. Write it up

    10 min

    Summarise: three features, your best description of each, one surprising failure. Add one question you would want a researcher to answer.

Try next

TechnicalA weekendGuided

Find a pattern-copying circuit

Follow a guided notebook to find the induction heads in a small transformer, the tiny circuit that lets models continue patterns they have seen earlier in the text.

CodeView project
PhilosophyA few weeksIntermediate

Does interpretability answer the octopus test?

Bender and Koller argue a system trained only on text cannot learn meaning. Use Neuronpedia’s free interface to Gemma Scope to inspect features for concepts that seem to need grounding (colours, spatial relations) and check whether they respond across very different wordings.

Chatbot experimentsReading & writingView project

Related projects

TechnicalA weekendGuided

Make your first open-source AI safety contribution

Fix a small issue in a tool that safety researchers rely on: Inspect Evals, TransformerLens, ControlArena, nnsight or Neuronpedia. No ML research experience needed, just software skills.

CodeView project
TechnicalA few weeksIntermediate

Tackle a beginner-graded open problem in interpretability

Neel Nanda’s list of 200 open problems grades each one by difficulty, including problems marked as good for beginners. Pick one, scope it down and investigate with TransformerLens.

CodeView project
TechnicalA weekendIntermediate

Make a safety paper’s code easy to re-run

Pick a safety paper with public code and make it reproducible: fix broken dependencies, write setup docs, or package it so others can run it in one command.

CodeView project

Further reading

Related mentored projects

All mentored projects