Most evals measure dangerous capabilities. Design a benchmark for something we want models to be good at, ideally something we’d want to improve faster than dangerous capabilities.
Beneficial capability evals give AI companies something to aim for. An eval is also most of the way to a training environment, which labs actively pay for.
Output: An eval specification with 20+ examples, and ideally a working Inspect implementation.
Read BlueDot’s case for beneficial capability evals.
Choose a capability that clearly helps safety or society (e.g. spotting misinformation, helping people reason, secure coding).
Write 20 examples by hand with a clear scoring rule. No code needed for this part.
Pilot it on two models; refine examples the models find ambiguous.
Implement it in Inspect if you code, or hand it to a coder friend.
All steps done
Share what you made, then try another project. The PDF version has room for notes.
Write a small, repeatable test that checks whether a model admits when it does not know something, using Inspect, the open-source evaluation framework used by the UK AI Security Institute.
Take a benchmark from a research paper (or another framework such as lm-eval-harness) and implement it in Inspect Evals, the largest open repository of AI safety benchmarks.
Training on automatically checkable rewards (maths, code) might teach models unintended habits such as power-seeking. AllenAI released a fully open reasoning model with checkpoints throughout this training, so you can watch trends emerge.
