Oversight of the most capable AI systems, the companies that build them and the chips they run on.
A small number of companies build the most capable AI models, using very large amounts of specialised computing hardware (“compute”). Frontier AI governance looks at how these companies and their models can be overseen: safety commitments, audits, export controls on chips and international agreements that can be verified.
Much of the work is quantitative or investigative, such as estimating how much compute a model used, tracking data centres or comparing company safety policies. Technical skills are in demand.
Short projects with numbered steps.
Read two frontier AI companies’ published safety frameworks and compare them side by side using METR’s list of the elements these policies usually share.
At the 2024 AI Seoul Summit, frontier AI companies signed voluntary safety commitments, including publishing a safety framework. Pick five signatories and check what they actually published.
Use the rule of thumb “training compute ≈ 6 × parameters × training tokens” to estimate the compute behind a model, then compare your answer with Epoch AI’s database.
Download Epoch AI’s free dataset of notable AI models, chart training compute over time in Python, estimate how fast it grows, and count how many models cross the EU AI Act’s compute threshold.
Pick a known AI cluster from Epoch AI’s GPU cluster data, find it in free satellite imagery and document how it has grown over time.
Why do so many governance proposals focus on chips? Read one key paper and write a jargon-free explainer for someone outside tech.
The MIRI reading list flags this report as outdated and suggests updating it as an intermediate project. Gather what has changed in Chinese language models since it was written.
Export controls target data-centre chips. But consumer hardware keeps getting better, and frontier-level performance reaches it within about a year. Update the analysis with the latest hardware.
Pick one verification mechanism, such as network taps, trusted execution environments, chip telemetry or satellite monitoring, and write a short, clear analysis of how it works and how it could fail.
Could algorithmic progress continue under a compute halt? Catalogue the innovations in a major model paper and estimate the compute used in the experiments behind each, extending published MIRI TGT work.
MIRI TGT argued that catching up to a given capability level gets cheaper by roughly 60× per year. Re-run the analysis with newer models and test how robust that number is.
MIRI TGT published a draft international agreement to prevent premature superintelligence. Pick one article and ask: how would it be verified, enforced and evaded?
Will society respond to early AI accidents the way it should? Collect historical warning shots from other technologies and analyse which ones actually changed policy.
New methods let models train across many loosely connected computers. Use Epoch AI’s simulator and recent papers to analyse whether this undermines governance based on big data centres.
Reuel et al. map open problems by capacity (assessment, access, verification, security, operationalisation and ecosystem monitoring) across data, compute, models and deployment. Choose one cell and scope a project.
The Future of Life Institute’s AI Safety Index grades leading companies on public evidence. Apply its indicators to a company it does not cover, or re-score one with newer evidence.
Pick three lessons Zaidi and Dafoe draw from the failed attempt at international control of nuclear weapons. For each, argue whether it transfers to AI compute: chips are easier to track than ideas but harder than uranium.
Projects from other themes, and general projects that work in any theme.
AI biology tools (protein design, CRISPR design and more) sometimes publish biosecurity statements. Add new tools to the repository and compare what they promise.
Write a short, jargon-free explainer of one AI safety idea (reward hacking, sycophancy, compute governance, DNA screening) for someone outside tech.
Define ten key terms from the theme you’re most curious about, in one sentence each with an everyday example, then compare them with the site’s glossary.
Pick a beginner-friendly paper linked from any project on this site and summarise it in ten clear posts: the question, what they did, what they found, and why it matters.
A short video, a carousel, an illustrated explainer or a podcast segment: make one piece of honest, accurate AI safety content for an audience you understand.
Pick one short reading, invite 4–8 people, and run a focused one-hour discussion with three good questions.
Existing biosecurity guidance increasingly recognizes artificial intelligence, information security, and the role of Institutional Biosafety Committees (IBCs), but institutions still need practical ways to translate these principles into project-level review. This project will develop and pilot-test a resource, calibrated screening framework, comprising an intake checklist, risk tiers, and escalation decision tree, to help IBCs identify and manage risks arising from AI-enabled biological research.
BiosecurityTechnical governanceInternational governance
Starter project: Compare two AI companies’ safety promises or Explain DNA synthesis screening in one page
States are integrating AI into military and diplomatic functions, with consequences for deterrence, crisis stability, and agreement-making that remain underanalyzed. This project produces analytical papers and policy briefs on two questions: under what conditions delegation to AI systems destabilizes crises, and whether machine-verifiable commitments could improve the verifiability of future agreements.
AI strategyInternational governance
Starter project: Did they keep their promise? Check the Seoul commitments or Spot an AI data centre from space
Build quantitative forecasts of China's indigenous compute production chain, refining existing DUV/EUV lag estimates and/or extending the approach to other bottlenecks such as HBM.
Compute governanceUS-China governanceAI strategy
Starter project: Estimate how much compute a famous model needed or Plot AI compute trends with real data
Keeping the most advanced AI within the control of democracies depends on integrating democratic allied nations into the US AI stack, but US extraterritorial surveillance and data access laws are eroding those allies' trust in its tech stack. This project aims to propose a reformed regime that reconciles legitimate US security interests with the trust and assurance allies need to build on American infrastructure.
International governanceUS policyAI strategy
Starter project: Spot an AI data centre from space or Plot AI compute trends with real data
Dylan Bowman and collaborators will mentor a paid, largely self-directed project in strategy and futurism for AI safety. Potential topics include forecasting, applied game theory, and AI evaluations.
AI strategyEvaluations
Starter project: Did they keep their promise? Check the Seoul commitments or Spot an AI data centre from space
In the case of a Middle Power Frontier AI coalition, what would be the most effective routes to sufficiently large compute buildout? How quickly could middle powers acquire and operationalise substantial compute? Which constraints determine their progress?
Compute governanceAI strategyInternational governance
Starter project: Explain compute governance to a friend or Estimate how much compute a famous model needed
