AI systems are advancing quickly, and their potential is vast. To realise the benefits of more capable AI, we need confidence that powerful systems will do what we intend, even in complex, high-stakes settings.
AI alignment is a crucial subset of that challenge. Alignment research asks how to build and deploy AI that follows human intent, avoids harmful side effects, and remains under human oversight and control. As systems become more capable and autonomous, methods that work for today’s models may not hold up tomorrow.
That’s why we set up the Alignment Project, a global funding programme to accelerate progress in alignment research and innovation. Today we’re announcing our first 60 grant awardees, alongside new partners and £12m in additional funding, bringing total support for alignment research to £27m.
A growing coalition
The Alignment Project launched in July 2025 with over £15 million in funding, backed by an international coalition including the Canadian AI Safety Institute, CIFAR, Schmidt Sciences, Amazon Web Services (AWS), Anthropic, Halcyon Futures, the Safe AI Fund, UK Research and Innovation, and the Advanced Research and Invention Agency (ARIA).
We now welcome OpenAI, Microsoft, the Australian Department of Industry, Science and Resources’ AI Safety Institute, the AI Safety Tactical Opportunities Fund, Sympatico Ventures, and Renaissance Philanthropy.
Together, these partners bring total funding available for alignment research to £27m, including £5.6m from OpenAI. This funding will help remove barriers that have historically limited alignment research. It supports the people, time, compute and collaboration needed to stress-test ideas at scale.
Announcing the first 60 grantees
Interest in the Alignment Project’s first funding round was extraordinary. We received 800+ applications from 466 institutions across 42 countries.
We shortlisted 101 proposals for full applications after assessing every submission for relevance, feasibility, innovation, actionability, and team capability. Shortlisted applicants then worked with our team to sharpen their theory of change, execution plan and budgets, before full proposals were assessed in depth by expert reviewers and a moderation board.
The 60 successful projects span fields like mathematics, learning theory, economics, cognitive science and more, reflecting our belief that alignment progress requires a multidisciplinary effort. (Full list of awardees.)
Below are three projects that illustrate the range of the portfolio:
1. Scientist AI
LawZero, a non-profit founded by Yoshua Bengio, is developing a safe-by-design AI system it calls Scientist AI. The project focuses on two linked ideas: improving how models judge the reliability of information and making their reasoning easier for people to inspect.
Scientist AI includes a “contextualization” system that tracks the provenance and trustworthiness of web data, helping the model separate fact from opinion and reducing the risk of picking up undesirable behaviours from human discourse. It also uses a prover-verifier style setup with two components: one proposes hypotheses and the other checks them. By having these components communicate in natural language, the system aims to keep its reasoning more transparent. The verifier is also trained to minimise agency, reducing risks like deception or hidden agendas.
In exploring concrete design choices that could make future systems easier to supervise and less prone to manipulative behaviour, this work lays essential groundwork for aligning superintelligent systems.
2. Mechanism and information design for alignment and control
Dirk Bergemann (Douglass and Marion Campbell Professor of Economics at Yale), Stephen Morris (Peter Diamond Professor of Economics at MIT) and colleagues are bringing tools from mechanism and information design, the economics of rules, incentives, and information, to the problem of aligning and controlling advanced AI.
The project aims to build a more systematic framework to balance capability and safety, as well as to design institutions and “rules of the game” around deployed AI systems. It draws on two threads. First, it models how an AI system’s behaviour adapts to user preferences over time, and what trade-offs that creates. Second, it studies “memory design”, since AI systems can be built to remember or deliberately forget, which may create new ways to shape behaviour that do not exist in human-only settings. This work aims to give policymakers and AI developers principled tools for governing AI behaviour as these systems agents become more advanced and enmeshed in complex systems.
3. Precision optimisation and symmetry breaking for improving AI predictability
Eva Silverstein and Surya Ganguli (Stanford) are working on ways to make AI training more predictable, with the long-term goal of making it easier to steer models toward safer outcomes.
Optimisers are a crucial part of training any modern AI system. Today’s optimisers can produce strong models, but it is hard to predict which solution they will converge on. Their approach, Energy Conserving Descent (ECD) - developed by G. Bruno De Luca and Silverstein) is designed so its behaviour follows a known probability distribution, which could make training outcomes for modern AI systems easier to anticipate and influence. The project will test ECD at meaningful scale, including whether a key technical change in transformer attention blocks can speed up convergence and improve performance. They will also examine whether this predictability translates into practical control, such as steering training toward more complex, aligned solutions rather than simpler, misaligned ones.
As AI models become more powerful, it becomes more important to predict and steer training dynamics in a principled and understandable way.
What’s next
Making substantial breakthroughs in alignment research requires a multidisciplinary effort which brings ideas and tools from different fields to bear on open research questions.
The scale and breadth of applications in this first round give us optimism about what the field can achieve with sustained support. Applications for the next round will open this summer. If you’re interested in future calls or partnership opportunities, visit the Alignment Project website for updates.
