Blog Posts

Most of the Tools Built to Fix AI Governance Haven't Been Tested Yet

Policy & Regulation AI Strategy

Key takeaways

  • Globally, AI governance tools have outpaced evidence: Sandboxes, fellowships, and impact assessments have recently grown fast, but most were never designed to demonstrate whether they work.
  • Where they have been tested, results are mixed: the FCA sandbox and AAAS fellowships show real impact, Canada's assessments haven’t clearly changed outcomes, and NYC saw only 18 of 391 employers comply.
  • The fix is building in measurement from the start: Funders and regulators should require common metrics and independent review before more of these tools are built.

From August 2026, every EU member state must have an AI regulatory sandbox up and running – a testing environment where developers can trial AI products under regulatory supervision before full deployment. At least 12 member states have already missed earlier deadlines under the AI Act. But the sandboxes will be built, because the law requires it.

Across the Atlantic, AI governance fellowship programmes have multiplied: researchers are being embedded inside governments in Washington (IAPS), Brussels (Talos), London (GovAI), and Montreal (Mila) – city-based, not national programmes yet. Algorithmic impact assessments are now mandatory in Canada, the EU, the Netherlands (for public sector AI only), and New York City (the first jurisdiction globally to mandate bias audits for commercial AI systems).
AI governance infrastructure indeed has been built – but does any of it actually work?
The honest answer, based on available evidence, is that we largely do not know. And in a handful of cases where we do have data, the results are not encouraging.

Take algorithmic impact assessments – structured reviews meant to catch AI risks before systems go live. Canada has run the longest-standing regime in the world since 2019, mandatory across 97 federal institutions. It has produced 22 published assessments. Whether those assessments changed what systems were built, or only documented them, is almost impossible to establish. New York City’s version is a starker story: a FAccT 2024 study found that only 18 of 391 employers required to post bias audit reports had done so. A December 2025 audit by the New York State Comptroller described enforcement as ineffective.

Fellowship programmes – placing researchers directly inside government – are arguably the most promising structural fix to the expertise gap that makes AI governance so difficult. The AAAS Science and Technology Policy Fellowships, which place over 250 scientists in government each year, found that 76% of alumni went on to advocate for specific policies. That is genuinely encouraging. But the AAAS programme is the only one in this field with a comprehensive independent evaluation. Newer programmes – IAPS, Talos, GovAI, Mila – are proliferating without shared frameworks for measuring whether they change policy outcomes. Meanwhile, funding continues to flow in.

Regulatory sandboxes follow the same pattern. The UK’s Financial Conduct Authority sandbox has the strongest track record – a peer-reviewed study found that participants were 50% more likely to secure investment than comparable firms, and raised 15% more when they did. That evidence base took years to build. The EU’s mandate for 27 national AI sandboxes by August 2026 is being implemented without equivalent measurement requirements. Spain’s AESIA sandbox, one of the most advanced in the EU, is explicitly designed to produce harmonised compliance guidance – but whether it does will depend on evaluation systems that do not yet exist.

This is not a criticism of the people running these programmes. Many are thoughtful, well-resourced, and operating in good faith. Yet the pattern is consistent: the field of AI governance has invested heavily in building mechanisms and almost nothing in measuring them. A 2025 CHI scoping study found lack of systematic evaluation of whether algorithmic impact assessment tools achieve their intended outcomes. The most comprehensive analysis of research-to-policy translation in AI governance, published in Internet Policy Review in November 2025, found that most bridging mechanisms rely on self-reporting or process metrics rather than outcome evidence.

The consequences are about to follow. Mechanisms that cannot demonstrate impact get replicated anyway, because they signal seriousness. Governments facing political pressure to act on AI adopt the tools other governments have adopted, without the evidence base to know whether those tools produce the outcomes they promise. The OECD’s review of 200 AI use cases across 11 government functions found that scaling is rare – and only 15% of OECD countries had an investment framework for public AI. As a result, institutions’ efforts may come to naught.

Fixing that requires a deliberate choice to make it. Fellowship funders – governments, foundations, private donors – should require common output metrics, follow-on surveying, and periodic independent review as standard conditions of support. The AAAS evaluation model already exists – applying it consistently across programmes is a coordination problem. Sandbox mandates, like those in the EU AI Act, should include explicit requirements to publish findings in formats that feed regulatory guidance. And governments commissioning impact assessments should fund parallel evaluation of whether those assessments change decisions, not just whether they get filed.

None of this slows AI governance down – rather the opposite, it makes all the infrastructure being built worth having.

Dozens of programmes are scaling right now, in jurisdictions that will spend years and significant public funds on mechanisms whose effectiveness they have no systematic way to measure. It is high time governments asked whether the tools match the task – before the architecture is locked in.

References

AAAS. Science & Technology Policy Fellowships. https://www.aaas.org/programs/science-technology-policy-fellowships

AAAS. Science Policy Fellowship Has Durable Impact, Independent Evaluation Shows. https://www.aaas.org/news/science-policy-fellowship-has-durable-impact-independent-evaluation-shows

ACM / CHI 2024. Berman, Goyal & Madaio. A Scoping Study of Evaluation Practices for Responsible AI Tools: Steps Towards Effectiveness Evaluations. https://dl.acm.org/doi/10.1145/3613904.3642398

ACM / FAccT 2024. Wright et al. Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. https://dl.acm.org/doi/10.1145/3630106.3658998

AESIA (Spain). Guides. https://aesia.digital.gob.es/en/guides

ECNL (European Center for Not-for-Profit Law). Netherlands Sets Precedent for Human Rights Safeguards in Use of AI. https://ecnl.org/news/netherlands-sets-precedent-human-rights-safeguards-use-ai

EU Artificial Intelligence Act. Article 27: Fundamental Rights Impact Assessment for High-Risk AI Systems. https://artificialintelligenceact.eu/article/27/

EU Artificial Intelligence Act. Article 57: AI Regulatory Sandboxes. https://artificialintelligenceact.eu/article/57/

FCA (UK Financial Conduct Authority). Regulatory Sandbox. https://www.fca.org.uk/firms/innovation/regulatory-sandbox

GovAI. UK Winter Fellowship 2027, Research Track. https://www.governance.ai/opportunities/uk-winter-fellowship-2027

IAPS (Institute for AI Policy and Strategy). Fellowship. https://www.iaps.ai/fellowship

Internet Policy Review. Oduro, Marwick, Johnson & Meyer. Troubling Translation: Sociotechnical Research in AI Policy and Governance (18 Nov 2025). https://policyreview.info/articles/analysis/troubling-translation-ai-policy-and-governance

Mila. AI Policy Fellowship Publications. https://mila.quebec/en/ai4humanity/ai-governance-policy-and-inclusion/mila-ai-policy-fellowship

NYC Department of Consumer and Worker Protection (DCWP). Automated Employment Decision Tools (AEDT). https://www.nyc.gov/site/dca/about/automated-employment-decision-tools.page

OECD Publishing. Governing with Artificial Intelligence: The State of Play and Way Forward in Core Government Functions. https://www.oecd.org/en/publications/2025/06/governing-with-artificial-intelligence_398fa287.html

Office of the New York State Comptroller. DiNapoli: New Yorkers Deserve a Transparent Hiring Process When Artificial Intelligence Is Used To Vet Their Job Applications. https://www.osc.ny.gov/press/releases/2025/12/dinapoli-new-yorkers-deserve-transparent-hiring-process-when-artificial-intelligence-used-vet-their

Review of Finance (Oxford Academic). Merrouche. Regulatory Sandboxes and Fintech Funding: Evidence from the UK. https://academic.oup.com/rof/article/28/1/203/7140150

Talos Network. Talos Fellowship. https://www.talosnetwork.org/talos-fellowship

Transformer. Schaefer. Hiring Struggles Are Plaguing the EU AI Office. https://www.transformernews.ai/p/eu-is-struggling-to-hire-ai--act-office-safety-unit

Treasury Board of Canada Secretariat. Directive on Automated Decision-Making. https://www.tbs-sct.canada.ca/pol/doc-eng.aspx?id=32592