Google has just dropped out of the top 5 AI model providers on the Artificial Analysis Index, but it seems to be keeping thinking about AI regulation deeply all the same.
In a new essay, Google DeepMind CEO Demis Hassabis has laid out a proposal for how the US could govern the development of frontier AI models, calling for a new industry oversight body modelled on the Financial Industry Regulatory Authority, the organisation that polices US brokerages and stock exchanges.
Hassabis frames the essay around what he calls the “foothills of the singularity,” arguing that AGI is likely only a few years away and that its arrival will reshape society at roughly ten times the scale and speed of the Industrial Revolution. He is candid about the risks that come with that trajectory, pointing to cybersecurity threats already visible in current models, and warning that nuclear and biological risks could emerge as capabilities keep climbing. His broader worry is that competitive pressure between labs and nations is pushing progress ahead of the field’s ability to understand what it’s building.
What The Body Would Actually Do
The proposal is for a Standards Body structured as a federally overseen public-private partnership, similar to FINRA, with a board made up of independent technical experts and open-source representatives. Funding would come mostly from industry itself, which Hassabis argues is necessary to hire top technical talent and pay for the compute needed to run large-scale evaluations.
Models would earn a “Frontier-class” designation by clearing benchmark thresholds set by the body, and companies that produce such models would be labelled “Frontier Labs.” Those labs would be nudged toward practices like publishing model cards, vetting key personnel internally, running strong cybersecurity programs, and funding dedicated safety research. In the initial phase, participation would be voluntary, with labs sharing models for review up to 30 days ahead of release. Hassabis expects that once the assessment process proves reliable, it could be formalised so that clearing the Standards Body becomes a requirement for deployment in the US market.
On the technical side, the evaluations would cover cybersecurity and biological threat capabilities, along with tests for whether agentic systems try to bypass safety guardrails or show signs of deception. Hassabis also wants labs to adopt watermarking for AI-generated images and to keep model reasoning in human-readable form. The benchmarks themselves would be revised on a quarterly cycle at first, with the body eventually building its own held-out tests so labs can’t simply train toward the known evaluation criteria. He also floats the idea of the body coordinating a deliberate slowdown across Frontier Labs if the situation calls for it, and says the framework should apply to any qualifying model regardless of where it was built or whether it’s open or closed source, while leaving smaller startups and academic projects untouched.
The Case For And Against
The strongest argument for the proposal is that it tries to match the pace of the industry it’s regulating. A body funded by the labs it oversees, with quarterly benchmark updates and the ability to build independent held-out tests, is a meaningfully different animal from the kind of static legislation that tends to lag years behind what frontier models can actually do. Modelling it on FINRA also gives it a credible template — a self-regulatory body with government backing has precedent in finance, and applying that structure to AI could sidestep the gridlock that comes with getting Congress to pass detailed technical rules. The voluntary-to-mandatory pathway is also a sensible way to test the system before locking companies into it.
The obvious tension is that an organisation funded largely by the companies it evaluates, developed initially in consultation with those same companies, invites questions about how independent its judgment can really be. Frontier Labs would have real influence over which benchmarks get built and how thresholds get set, at least until the body develops its own testing capacity — and getting there could take years, during which the incumbents effectively help write the rules they’re graded against. There’s also a geopolitical wrinkle: applying the framework to models “no matter their country of origin” sounds neat on paper, but enforcing US-set standards on labs in China or elsewhere is a different proposition entirely from getting American companies to comply, and Hassabis’s essay doesn’t really wrestle with how that gap gets closed. It remains to be seen how the proposal is received, but given Hassabis’s proposal, and the recent open letter signed by top scientists and researchers, the best minds in AI seem just as concerned about regulating it as building it at the moment.