Trillium Labs Launches to Do High-Stakes AI Research in the Open

Two industry scientists have founded a nonprofit, Trillium Labs, that will conduct AI research on some of the field's most sensitive topics — including recursive self-improvement and autonomous agents — and publish the details so outside researchers can study and replicate the work.

The launch, announced today, is a direct challenge to the way the largest AI companies handle their most powerful systems. Nathan Lambert and Tom Zick, the pair behind the nonprofit, argue that secrecy at frontier labs is actively degrading the field's ability to understand and manage the technology they are building.

"Over the past few millennia, humanity has had the scientific method in our toolbox as a way to mitigate harms and build better futures," Lambert tells WIRED. "The current closed trajectory of frontier AI development is taking us a step backwards."

Why the closed-versus-open fight matters

The stakes in this argument have grown sharply. Frontier models can now automate the discovery of new software vulnerabilities and automatically probe and hack into systems. Recent high-profile hacking sprees have prompted even greater scrutiny of who should control that capability.

The industry is split. Some of the biggest AI companies operate on the assumption that keeping models locked inside labs — accessible only to a chosen few — limits the damage they can do in the real world while researchers work out what the systems are capable of. Proponents of that limited-access view say it is crucial to keep powerful capabilities in the hands of a trusted few.

Lambert and Zick sit firmly in the other camp. They believe a shared understanding of the risks makes everyone better off, and that the current state of affairs leaves outside experts unable to scrutinize how frontier models are built, tuned, and behave.

The practical reality of access today reinforces their point. The world's most powerful models, like those from OpenAI and Anthropic, can only be reached through an app or an application programming interface. That access usually comes at the cost of transparency about how the model was constructed and how it behaves under different conditions.

There are counterexamples. Companies in China have released relatively powerful models that users can download and run on their own hardware. Xiaomi recently published live details of a major training run involving one of its models. At Stanford, researchers are pretraining the AI model Marin in the open. Trillium Labs aims to push that open approach into territory most actors avoid.

Who is behind it

Lambert previously worked at Ai2, a research lab that has taken an unusually open approach to AI, publishing details of the data and training methods used to build its models alongside the models themselves. Before that, he worked at Hugging Face. He runs a popular technical blog and founded American Truly Open Models, an initiative that encourages US companies to release more open models.

Zick worked at Harvard University and helped the brokerage Charles Schwab devise policies around "responsible AI."

The two met over Zoom during the COVID-19 pandemic, when both were graduate students at UC Berkeley working on AI. They got the idea for the nonprofit after watching industry AI research drift apart from academic work. Lambert says professors and students frequently cannot replicate what happens inside big company labs because they lack the necessary resources. Trillium Labs is their attempt to close that gap by making the experiments themselves public.

Post-training and recursive self-improvement first

Zick says Trillium Labs will initially focus on post-training — the process of fine-tuning large models after they have been built. Another priority is recursive self-improvement, or RSI, a process for developing new models by having AI contribute research. RSI has become one of the most contested ideas in the field: many researchers are alarmed by the prospect that ongoing progress could continue indefinitely, leading to a loss of human control.

The issue reached mainstream attention earlier this month when an Anthropic researcher left the company and warned that RSI could pose an existential threat to humankind. Trillium Labs intends to study exactly that territory — in public.

The nonprofit will also examine how reinforcement learning, which rewards a model for good results and punishes it for bad outcomes, can improve model capabilities. That approach has made agents far more capable, but it has also made them more inclined to do unexpected things. Lambert and Zick plan to study how reinforcement learning shapes the character and behavior of AI models — a pressing question when a model becomes overly sycophantic, for example.

"To understand something like how reinforcement learning scales in post-training, you need significant compute and a lot of careful experimentation," Zick says. She believes publishing details of how reinforcement training runs work could yield surprising insights once outside researchers begin scrutinizing them.

Funding and reception

Trillium Labs has raised an undisclosed sum from Schmidt Sciences, Halcyon Futures, and other backers. The founders say they aim to raise between $40 million and $100 million in total, and plan to spend $30 million on training over the next 18 months.

The approach has drawn support from policy circles. "I'm a massive fan of much more transparency than we currently have in R&D," Tim Fist, director of emerging technology policy at the Institute for Progress, a policy thinktank, tells WIRED.

A bid for nuance

Beyond the technical agenda, Lambert and Zick want Trillium Labs to change the character of the public debate over how AI should be built and governed.

"We're in an era of AI discourse dominated by a few world views," Lambert says. "We believe that the scientific method and careful measurement of recent events is the best way to understand new behaviors of AI models."

That framing positions the nonprofit as a third force in a fight that has largely polarized between closed frontier labs and the open-weights movement. By applying serious compute and rigorous, published experimentation to areas like RSI and agentic reinforcement learning — the very topics the closed labs treat as too dangerous to expose — Trillium Labs will test whether transparency can scale to the highest-stakes problems in AI. If its published results over the next 18 months draw meaningful replication and scrutiny from academic researchers, the case that openness and safety research are compatible gets much harder to dismiss.