OpenAI tests AI agents on 100 wet-lab tasks with BioBasicsBench

By Rebecca Stone

OpenAI introduced BioBasicsBench on November 6, 2026, with 100 wet-lab tasks developed alongside enzyme-design company Ligo to test whether AI agents can interpret raw evidence from routine laboratory experiments.

OpenAI says BioBasicsBench addresses a limitation in evaluations built around charts and images selected for scientific papers. Everyday lab work also involves failed experiments, ambiguous images, conflicting observations and troubleshooting, so OpenAI and Ligo built tasks around the raw record of routine experiments.

The benchmark matters for progress toward laboratory assistance. OpenAI says development in these capabilities will also inform its assessment of their biosafety and biosecurity consequences.

What does BioBasicsBench measure?

OpenAI reports results as mean reward, which it defines as the average task score expressed as a percentage of available credit. Its task rubrics award credit for elements including numerical results, supporting reasoning and “recognition that the evidence is insufficient.”

Scientists created the tasks from their own experiments and wrote the grading rubrics, OpenAI said. The tasks cover:

• protein methods;
• imaging and cytometry;
• quantification and assays;
• nucleic acid methods; and
• molecular cloning.

OpenAI says the tasks ask agents to interpret experimental data, spot errors and recognize when the evidence cannot justify a conclusion. Credit therefore depends on whether an answer respects the record, including cases in which no supported conclusion is possible.

OpenAI lists the evidence supplied to agents:

• gel and blot images;
• colony plates;
• microscopy stacks;
• sequencing reads;
• plate-reader exports;
• bench photographs; and
• protocol videos.

Individual tasks can combine several evidence types with supporting reference sheets, such as ladder charts and plasmid maps. Agents return structured conclusions, measurements or recommendations. The rubrics evaluate the result, the supporting reasoning and whether the agent recognizes when the evidence cannot support a definitive answer.

How did OpenAI build and score the benchmark?

OpenAI says each task received an internal review by a Ligo scientist who had not written it. A PhD-level wet-lab scientist conducted an external review, and a final internal review covered the prompt, evidence, rubric, author notes and a graded model attempt. The final pass therefore covered both the written materials and the behavior of a model on the task.

The evaluation covered 37 model and reasoning-effort settings. OpenAI ran five attempts on every task for each setting, producing 18,500 results. It calculated confidence intervals by resampling whole tasks while keeping each task’s five attempts together. The 18,500-result count includes every task, setting and attempt rather than only unique questions.

At “xhigh,” the highest reasoning-effort setting reported for all seven models tested, GPT-6 Astra reached 66.7% mean reward with a 95% confidence interval of 60.3% to 73.1%. GPT-5.6 Sol scored 43.8%, with a 95% confidence interval of 37.6% to 50.0%.

GPT-6 Astra averaged 4,704 output tokens per task, compared with 9,099 for GPT-5.6 Sol. OpenAI’s results place Astra higher on mean reward while recording lower output-token use at the reported xhigh setting.

What separated stronger and weaker conclusions?

OpenAI’s first worked example concerns cell imaging. GPT-5.6 Sol classified cells as dead from a dye signal. GPT-6 Astra recognized that overlapping detection windows could allow interference from a second dye and called for controls that could separate the signals.

In a DNA-cloning task, Sol based its conclusion on seeing no colonies on a negative-control plate. Astra noticed that the plate was labeled for kanamycin rather than the ampicillin used in the experiment, and rejected it as a reliable control.

A third task showed cells moving in and out of focus. Sol sought a biological explanation for their apparent disappearance. Astra identified a z-stack, meaning a series of images captured at different depths, rather than treating the changing appearance as a biological change.

In each example, the stronger response used the experimental setup or the form of the image to limit the conclusion. OpenAI interprets the cases as evidence of better multimodal handling, use of experimental context and recognition of what data can and cannot establish.

The company says the examples also indicate that stronger native multimodal capabilities may be important for routine wet-lab analysis. OpenAI presents these as comparisons between model outputs, not as proof that either system can handle laboratory work reliably.

Where did GPT-6 Astra still fail?

OpenAI cautions that “higher scores do not yet amount to reliable laboratory judgment.” Its worked failures show how an agent can make a useful observation and still fail to carry it into a sound experimental decision.

In a colony-PCR task, Astra selected lanes 4, 6, 7 and 8 after assuming that a larger PCR product indicated successful cloning. The reference interpretation selected lanes 5, 9 and 10. Astra’s choice would have led to purification of the incorrect plasmid.

Another failure involved vacuoles, compartments inside cells. Astra generalized that vacuoles are generally darker than surrounding cell material despite lighter vacuoles in the same image. Changing focal depth can reverse the contrast, and Astra’s own intermediate observations noted that contrast varied with focus.

In a colony-counting task, OpenAI reports that Astra failed to separate close neighbors, missed visible colonies and marked empty agar as containing colonies. The result was an approximately 12% overcount relative to the benchmark reference.

With plated volume and dilution fixed, OpenAI says that overcount inflates the estimated efficiency of DNA introduction by the same proportion. The error therefore affects the reported experimental estimate, not just the number of objects detected.

OpenAI also reports cases in which correct observations failed to govern the final answer. Those cases show why the benchmark cannot be reduced to its aggregate score: an agent may recognize relevant evidence while still violating controls, misreading images or overstating what an assay shows.

What must improve next?

OpenAI’s benchmark turns several laboratory requirements into targets for improvement. An agent must interpret experimental data, spot errors, check controls and withhold conclusions when the evidence cannot support them. It must also carry accurate observations through to its final conclusion or recommendation.

The remaining errors have direct consequences. Astra’s PCR selection would purify the wrong plasmid. Its colony count would inflate estimated DNA-introduction efficiency by about 12%. Its focus-related conclusion overlooked a contrast reversal already visible in its intermediate observations.

OpenAI says the practical bar is higher than a better aggregate score. For laboratory agents, the next measure of progress is whether higher scores translate into reliable experimental decisions, because a wrong plasmid or an inflated DNA-introduction estimate changes the result at the bench.