OpenAI's BioBasicsBench tests agents on raw wet-lab evidence

OpenAI introduced BioBasicsBench on November 6, 2026, a 100-task benchmark developed with enzyme-design company Ligo to test whether AI agents can interpret raw evidence from routine wet-lab experiments. At xhigh, GPT-6 Astra reached 66.7% mean reward, while GPT-5.6 Sol scored 43.8%.

OpenAI reports that Astra averaged 4,704 output tokens per task, compared with 9,099 for GPT-5.6 Sol. The company also cautions that “higher scores do not yet amount to reliable laboratory judgment.” OpenAI says the practical bar extends beyond an aggregate score to the quality of each experimental decision.

BioBasicsBench addresses a gap in evaluations built around charts and images selected for scientific papers. OpenAI says routine lab work also involves failed experiments, ambiguous images, conflicting observations and troubleshooting. BioBasicsBench asks agents to handle that combination while producing a conclusion, measurement or recommendation the evidence can support.

OpenAI and Ligo built the benchmark around evidence from working experiments. OpenAI says development of the tested capabilities will also inform its assessment of their biosafety and biosecurity consequences.

What does BioBasicsBench measure?

OpenAI defines mean reward as the average task score expressed as a percentage of available credit. The evaluation covered 37 model and reasoning-effort settings, with five attempts on every task in each setting, producing 18,500 results.

OpenAI reports that at xhigh, the highest reasoning-effort setting reported for all seven models tested, GPT-6 Astra reached 66.7%, with a 95% confidence interval of 60.3–73.1%. GPT-5.6 Sol scored 43.8%, with an interval of 37.6–50.0%.

OpenAI calculated those confidence intervals by resampling whole tasks while keeping each task’s five attempts together. The method preserves the five-attempt grouping rather than treating every run as independent.

The reported score gap is clear at xhigh, but OpenAI’s warning keeps the result in perspective: the benchmark measures current capability, not reliable laboratory judgment.

How did OpenAI and Ligo build it?

OpenAI says its tasks ask agents to “interpret experimental data, spot errors and recognize when the evidence cannot justify a conclusion.” The benchmark covers protein methods, imaging and cytometry, quantification and assays, nucleic acid methods and molecular cloning.

Scientists created the tasks from their own experiments and wrote grading rubrics for them. OpenAI says those inputs capture the failures, ambiguities and conflicting observations that routine experiments can produce.

OpenAI lists the task inputs:

• Gel and blot images
• Colony plates
• Microscopy stacks
• Sequencing reads
• Plate-reader exports
• Bench photographs
• Protocol videos

OpenAI says individual tasks can combine several evidence types with supporting reference sheets, such as ladder charts and plasmid maps. Agents return structured conclusions, measurements or recommendations.

Under OpenAI’s rubric, a system can earn credit for declining an unsupported conclusion or identifying the limit of its evidence. The rubric also rewards numerical results and supporting reasoning.

OpenAI says every task received an internal review by a Ligo scientist who had not written it. Each task also received an external review by a PhD-level wet-lab scientist and a final internal review.

The final internal review covered the prompt, evidence, rubric, author notes and a graded model attempt. OpenAI says the three reviews examine both task construction and a sample model response.

Where did GPT-6 Astra outperform GPT-5.6 Sol?

OpenAI uses three worked examples to explain the score difference between GPT-6 Astra and GPT-5.6 Sol. Each turns on whether the model checks the cause of a signal, the validity of a control or the imaging context behind a visible result.

In OpenAI’s cell-imaging example, GPT-5.6 Sol classified cells as dead from a dye signal. GPT-6 Astra recognized that overlapping detection windows could permit interference from a second dye and called for controls that could separate the signals.

OpenAI’s DNA-cloning example shows GPT-5.6 Sol basing its conclusion on seeing no colonies on a negative-control plate. GPT-6 Astra noticed that the plate carried a kanamycin label rather than the ampicillin used in the experiment. It rejected the mismatched plate as a reliable control.

In OpenAI’s focus example, cells moved in and out of focus. GPT-5.6 Sol sought a biological explanation for their apparent disappearance. GPT-6 Astra identified a z-stack, a series of images captured at different depths.

OpenAI interprets these examples as evidence of better multimodal handling, use of experimental context and recognition of what data can and cannot establish. The company also presents them as an indication that stronger native multimodal capabilities may be important for routine wet-lab analysis.

Why did higher scores still produce errors?

OpenAI’s warning is not theoretical. Its worked failures include a wrong selection of PCR lanes, a conclusion that ignored a focus-dependent image change and a colony count that overstated DNA introduction efficiency.

In OpenAI’s colony-PCR example, GPT-6 Astra selected lanes 4, 6, 7 and 8 after assuming that a larger PCR product indicated successful cloning. The reference interpretation selected lanes 5, 9 and 10, so Astra’s choice would lead to purification of the incorrect plasmid.

OpenAI’s next failure involved vacuoles, compartments inside cells. GPT-6 Astra generalized that vacuoles are generally darker than surrounding cell material despite lighter vacuoles in the same image.

Changing focal depth can reverse that contrast. Astra’s own intermediate observations noted that contrast varied with focus, but its final conclusion still relied on the general rule.

In a colony-counting task, OpenAI says Astra failed to separate close neighbors, missed visible colonies and marked empty agar. The result was an approximately 12% overcount relative to the benchmark reference.

OpenAI notes that, with plated volume and dilution fixed, that overcount inflates the estimated efficiency of DNA introduction by the same proportion. The measurement error therefore changes the scientific estimate, not just the agent’s output.

OpenAI says these cases expose a gap between accurate observation and sound experimental judgment. A model can record a visible feature correctly and still draw a conclusion that a control, focus change or plate label contradicts.

What must laboratory agents improve?

OpenAI frames the next standard as consistency across an entire workflow. The agent must carry accurate observations into its conclusion, check controls and withhold conclusions that the evidence cannot support.

BioBasicsBench converts those requirements into graded targets:

• Carry correct observations through to the final answer
• Check experimental controls before relying on them
• Recognize and report insufficient or contradictory evidence

The rubric therefore scores more than the final answer. It rewards the numerical result, the reasoning and an explicit recognition of evidential limits.

OpenAI says following development of these capabilities will inform its assessment of their biosafety and biosecurity consequences. That makes BioBasicsBench relevant beyond model comparisons. It links multimodal laboratory assistance to safety questions raised by systems that interpret experimental evidence.

As OpenAI develops these systems, BioBasicsBench will test whether higher rewards track better decisions, not merely more confident descriptions of incomplete evidence.