Introducing GeneBench-Pro
OpenAI introduced GeneBench-Pro, a research benchmark for testing whether AI agents can make the judgment calls needed in computational biology. The benchmark is aimed at a gap in ordinary coding tests: an agent may execute code correctly and still choose an unsuitable analysis, overlook data-quality problems, or draw a conclusion the data cannot support.
What the benchmark tests
GeneBench-Pro contains 129 synthetic problems across 10 domains and 21 subdomains. Each task gives an agent a realistic dataset, short experimental context, and a target question. The agent must inspect the data, select and revise an analytical approach, and return a result tied to a downstream decision. Because the data are generated from known causal structures, the authors can grade answers against known targets and test whether plausible but incorrect methods fail.
OpenAI reports that its strongest tested model, GPT‑5.6 Sol, passed 28.7% of problems at its highest reasoning setting and 31.5% with Pro mode. Those figures show progress on this specific benchmark, while also showing that the model did not reliably complete most tasks. OpenAI says reviewers estimated a typical problem could take a human expert 20–40 hours; that estimate came from a survey, not a direct time study.
Why it matters—and what it does not show
The benchmark focuses on exploratory reasoning: detecting artifacts, revising assumptions, and knowing when evidence is strong enough for a decision. It does not establish that AI can replace biologists or safely run research without supervision. The results are from an OpenAI-authored evaluation, so readers should consider the benchmark design and seek independent replication.
Source: OpenAI’s GeneBench-Pro announcement and methodology.