Terminal-Bench-Science measures agents on research workflows, not textbook questions. A vendor-run result released days later shows how quickly its baseline can move.
Determine the orbit of a newly observed near-Earth object. Vet TESS transit candidates. Quantify the phases in a multiphase powder from its X-ray diffraction pattern. Reconstruct cell lineages from long-horizon live-cell imaging. Harmonize MRI scans collected on different scanners. Formalize Onsager's solution of the two-dimensional Ising model in Lean.
Those are six of the 70 tasks in Terminal-Bench-Science v0.1, released on August 26 by the team behind Terminal-Bench and the Harbor evaluation framework. The benchmark places an LLM agent in a sandboxed terminal, gives it a research workflow, and grades the artifacts it produces, including analyses, simulations, proofs, code, and data products, with reproducible task-specific tests. Oracle solutions accompany the tasks.
At launch, the top score on the public leaderboard was 30.0 percent. Five days later, that figure was no longer a plausible ceiling.
Claude Opus 5 running under Claude Code resolved 30.0 percent of tasks across three independent trials per task, according to the launch announcement. GPT-5.6 Sol with Codex reached 22.4 percent, and Claude Fable 5 reached 21.4 percent. Claude Opus 4.8 managed 10.5 percent; GPT-5.6 Terra, Kimi K3, and Grok 4.6 all finished below 10 percent. GLM 5.3 was the strongest open model listed at 8.1 percent, while GPT-5.6 Luna finished last at 3.3 percent.
On September 1, Anthropic reported that Claude Fable 5.1 scored 52.6 percent in the company's own Terminal-Bench-Science evaluation. Anthropic said its setup reproduced Opus 5 at 29.0 percent and Fable 5 at 24.7 percent, both within the reported uncertainty of the public results. The 52.6 percent figure was not yet listed on the benchmark's launch leaderboard when this article was edited, so it should be read as a vendor-reported result rather than a directly accepted leaderboard entry. Even with that caveat, it makes the central point vivid: a continuous benchmark can become stale within days if new models are not incorporated quickly.
The benchmark team says Terminal-Bench-Science separates systems about as well as Terminal-Bench 3.0 while reducing resolution rates by more than 10 percentage points for every model evaluated on both. That gap was partly deliberate. During review, tasks were calibrated against Opus 5 and GPT-5.6 Sol so that the first release would challenge the frontier rather than arrive saturated.
Agents have spent 2026 moving from demos into operational roles, inside codebases and corporate networks alike. Scientific research is a natural next target. Terminal-Bench-Science's contribution is not proof that agents can do science autonomously; it is a public test of how reliably particular agent-model combinations can complete bounded computational workflows.
The launch results also expose an open-model gap. GLM 5.3 led the open models at 8.1 percent, and Kimi K3 was one of only two systems, alongside Opus 5, that appeared on both the cost and token-efficiency Pareto frontiers. Those are results on this benchmark release, not a general comparison across all agent or coding benchmarks.
The project is led by Steven Dillmann, a Ph.D. student in Stanford's Institute for Computational and Mathematical Engineering. Sanmi Koyejo and Ludwig Schmidt are listed with him under project leadership, and the launch announcement thanks them as project-lead advisors. Stanford and the Laude Institute host the collaboration, in partnership with the Stanford AI Lab, Stanford HAI, Stanford AIMS, the NSF AI Institute for Foundations of Machine Learning, the Allen Institute, and Ai2.
The announcement states the governance thesis plainly: scientists, rather than model developers or data vendors, should set the bar for scientific capability in AI.
That thesis is visible in the submission funnel. Contributors filed 920 task proposals; 464 were approved for implementation, 386 became pull requests, and 70 were included in v0.1. The project reports 376 contributors across 22 countries. Each merged task passed automated checks, parallel scientific and technical review, and final bar-raiser approval. The repository documents a 39-criterion harbor check, TF-IDF similarity screening, oracle and no-op validation, and adversarial cheat trials intended to expose weak graders.
The team is equally explicit about what it does not want to build: a benchmark published once and then abandoned. Terminal-Bench-Science is versioned like software. Regular releases can add tasks, retire saturated or underspecified ones, and recalibrate difficulty against the current frontier. Trials can be reused, regraded, or rerun with Harbor. The Apache-2.0 repository has a Zenodo concept DOI and a public task dashboard. Argonne made a related maintenance choice last month when it moved its ChemGraph leaderboard to monthly runs.
That maintenance model matters to anyone assessing claims about agentic AI for science. Public programs are already testing the premise, including DOE's Genesis-Science-1 partnership with Arcee AI. A scientist-led, reproducible benchmark can help funders and laboratories distinguish demonstrated workflow performance from product claims. Its first lesson is not a timeless 30 percent ceiling, but a baseline that moved sharply almost as soon as it was published.
The launch runs show that evaluation itself is an infrastructure expense. A full three-trial evaluation of Claude Opus 5 across the 70 tasks cost about $7,000 in inference. Claude Fable 5 cost $14,180, and GPT-5.6 Sol cost $4,220. In the launch announcement's comparison, Sol nearly matched Fable 5's public score for less than one-third of the cost.
Tokens tell a different story. Fable 5 consumed 6.4 billion tokens to Sol's 8.4 billion, about 24 percent fewer, despite similar launch performance. Kimi K3 and Opus 5 were the only two systems on both the dollar-cost and token-use Pareto frontiers.
SCN has argued before that agentic workloads carry real infrastructure economics even as per-token prices fall. Evaluation belongs in that bill. Comparing several frontier systems can create a five-figure expense per release cycle, and cost per resolved task may be more useful for procurement than cost per token alone.
The leaderboard evaluations were supported by API credits from Bespoke Labs, Anthropic, Google, Moonshot AI, xAI, and Z.ai, with Modal among the organizations providing infrastructure support. That means the benchmark's governance design must carry real weight: vendors can supply evaluation resources, but scientists need control over task selection, review, and scoring.
Version 0.2 is already open, with pull requests due October 5, 2026. The README says the project is growing toward more than 100 tasks. Contributions move through a public propose-build-review process on GitHub and Discord, with weekly meetings and office hours listed on an open calendar.
The launch domain results show where the first release was hardest. Mathematical sciences was the only domain in which Opus 5 did not lead: Fable 5 scored 33.3 percent and GPT-5.6 Sol 31.4 percent. Engineering sciences produced the lowest leading scores, with Grok 4.6 and Sol tied for second at 14.8 percent. Anthropic and OpenAI systems held the top two positions in every launch domain except engineering sciences.
For practitioners, the useful product is narrower than any headline percentage. The task set is a public, versioned map of bounded research workflows that agents can and cannot complete under a specific harness. The next question is not whether one score proves that agents can do science. It is how quickly the map changes, whether new results reproduce outside vendor evaluations, and whether the benchmark's review process keeps pace.
The SCN Staff is a small AI editorial squad working under human direction. Each agent owns one job.
Scout does the research. It runs down primary sources and checks what's already been published, on SCN and everywhere else, before a story gets written. If a claim can't be traced back to a real document, Scout flags it.
Forge writes. It takes what Scout found and turns it into a draft, argument and sentences and all. Every SCN piece starts here, then gets sharpened.
Cipher handles search: the titles, descriptions, and keyphrase work that decides whether a good article ever gets found. Least glamorous job on the squad. Also one that matters more than it looks.
Pixel makes the visuals. Images, charts, the occasional diagram, all built to SCN's brand instead of pulled from a stock library. When something's easier to see than to read, it goes to Pixel.
Editorial judgment and the final call stay with the humans. So does the fact-checking.
