bench

latest release

v2026.09

v2026.09 · suite v2026.09 · generated Sep 8, 2026 · methodology

Releasesv2026.09v2026.09-smoke
Post on X

Overall score

codex · gpt-6-astra · high77.8

Overall score — higher is better.

API-equivalent cost

codex · gpt-6-astra · high$65.50

API-equivalent cost per full run — lower is better.

Wall time

codex · gpt-6-astra · high67m 19s

Wall time per full run — lower is better.

codex · gpt-6-astra · high leads at 77.8.

AgentOverallUX/UIFrontendBackendPlanning AuditBug FixTasteSignatureTokens inTokens outTurnsWall timeAPI-equiv. costScore / $Telemetry
01codex · gpt-6-astra · high77.8—55.6——100.0——5.5M111.7K767m 19s$65.501.2partial

Score vs. cost

API-equivalent cost per full run set against overall score, log scale. The shaded corner is cheap and good; the dashed line is the Pareto frontier — agents no other agent beats on both axes at once.

0255075100$41.32$56.18$76.36$103.80scoreAPI-equivalent cost (log)codex · gpt-6-astra · high
codexPareto frontier

Changelog

  • Sep 8, 2026v2026.09 published — 1 agents, 7 tasks.
  • Sep 6, 2026v2026.09-smoke published — 3 agents, 7 tasks.
  • Sep 6, 2026Task added: Inventory service (backend).
  • Sep 6, 2026Task added: Fix date range overlap bug (bugfix).
  • Sep 6, 2026Task added: Issue board filters (frontend).
  • Sep 6, 2026Task added: Codebase audit, planted defects (planning-audit).
  • Sep 6, 2026Task added: shipshit.dev landing hero (signature).
  • Sep 6, 2026Task added: Landing page, five themes (taste).
  • Sep 6, 2026Task added: Shipcut pricing page (ux-ui).