Source: LiveBench 2.0, release 2026-10-14. Snapshot checked . AI Tools Ratings organizes and derives the displayed ranking from published results; we did not run these evaluations.
What does each suite measure?
The current release contains seven suites. Short-answer tasks, produced artifacts and long-running tool use have different grading conditions. The same 0–100 display range does not make every point an interchangeable unit of real-world usefulness.
| Suite | Scope |
|---|---|
| Long Agentic | Long-running work across changing requirements and codebase operations. |
| Reasoning | Short logic, spatial and theory-of-mind tasks with checkable answers. |
| Coding | Bug-fix tasks from merged pull requests, evaluated on a frozen codebase. |
| Browser | Multi-step web-application tasks performed through a browser. |
| Data Analysis | Small-table analysis and data-format transformations. |
| Research | Written research briefs assessed against a task rubric. |
| Design | Presentation, PDF and document tasks assessed for visual and structural quality. |
How is the overall score calculated?
For a model with all seven suite scores, overall is their arithmetic mean: (long_agentic + reasoning + coding + browser + data_analysis + research + design) / 7. Each suite has equal weight. We do not mix this score with preference-voting Elo, vendor launch numbers or another benchmark version.
Example using invented numbers: seven suite scores of 70 have an overall mean of 70. If one score rises to 84 while the other six remain 70, the mean becomes 72. This arithmetic illustration is not a model result or an executed trial.
What happens when a score is missing?
A missing suite is stored as null and displayed as unavailable. It is never turned into zero. Models with incomplete coverage receive no overall rank on this site; they can still receive a rank in a suite with a recorded score. A measured zero remains a real result and participates in the calculation.
This eligibility rule is deliberately stricter than averaging only the available suites. It prevents an incomplete run from receiving an apparently comparable overall rank. The original LiveBench view may show an average across available suites, so incomplete models can appear differently there.
How do ranking, filtering and ties work?
The active suite determines the rank. Search and provider filters preserve that full-board rank instead of renumbering a small selection. Ordering by cost or name changes the display order, not the underlying capability rank. Overall and suite scores are sorted using unrounded values and displayed to two decimal places.
Equal unrounded scores share the same rank, with the next position skipped: 1, 1, 3. Names provide a stable display order for ties without implying that one tied model performed better. A small difference is not a confidence interval or proof of a practically important advantage.
What does USD per benchmark task mean?
The cost column uses the release’s recorded total task cost and question counts. Within the selected scope, we compute sum(recorded USD) / sum(measured questions). Overall cost requires usable cost and count data for all seven suites; an unavailable measurement remains blank. Costs are question-weighted, while the quality mean is suite-weighted.
This is the average recorded cost of attempted benchmark tasks, not cost per successful task, current API input/output token pricing, a subscription fee or the cost of your own workflow. Tasks can differ greatly in length and tool usage. Do not divide or compare these amounts as if every workload were identical. No currency conversion, guessed price or inferred token count is added.
If you need a production cost estimate, use the provider’s current pricing, a representative input mix and observed correction effort. Benchmark cost can identify a candidate worth testing, but cannot establish your invoice or return on investment.
Which model configuration is being compared?
Model IDs, display names, effort labels and source-reported open-weight flags come from the same release’s model manifest. They are not a guarantee that a similarly named chat application, quantized download or another provider endpoint behaves identically. The open-weight label is not a judgment that all licenses permit every intended use.
The measured tool harness, effort and task definitions matter. For a decision, compare two or three runs in the same release, then try a task representative of your own system. Inspect the actual output, not just the completion message, and record both failures and manual corrections.
How are snapshots updated and audited?
The source files are fetched as data, validated for model identity, known suite names and numeric ranges, and stored with SHA-256 hashes. Refreshing the data is a separate command from building or deploying the site. A failed fetch or an unexpected schema must not replace the last usable snapshot. The page shows both the source release and the date we inspected it; it does not claim a live connection.
Our additions are the full-coverage ranking gate, search and comparison interface, per-run evidence panels and task-selection guidance. The reported scores belong to LiveBench. No proprietary competitor composite score or copied visual assets are used.
Current source evidence
- table.csv
SHA-256 ea8778ffebcdc7acc99f4607c39441b2e98f303dde44a08a8c7008410d754363 - categories.json
SHA-256 b1e46eedc4a461e8ad8a414c24203c35aa760e759096b4211826b40a45f97279 - models.json
SHA-256 98012c492c898b2b72e5e622f166eebff6a20c4f6d217dcbb83bda84e2275439 - cost.csv
SHA-256 ed4712081baad05088e0bb81c490479805a1da0f0c63ac7005e256e02b6ffce1 - notes.json
SHA-256 a6bd6c8fe1ba7ce26dbdb46e20df6e270ce87e5e6677b90ac51fbe809da6fdbc
For the benchmark’s own interpretation, use the original LiveBench methodology. The downloadable snapshot records the release and check date with the scores. Source data can change under the same release URL; hashes identify the inspected bytes.
How should you use the board?
Start with the suite nearest your task, inspect coverage and effort, then compare the run’s strongest and lowest-scoring suites. Those extremes are descriptive within this benchmark and may reflect different task difficulty. Pilot the candidates using an unchanged fixture and judge whether their output meets your acceptance criteria.
Use our workflow evaluation guide for baseline and artifact checks, or read the model-family profiles for broader context. Report a source, identity or calculation concern so a correction can be traced to evidence.