Benchmarks / Spider 2.0

Spider 2.0

Built by HKU and Salesforce AI Research · Fangyu Lei, Jixuan Chen, Yuxiao Ye, Ruisheng Cao, et al. · released 12 Nov 2024

Enterprise text-to-SQL workflow problems built from real business databases, often with more than 1,000 columns, hosted on BigQuery, Snowflake, SQLite, and DuckDB (dbt). An agent must explore the schema, read dialect documentation, and write long, multi-step SQL to return the right result table.

Frontier

96.7%

Execution accuracy (Spider 2.0-Snow)

Genloop's Sentinel Agent v2 Pro · agent: Sentinel Agent v2 Pro (Genloop)

1 Mar 2026 · Source: HKU (benchmark maintainers)

Top of the Spider 2.0-Snow table on 2026-09-23. The entry does not name the underlying model. Native mini (usenative.ai) is second at 96.53, dated 2026-02-18.

Saturated

The best result is at 90% or more of the ceiling.

See the full leaderboard at HKU

Dots are recorded results; the line is the best result so far; the red dot is the current frontier. Sources for every point are in the table below.

Embed this card

The image names the institutions behind the numbers and links back to this page.

HTML:

<a href="https://canagentswork.com/benchmarks/spider-2/"><img src="https://canagentswork.com/og/benchmarks-spider-2.png" width="600" height="315" alt="Spider 2.0: the best result is 96.7% (Genloop's Sentinel Agent v2 Pro, 1 Mar 2026)." loading="lazy"></a>

Markdown:

[![Spider 2.0: the best result is 96.7% (Genloop's Sentinel Agent v2 Pro, 1 Mar 2026).](https://canagentswork.com/og/benchmarks-spider-2.png)](https://canagentswork.com/benchmarks/spider-2/)

What it measures

Whether an agent can turn a business question into correct SQL against a large real-world warehouse. The headline here is Spider 2.0-Snow: 547 questions on Snowflake, graded by comparing the returned table with a gold table. The site also runs Spider 2.0-Lite (547 questions across BigQuery, Snowflake, and SQLite) and Spider 2.0-DBT (68 repository-level dbt tasks). We chose Snow because it is the default tab, has the most entries (64 on 2026-09-23), and is free to run.

Execution accuracy (Spider 2.0-Snow): Share of the 547 Snow questions where the submitted result table matches the gold table under the official evaluation suite. Entries that use oracle (ground-truth) tables are excluded from ranking. Higher is better.

Status compares the frontier with a ceiling of 100%.

Facts

Grain: Task level: bounded tasks with a clear spec
Environment: cli, api-tools
Grading: automated-tests
Tasks: 547
Contamination: Gold SQL and answers are public since 2024-12-24. Annotators rewrote each SQL to reduce leakage from the source projects. The maintainers ask users not to use the gold SQL for supervised fine-tuning.
Reuse: Repository is MIT licensed. All examples and gold answers were released for self-evaluation in December 2024. Maintainers ask users not to fine-tune on the gold SQL. (open-mit)

Limits to keep in mind

Leaderboard scores are self-reported by teams and then validated by the maintainers. Many top entries are closed commercial agents with no paper or code. Source
The evaluation suite was fixed on 2025-10-29 and affected scores were refreshed, so scores from before that date may not match current runs. Source
Snow scores above 96% since March 2026 leave little headroom on this variant. Lite (76.23 best) and DBT (65.6 best) still have room. Source
The shared Snowflake evaluation account was suspended in August 2026, which disrupted access to Spider 2.0-Snow. Source

Recorded results

A selection that shows the frontier over time. The full leaderboard is at the source.

System	Execution accuracy (Spider 2.0-Snow)	Date	Source
Genloop's Sentinel Agent v2 Pro · frontier	96.7%	1 Mar 2026	HKU · primary
Prism Swarm with Deepthink + Claude-Sonnet-4.5	90.5%	27 Jan 2026	HKU · primary
LingXi Agent + Claude-Sonnet-4.5	79.9%	5 Dec 2025	HKU · primary
ReFoRCE + o1-preview harness: ReFoRCE	31.3%	28 Jan 2025	HKU · primary
Spider-Agent + o1-preview harness: Spider-Agent	23.6%	30 Nov 2024	HKU · primary

Timeline

1 Mar 2026 — Spider 2.0-Snow leader reaches 96.70%. Source
12 Nov 2024 — Spider 2.0 launches with 632 enterprise text-to-SQL problems. Source

Where it sits in the atlas

Data and analytics Software engineering (partial)

Data engineering and SQL

Go to the source

Website spider2-sql.github.io
Paper arxiv.org
Full leaderboard spider2-sql.github.io
Code github.com

Last checked 23 Sep 2026 against 4 primary sources. See an error? Tell us.

How to cite

Credit the original work first: Spider 2.0 by HKU and Salesforce AI Research (https://arxiv.org/abs/2411.07763).

Then, if you used this page:

Can Agents Work. "Spider 2.0: frontier results and sources." https://canagentswork.com/benchmarks/spider-2/ (accessed 2026-09-24). CC BY 4.0.

@misc{caw-spider-2,
  title        = {{Spider 2.0: frontier results and sources}},
  author       = {{Can Agents Work}},
  year         = {2026},
  howpublished = {\url{https://canagentswork.com/benchmarks/spider-2/}},
  note         = {Accessed 2026-09-24. CC BY 4.0}
}