Executive brief from metrics
Summarize key metric changes into an executive-ready brief.
Provisional leader
gpt-5-2025-08-07
Current leader based on limited benchmark evidence. Treat this ranking as directional until coverage improves.
22.7%
Best benchmark score
29.8%
Confidence
All ranked models — top 3
Ranked Models
30
Evidence Quality
81%
Evidence Points
27
Top Signal
LiveSQLBench: success_rate_pct
All Ranked Models
| Rank | Model | Score |
|---|---|---|
| 🥇 | gpt-5-2025-08-07 Strong on LiveSQLBench success_rate_pct and Spider2.0 Snow Text-to-SQL snow_text_to_sql_score_pct | 22.7% |
| 🥈 | deepseek-r1 Strong on DuckDB NSQL Leaderboard all_execution_accuracy and LiveSQLBench success_rate_pct | 22.5% |
| 🥉 | gpt-4o Strong on DuckDB NSQL Leaderboard all_execution_accuracy and JSONSchemaBench Leaderboard medium_schema_compliance_pct | 21.8% |
| #4 | o3-20250416 Strong on LiveSQLBench success_rate_pct and Spider2.0 Snow Text-to-SQL snow_text_to_sql_score_pct | 21.0% |
| #5 | qwen-2.5-72b-instruct Strong on DuckDB NSQL Leaderboard all_execution_accuracy and JSONSchemaBench Leaderboard medium_schema_compliance_pct | 21.0% |
| #6 | claude-sonnet-4 Strong on LiveSQLBench success_rate_pct and Galileo Agent Leaderboard v2 Avg AC | 20.4% |
| #9 | Claude-3.5-Sonnet Strong on DuckDB NSQL Leaderboard all_execution_accuracy and LLM-AggreFact Leaderboard average_score_pct | 18.6% |
| #10 | gpt-4o-20241120 Strong on DuckDB NSQL Leaderboard all_execution_accuracy and DuckDB NSQL Leaderboard hard_execution_accuracy | 18.1% |
| #11 | gpt-4.1 Strong on DuckDB NSQL Leaderboard all_execution_accuracy and SciArena Leaderboard rating_elo | 15.8% |
| #13 | gpt-4o-mini-2024-07-18 Strong on DuckDB NSQL Leaderboard all_execution_accuracy and DuckDB NSQL Leaderboard hard_execution_accuracy | 14.1% |
| #14 | gemini-2.5-pro Strong on Galileo Agent Leaderboard v2 Avg AC and Galileo Agent Leaderboard v2 Avg TSQ | 13.8% |
| #16 | gpt-4o-2024-08-06 Strong on DuckDB NSQL Leaderboard all_execution_accuracy and DuckDB NSQL Leaderboard hard_execution_accuracy | 13.3% |
| #17 | o4-mini Strong on LiveSQLBench success_rate_pct and SciArena Leaderboard rating_elo | 13.0% |
| #19 | gpt-5-mini-2025-08-07 Strong on SciArena Leaderboard rating_elo and Vals MedQA overall_accuracy_pct | 12.7% |
| #21 | gemini-3.1-pro-preview Strong on Vals LiveCodeBench overall_accuracy_pct and Vals SWE-bench overall_accuracy_pct | 12.5% |
| #24 | Grok-4-0709 Strong on Galileo Agent Leaderboard v2 Avg TSQ and Galileo Agent Leaderboard v2 Avg AC | 11.6% |
| #25 | gemini-2.0-flash-001 Strong on DuckDB NSQL Leaderboard all_execution_accuracy and DuckDB NSQL Leaderboard hard_execution_accuracy | 11.4% |
| #26 | gemini-3-pro-preview Strong on SciArena Leaderboard rating_elo and Vals Mortgage Tax overall_accuracy_pct | 11.3% |
| #31 | gpt-4.1-20250414 Strong on Galileo Agent Leaderboard v2 Avg AC and Galileo Agent Leaderboard v2 Avg TSQ | 10.9% |
| #35 | gemini-2.5-flash Strong on Galileo Agent Leaderboard v2 Avg TSQ and Galileo Agent Leaderboard v2 Avg AC | 10.3% |
| #36 | Llama-3.3-70B-Instruct Strong on DuckDB NSQL Leaderboard all_execution_accuracy and DuckDB NSQL Leaderboard hard_execution_accuracy | 10.3% |
| #37 | gemini-3-flash-preview Strong on Vals Legal Bench overall_accuracy_pct and Vals MedQA overall_accuracy_pct | 10.1% |
| #39 | gpt-5.2-2025-12-11 Strong on FACTS Benchmark Suite facts_grounding_score_pct and Vals Tax Eval v2 overall_accuracy_pct | 10.0% |
| #40 | claude-opus-4-6 Strong on LiveSQLBench success_rate_pct and AgentSet LLM Leaderboard elo_score | 9.8% |
| #41 | deepseek-v3 Strong on LiveSQLBench success_rate_pct and Galileo Agent Leaderboard v2 Avg AC | 9.8% |
| #42 | minimax-m2.1 Strong on LiveSQLBench success_rate_pct and Vals SWE-bench overall_accuracy_pct | 9.7% |
| #43 | gpt-5.4-2026-03-05 Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and Vals MedQA overall_accuracy_pct | 9.6% |
| #45 | gemma-2-27b-it Strong on DuckDB NSQL Leaderboard all_execution_accuracy and DuckDB NSQL Leaderboard hard_execution_accuracy | 9.4% |
| #46 | Qwen3-30B-A3B Strong on DuckDB NSQL Leaderboard all_execution_accuracy and DuckDB NSQL Leaderboard hard_execution_accuracy | 9.4% |
| #47 | gpt-5.1-2025-11-13 Strong on Vals Case Law v2 overall_accuracy_pct and Vals MedScribe overall_accuracy_pct | 9.3% |
Compare Models
▶Ranking diagnostics & missing models
Source lift
Ranked
50
Sources
8
Quality
Low
Vals Legal Bench
DuckDB NSQL Leaderboard
Vals MedQA
Vals Tax Eval v2
Missing frontier models
grok-4-1-fast-non-reasoning
Thin evidence after weightingRank #15
14.9%
▶Taxonomy & task details
Core tasks
Required modes
Domains
Related in Data & Analytics
SQL debugging
Diagnose and fix SQL queries for correctness and performance.
Metric definition workshop
Turn ambiguous KPI definitions into precise, measurable specs.
Dashboard narratives
Generate weekly KPI narratives and investigation suggestions.
Chart & Data Visualization Interpretation
Reading charts, graphs, and dashboards to extract insights and answer questions.