Ticket thread summary
Summarize long ticket threads into status, attempts, and next steps.
Provisional leader
gemini-3.1-pro-preview
Current leader based on limited benchmark evidence. Treat this ranking as directional until coverage improves.
22.6%
Best benchmark score
25.9%
Confidence
All ranked models — top 3
Ranked Models
30
Evidence Quality
80%
Evidence Points
21
Top Signal
SimpleQA Verified: simpleqa_verified_score_pct
All Ranked Models
| Rank | Model | Score |
|---|---|---|
| 🥇 | gemini-3.1-pro-preview Strong on SimpleQA Verified simpleqa_verified_score_pct and Vals Finance Agent overall_accuracy_pct | 22.6% |
| 🥈 | gemini-2.5-pro Strong on Galileo Agent Leaderboard v2 Avg AC and FACTS Benchmark Suite facts_grounding_score_pct | 21.8% |
| 🥉 | gpt-4.1-20250414 Strong on Galileo Agent Leaderboard v2 Avg AC and Vectara HHEM Leaderboard overall_hallucination_error_pct | 19.8% |
| #4 | gpt-5-2025-08-07 Strong on SciArena Leaderboard rating_elo and FACTS Benchmark Suite facts_grounding_score_pct | 19.5% |
| #5 | Grok-4-0709 Strong on Galileo Agent Leaderboard v2 Avg AC and Vals Finance Agent overall_accuracy_pct | 19.3% |
| #6 | gpt-5-mini-2025-08-07 Strong on Vals Finance Agent overall_accuracy_pct and SciArena Leaderboard rating_elo | 18.8% |
| #7 | gemini-3-pro-preview Strong on SimpleQA Verified simpleqa_verified_score_pct and Vals Finance Agent overall_accuracy_pct | 18.2% |
| #8 | claude-sonnet-4 Strong on Galileo Agent Leaderboard v2 Avg AC and Vectara HHEM Leaderboard overall_hallucination_error_pct | 18.2% |
| #9 | gemini-2.5-flash Strong on Galileo Agent Leaderboard v2 Avg AC and Vectara HHEM Leaderboard overall_hallucination_error_pct | 16.1% |
| #10 | gpt-5.2-2025-12-11 Strong on FACTS Benchmark Suite facts_grounding_score_pct and Vals Finance Agent overall_accuracy_pct | 16.0% |
| #11 | gemini-3-flash-preview Strong on Vectara HHEM Leaderboard overall_answer_rate_pct and Vals Finance Agent overall_accuracy_pct | 15.6% |
| #12 | claude-sonnet-4.6 Strong on Vals Finance Agent overall_accuracy_pct and Vectara HHEM Leaderboard overall_answer_rate_pct | 15.5% |
| #13 | gemini-3.1-flash-lite-preview Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and FACTS Benchmark Suite facts_grounding_score_pct | 14.7% |
| #14 | gpt-5.4-2026-03-05 Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and Vals Finance Agent overall_accuracy_pct | 14.7% |
| #15 | gpt-5.1-2025-11-13 Strong on Vals Finance Agent overall_accuracy_pct and Vals Case Law v2 overall_accuracy_pct | 13.6% |
| #16 | o3-20250416 Strong on SciArena Leaderboard rating_elo and SimpleQA Verified simpleqa_verified_score_pct | 13.5% |
| #17 | claude-opus-4-5-20251101 Strong on Vectara HHEM Leaderboard overall_answer_rate_pct and Vectara HHEM Leaderboard overall_hallucination_error_pct | 13.0% |
| #18 | grok-4-fast-reasoning Strong on Vectara HHEM Leaderboard overall_answer_rate_pct and Vals Finance Agent overall_accuracy_pct | 12.4% |
| #19 | qwen-2.5-72b-instruct Strong on Galileo Agent Leaderboard v2 Avg AC and DuckDB NSQL Leaderboard all_execution_accuracy | 12.3% |
| #20 | gpt-4o Strong on CRMArena Function Calling overall_score_pct and JSONSchemaBench Leaderboard medium_schema_compliance_pct | 11.9% |
| #24 | grok-4-1-fast-reasoning Strong on Vals Finance Agent overall_accuracy_pct and Vectara HHEM Leaderboard overall_answer_rate_pct | 11.2% |
| #25 | Qwen3-Embedding-4B Strong on MTEB Retrieval and Rerank (Official) retrieval_score_pct and MTEB Classification Official classification_score_pct | 11.2% |
| #33 | Claude-3.5-Sonnet Strong on CRMArena Function Calling overall_score_pct and DuckDB NSQL Leaderboard all_execution_accuracy | 10.5% |
| #36 | claude-opus-4-6-thinking Strong on Vals Finance Agent overall_accuracy_pct and Vals Finance Agent complex_retrieval_accuracy_pct | 10.3% |
| #43 | gemini-2.5-flash-lite Strong on Galileo Agent Leaderboard v2 Avg AC and Vectara HHEM Leaderboard overall_hallucination_error_pct | 9.7% |
| #48 | claude-opus-4-5-20251101-thinking Strong on Vals Finance Agent overall_accuracy_pct and Vals Case Law v2 overall_accuracy_pct | 9.6% |
| #55 | deepseek-v3 Strong on Galileo Agent Leaderboard v2 Avg AC and Vectara HHEM Leaderboard overall_hallucination_error_pct | 9.3% |
| #57 | kimi-k2.5-thinking Strong on Vals Finance Agent overall_accuracy_pct and Vals Finance Agent complex_retrieval_accuracy_pct | 9.3% |
| #59 | grok-3 Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and Vectara HHEM Leaderboard overall_answer_rate_pct | 9.2% |
| #60 | claude-opus-4-1-20250805 Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and Vectara HHEM Leaderboard overall_answer_rate_pct | 9.2% |
Compare Models
▶Ranking diagnostics & missing models
Source lift
Ranked
54
Sources
8
Quality
Low
Vals Legal Bench
Vals Tax Eval v2
Vals MedQA
Vals LiveCodeBench
Missing frontier models
No obvious gaps right now.
▶Taxonomy & task details
Core tasks
Required modes
Domains
Related in CX
Agent-assist reply suggestions
Draft replies for human agents with tone and policy constraints.
Support dialogue agent
Multi-turn support conversations with escalation and policy awareness.
Support bot (RAG grounded)
Support chatbot grounded in docs with optional citations and escalation.
Customer feedback theme mining
Extract themes and trends from reviews, tickets, and surveys.