Underwriting submission ingest
Convert messy submission docs into structured underwriting fields.
Provisional leader
gpt-4.1-20250414
Current leader based on limited benchmark evidence. Treat this ranking as directional until coverage improves.
20.9%
Best benchmark score
31.4%
Confidence
All ranked models — top 3
Ranked Models
30
Evidence Quality
80%
Evidence Points
20
Top Signal
MMLongBench-Doc Leaderboard: acc_score_pct
All Ranked Models
| Rank | Model | Score |
|---|---|---|
| 🥇 | gpt-4.1-20250414 Strong on MMLongBench-Doc Leaderboard acc_score_pct and Galileo Agent Leaderboard v2 Insurance AC | 20.9% |
| 🥈 | claude-sonnet-4 Strong on Galileo Agent Leaderboard v2 Insurance TSQ and Galileo Agent Leaderboard v2 Insurance AC | 19.5% |
| 🥉 | gemini-2.5-pro Strong on Galileo Agent Leaderboard v2 Insurance AC and Galileo Agent Leaderboard v2 Insurance TSQ | 18.6% |
| #4 | Grok-4-0709 Strong on Galileo Agent Leaderboard v2 Insurance TSQ and Galileo Agent Leaderboard v2 Insurance AC | 18.3% |
| #5 | gpt-5-2025-08-07 Strong on ExtractBench Paper Baselines credit_pass_rate_pct and Aider Polyglot Leaderboard percent_correct_pct | 16.6% |
| #6 | qwen-2.5-72b-instruct Strong on Galileo Agent Leaderboard v2 Insurance AC and DuckDB NSQL Leaderboard all_execution_accuracy | 15.6% |
| #7 | gemini-2.5-flash Strong on Galileo Agent Leaderboard v2 Insurance TSQ and Galileo Agent Leaderboard v2 Insurance AC | 14.7% |
| #14 | gemini-3.1-pro-preview Strong on Vals Mortgage Tax overall_accuracy_pct and Vals LiveCodeBench overall_accuracy_pct | 12.4% |
| #16 | Kimi-K2-Instruct Strong on Galileo Agent Leaderboard v2 Insurance AC and Galileo Agent Leaderboard v2 Insurance TSQ | 12.3% |
| #19 | gemini-3-pro-preview Strong on BFCL Multi-turn Official Multi Turn Acc and Vals Mortgage Tax overall_accuracy_pct | 11.7% |
| #21 | gpt-5.2-2025-12-11 Strong on BFCL Multi-turn Official Multi Turn Acc and FACTS Benchmark Suite facts_grounding_score_pct | 11.3% |
| #22 | gpt-5-mini-2025-08-07 Strong on Vals MedQA overall_accuracy_pct and Vals LiveCodeBench overall_accuracy_pct | 11.1% |
| #24 | o3-20250416 Strong on Aider Polyglot Leaderboard percent_correct_pct and Vals MedQA overall_accuracy_pct | 10.8% |
| #25 | gpt-4o-20241120 Strong on MMLongBench-Doc Leaderboard acc_score_pct and DuckDB NSQL Leaderboard all_execution_accuracy | 10.6% |
| #26 | gpt-4.1-mini-20250414 Strong on Galileo Agent Leaderboard v2 Insurance AC and Galileo Agent Leaderboard v2 Avg AC | 10.2% |
| #27 | gemini-2.5-flash-lite Strong on Galileo Agent Leaderboard v2 Insurance AC and Galileo Agent Leaderboard v2 Insurance TSQ | 10.2% |
| #28 | gpt-4o Strong on JSONSchemaBench Leaderboard medium_schema_compliance_pct and DuckDB NSQL Leaderboard all_execution_accuracy | 10.1% |
| #31 | gemini-3-flash-preview Strong on Vals Legal Bench overall_accuracy_pct and Vals MedQA overall_accuracy_pct | 10.0% |
| #34 | gpt-5.4-2026-03-05 Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and Vals MedQA overall_accuracy_pct | 9.5% |
| #35 | gpt-5.1-2025-11-13 Strong on Vals Case Law v2 overall_accuracy_pct and Vals MedScribe overall_accuracy_pct | 9.2% |
| #36 | claude-sonnet-4.6 Strong on Vals Finance Agent overall_accuracy_pct and Vals Tax Eval v2 overall_accuracy_pct | 9.1% |
| #39 | claude-opus-4-5-20251101 Strong on Vals Mortgage Tax overall_accuracy_pct and Vals MedQA overall_accuracy_pct | 8.9% |
| #41 | grok-4-1-fast-reasoning Strong on BFCL Multi-turn Official Multi Turn Acc and Vals CorpFin v2 overall_accuracy_pct | 8.7% |
| #42 | deepseek-v3 Strong on Galileo Agent Leaderboard v2 Insurance AC and Galileo Agent Leaderboard v2 Insurance TSQ | 8.7% |
| #46 | gemini-3.1-flash-lite-preview Strong on Vals Mortgage Tax overall_accuracy_pct and Vectara HHEM Leaderboard overall_hallucination_error_pct | 8.4% |
| #48 | o4-mini Strong on Aider Polyglot Leaderboard percent_correct_pct and Vals MedQA overall_accuracy_pct | 8.1% |
| #49 | kimi-k2.5-thinking Strong on Vals CorpFin v2 overall_accuracy_pct and Vals MedQA overall_accuracy_pct | 7.9% |
| #50 | gpt-4o-mini-2024-07-18 Strong on DuckDB NSQL Leaderboard all_execution_accuracy and LLM Trustworthy Leaderboard privacy | 7.8% |
| #51 | deepseek-r1 Strong on DuckDB NSQL Leaderboard all_execution_accuracy and Aider Polyglot Leaderboard percent_correct_pct | 7.8% |
| #53 | grok-4-fast-reasoning Strong on Vals CorpFin v2 overall_accuracy_pct and Vals Tax Eval v2 overall_accuracy_pct | 7.7% |
Compare Models
▶Ranking diagnostics & missing models
Source lift
Ranked
37
Sources
8
Quality
Low
Vals Legal Bench
Vals CorpFin v2
Vals MedQA
Vals Mortgage Tax
Missing frontier models
No obvious gaps right now.
▶Taxonomy & task details
Core tasks
Required modes
Domains
Related in Insurance
Litigation risk memo
Summarize a claim into litigation risk drivers and mitigation steps.
Policy wording comparison
Compare policy wording against a standard and flag material differences.
Fraud signal summary
Summarize potential fraud indicators with conservative evidence framing.
Claims summary
Summarize claim history into timeline, status, and open items.