Productivity

Executive briefing

Turn raw notes into a short executive brief with risks and actions.

task.write_memo_brieftask.tradeoff_analysis

Evidence quality is currently limited for this use case. Rankings below are useful for exploration, not a strong winner claim.

Provisional leader

gemini-3.1-pro-preview

Current leader based on limited benchmark evidence. Treat this ranking as directional until coverage improves.

31.9%

Best benchmark score

36.9%

Confidence

All ranked models — top 3

🥇

gemini-3.1-pro-preview

31.9%

🥈

gemini-2.5-pro

29.3%

🥉

gpt-5-2025-08-07

26.5%

Ranked Models

Evidence Quality

83%

Evidence Points

Top Signal

Vals Finance Agent: overall_accuracy_pct

All Ranked Models

30 of 30 models

Rank	Model	Score	Confidence	Price / 1M	Evidence sources
🥇	gemini-3.1-pro-preview Strong on Vals Finance Agent overall_accuracy_pct and SimpleQA Verified simpleqa_verified_score_pct	31.9%	37%	$4.50	Vals Finance AgentSimpleQA Verified
🥈	gemini-2.5-pro Strong on Galileo Agent Leaderboard v2 Avg TSQ and FACTS Benchmark Suite facts_grounding_score_pct	29.3%	47%	$3.44	Galileo Agent Leaderboard v2FACTS Benchmark Suite
🥉	gpt-5-2025-08-07 Strong on Vals Finance Agent overall_accuracy_pct and FACTS Benchmark Suite facts_grounding_score_pct	26.5%	35%	—	Vals Finance AgentFACTS Benchmark Suite
#4	claude-sonnet-4 Strong on Galileo Agent Leaderboard v2 Avg TSQ and LanguageBench Grammar/Clarity Official (Split) grammar_clarity_score_pct	26.0%	38%	$6.00	Galileo Agent Leaderboard v2LanguageBench Grammar/Clarity Official (Split)
#5	gpt-5-mini-2025-08-07 Strong on Vals Finance Agent overall_accuracy_pct and Vals Finance Agent complex_retrieval_accuracy_pct	25.9%	40%	—	Vals Finance AgentVals Finance Agent
#6	Grok-4-0709 Strong on Galileo Agent Leaderboard v2 Avg TSQ and Vals Finance Agent overall_accuracy_pct	24.5%	37%	—	Galileo Agent Leaderboard v2Vals Finance Agent
#7	gemini-2.5-flash Strong on Galileo Agent Leaderboard v2 Avg TSQ and LanguageBench Grammar/Clarity Official (Split) grammar_clarity_score_pct	23.7%	35%	$0.17	Galileo Agent Leaderboard v2LanguageBench Grammar/Clarity Official (Split)
#8	gemini-3-pro-preview Strong on Vals Finance Agent overall_accuracy_pct and SimpleQA Verified simpleqa_verified_score_pct	23.3%	34%	$4.50	Vals Finance AgentSimpleQA Verified
#9	claude-sonnet-4.6 Strong on Vals Finance Agent overall_accuracy_pct and Vals Finance Agent complex_retrieval_accuracy_pct	22.7%	29%	$6.00	Vals Finance AgentVals Finance Agent
#10	gpt-4.1-20250414 Strong on Galileo Agent Leaderboard v2 Avg TSQ and Vectara HHEM Leaderboard overall_hallucination_error_pct	22.0%	32%	—	Galileo Agent Leaderboard v2Vectara HHEM Leaderboard
#11	gpt-5.2-2025-12-11 Strong on Vals Finance Agent overall_accuracy_pct and FACTS Benchmark Suite facts_grounding_score_pct	21.9%	26%	—	Vals Finance AgentFACTS Benchmark Suite
#12	gemini-3.1-flash-lite-preview Strong on Vals Finance Agent overall_accuracy_pct and Vals Mortgage Tax overall_accuracy_pct	21.9%	32%	$0.56	Vals Finance AgentVals Mortgage Tax
#13	gpt-5.4-2026-03-05 Strong on Vals Finance Agent overall_accuracy_pct and Vals Mortgage Tax overall_accuracy_pct	21.9%	28%	—	Vals Finance AgentVals Mortgage Tax
#14	gemini-3-flash-preview Strong on Vals Finance Agent overall_accuracy_pct and FACTS Benchmark Suite average_score_pct	20.6%	31%	$1.13	Vals Finance AgentFACTS Benchmark Suite
#15	gpt-5.1-2025-11-13 Strong on Vals Finance Agent overall_accuracy_pct and Vals Case Law v2 overall_accuracy_pct	19.7%	29%	—	Vals Finance AgentVals Case Law v2
#16	claude-opus-4-5-20251101 Strong on Vals Mortgage Tax overall_accuracy_pct and FACTS Benchmark Suite facts_grounding_score_pct	18.4%	27%	—	Vals Mortgage TaxFACTS Benchmark Suite
#17	o3-20250416 Strong on Vals Mortgage Tax overall_accuracy_pct and SciArena Leaderboard rating_elo	17.6%	26%	$3.50	Vals Mortgage TaxSciArena Leaderboard
#18	grok-4-fast-reasoning Strong on Vals Finance Agent overall_accuracy_pct and Vals Finance Agent complex_retrieval_accuracy_pct	17.0%	36%	$0.28	Vals Finance AgentVals Finance Agent
#19	claude-opus-4-6-thinking Strong on Vals Finance Agent overall_accuracy_pct and Vals Mortgage Tax overall_accuracy_pct	15.8%	18%	—	Vals Finance AgentVals Mortgage Tax
#20	claude-opus-4-5-20251101-thinking Strong on Vals Finance Agent overall_accuracy_pct and Vals Mortgage Tax overall_accuracy_pct	14.8%	18%	—	Vals Finance AgentVals Mortgage Tax
#21	kimi-k2.5-thinking Strong on Vals Finance Agent overall_accuracy_pct and Vals Mortgage Tax overall_accuracy_pct	14.2%	21%	—	Vals Finance AgentVals Mortgage Tax
#22	claude-sonnet-4-5-20250929-thinking Strong on Vals Finance Agent overall_accuracy_pct and Vals Mortgage Tax overall_accuracy_pct	13.9%	18%	—	Vals Finance AgentVals Mortgage Tax
#23	grok-4-1-fast-reasoning Strong on Vals Finance Agent overall_accuracy_pct and Vectara HHEM Leaderboard business_answer_rate_pct	13.7%	25%	$0.28	Vals Finance AgentVectara HHEM Leaderboard
#24	claude-opus-4-1-20250805 Strong on Vals Mortgage Tax overall_accuracy_pct and Vectara HHEM Leaderboard overall_hallucination_error_pct	13.5%	25%	—	Vals Mortgage TaxVectara HHEM Leaderboard
#25	Claude-3.5-Sonnet Strong on LanguageBench Grammar/Clarity Official (Split) grammar_clarity_score_pct and LLM-AggreFact Leaderboard average_score_pct	13.2%	17%	$6.00	LanguageBench Grammar/Clarity Official (Split)LLM-AggreFact Leaderboard
#27	grok-3 Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and SimpleQA Verified simpleqa_verified_score_pct	12.1%	18%	$6.00	Vectara HHEM LeaderboardSimpleQA Verified
#28	grok-4.20-0309-reasoning Strong on Vals Finance Agent overall_accuracy_pct and Vals Finance Agent complex_retrieval_accuracy_pct	12.0%	18%	—	Vals Finance AgentVals Finance Agent
#29	gemini-2.5-flash-lite Strong on Galileo Agent Leaderboard v2 Avg TSQ and Vectara HHEM Leaderboard overall_hallucination_error_pct	11.8%	15%	$0.17	Galileo Agent Leaderboard v2Vectara HHEM Leaderboard
#30	claude-haiku-4-5-20251001-thinking Strong on Vals Finance Agent overall_accuracy_pct and Vals Mortgage Tax overall_accuracy_pct	11.8%	18%	—	Vals Finance AgentVals Mortgage Tax
#31	mistral-large-2512 Strong on Vals Mortgage Tax overall_accuracy_pct and Vectara HHEM Leaderboard business_answer_rate_pct	11.6%	24%	—	Vals Mortgage TaxVectara HHEM Leaderboard

Compare Models

Select two different models above to compare their evidence side by side.

▶Ranking diagnostics & missing models

Source lift

Ranked

Sources

Quality

Low

Vals Mortgage Tax

38 rows · 1.6% avg lift

Vals Legal Bench

38 rows · 0.5% avg lift

Vals MedQA

37 rows · 0.5% avg lift

Vals Tax Eval v2

36 rows · 0.4% avg lift

Missing frontier models

No obvious gaps right now.

▶Taxonomy & task details

Core tasks

task.write_memo_brieftask.tradeoff_analysis

Required modes

none

Domains

domain.general_business

Related in Productivity

Knowledge base Q&A (fast, no citations)

Answer KB questions grounded in retrieved text without citations.

Knowledge base Q&A (with citations)

Answer questions grounded in an internal KB, with evidence.

Document summarization

Summarize long business documents into scannable outputs.

Decision memo

Recommend a decision with options, constraints, and risks.