Productivity

Meeting Summarization

Summarize meeting transcripts into action items, decisions, and key points.

task.summarize_meeting_transcripttask.write_memo_brief

Evidence quality is currently limited for this use case. Rankings below are useful for exploration, not a strong winner claim.

Provisional leader

gemini-3.1-pro-preview

Current leader based on limited benchmark evidence. Treat this ranking as directional until coverage improves.

22.9%

Best benchmark score

27.5%

Confidence

All ranked models — top 3

🥇

gemini-3.1-pro-preview

22.9%

🥈

gemini-2.5-pro

22.2%

🥉

gpt-5-2025-08-07

20.9%

Ranked Models

Evidence Quality

80%

Evidence Points

Top Signal

FACTS Benchmark Suite: facts_grounding_score_pct

All Ranked Models

30 of 30 models

Rank	Model	Score	Confidence	Price / 1M	Evidence sources
🥇	gemini-3.1-pro-preview Strong on FACTS Benchmark Suite facts_grounding_score_pct and Vectara HHEM Leaderboard overall_hallucination_error_pct	22.9%	27%	$4.50	FACTS Benchmark SuiteVectara HHEM Leaderboard
🥈	gemini-2.5-pro Strong on FACTS Benchmark Suite facts_grounding_score_pct and Vectara HHEM Leaderboard overall_hallucination_error_pct	22.2%	36%	$3.44	FACTS Benchmark SuiteVectara HHEM Leaderboard
🥉	gpt-5-2025-08-07 Strong on FACTS Benchmark Suite facts_grounding_score_pct and Vals Finance Agent overall_accuracy_pct	20.9%	32%	—	FACTS Benchmark SuiteVals Finance Agent
#4	gemini-2.5-flash Strong on LanguageBench Grammar/Clarity Official (Split) grammar_clarity_score_pct and FACTS Benchmark Suite facts_grounding_score_pct	20.7%	30%	$0.17	LanguageBench Grammar/Clarity Official (Split)FACTS Benchmark Suite
#5	claude-sonnet-4 Strong on LanguageBench Grammar/Clarity Official (Split) grammar_clarity_score_pct and Vectara HHEM Leaderboard overall_hallucination_error_pct	20.6%	30%	$6.00	LanguageBench Grammar/Clarity Official (Split)Vectara HHEM Leaderboard
#6	gpt-4.1-20250414 Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and MMLongBench-Doc Leaderboard acc_score_pct	19.0%	29%	—	Vectara HHEM LeaderboardMMLongBench-Doc Leaderboard
#7	gpt-5-mini-2025-08-07 Strong on FACTS Benchmark Suite facts_grounding_score_pct and Vectara HHEM Leaderboard overall_hallucination_error_pct	18.6%	29%	—	FACTS Benchmark SuiteVectara HHEM Leaderboard
#8	gemini-3.1-flash-lite-preview Strong on FACTS Benchmark Suite facts_grounding_score_pct and Vectara HHEM Leaderboard overall_hallucination_error_pct	17.1%	25%	$0.56	FACTS Benchmark SuiteVectara HHEM Leaderboard
#9	Grok-4-0709 Strong on FACTS Benchmark Suite facts_grounding_score_pct and Galileo Agent Leaderboard v2 Avg TSQ	17.1%	26%	—	FACTS Benchmark SuiteGalileo Agent Leaderboard v2
#10	gpt-5.2-2025-12-11 Strong on FACTS Benchmark Suite facts_grounding_score_pct and Vals Finance Agent overall_accuracy_pct	16.8%	20%	—	FACTS Benchmark SuiteVals Finance Agent
#11	gemini-3-pro-preview Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and Vals Finance Agent overall_accuracy_pct	16.5%	25%	$4.50	Vectara HHEM LeaderboardVals Finance Agent
#12	gemini-3-flash-preview Strong on FACTS Benchmark Suite facts_grounding_score_pct and Vectara HHEM Leaderboard overall_hallucination_error_pct	15.4%	24%	$1.13	FACTS Benchmark SuiteVectara HHEM Leaderboard
#13	claude-sonnet-4.6 Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and Vals Finance Agent overall_accuracy_pct	15.3%	20%	$6.00	Vectara HHEM LeaderboardVals Finance Agent
#14	gpt-5.4-2026-03-05 Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and Vals Finance Agent overall_accuracy_pct	15.2%	20%	—	Vectara HHEM LeaderboardVals Finance Agent
#15	claude-opus-4-5-20251101 Strong on FACTS Benchmark Suite facts_grounding_score_pct and Vectara HHEM Leaderboard overall_hallucination_error_pct	14.7%	23%	—	FACTS Benchmark SuiteVectara HHEM Leaderboard
#16	gpt-5.1-2025-11-13 Strong on FACTS Benchmark Suite facts_grounding_score_pct and Vals Finance Agent overall_accuracy_pct	13.2%	21%	—	FACTS Benchmark SuiteVals Finance Agent
#17	grok-4-fast-reasoning Strong on Vals Finance Agent overall_accuracy_pct and FACTS Benchmark Suite facts_grounding_score_pct	11.5%	27%	$0.28	Vals Finance AgentFACTS Benchmark Suite
#18	o3-20250416 Strong on Vals Mortgage Tax overall_accuracy_pct and SciArena Leaderboard rating_elo	11.5%	20%	$3.50	Vals Mortgage TaxSciArena Leaderboard
#19	Claude-3.5-Sonnet Strong on LanguageBench Grammar/Clarity Official (Split) grammar_clarity_score_pct and LLM-AggreFact Leaderboard average_score_pct	10.9%	13%	$6.00	LanguageBench Grammar/Clarity Official (Split)LLM-AggreFact Leaderboard
#21	claude-opus-4-1-20250805 Strong on FACTS Benchmark Suite facts_grounding_score_pct and Vectara HHEM Leaderboard overall_hallucination_error_pct	10.9%	20%	—	FACTS Benchmark SuiteVectara HHEM Leaderboard
#22	grok-4-1-fast-reasoning Strong on Vals Finance Agent overall_accuracy_pct and BFCL Multi-turn Official Multi Turn Acc	9.9%	19%	$0.28	Vals Finance AgentBFCL Multi-turn Official
#23	claude-opus-4-6-thinking Strong on Vals Finance Agent overall_accuracy_pct and Vals Mortgage Tax overall_accuracy_pct	9.7%	11%	—	Vals Finance AgentVals Mortgage Tax
#24	phi-4 Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and LanguageBench Grammar/Clarity Official (Split) grammar_clarity_score_pct	9.6%	13%	—	Vectara HHEM LeaderboardLanguageBench Grammar/Clarity Official (Split)
#25	gemini-2.5-flash-lite Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and Galileo Agent Leaderboard v2 Avg TSQ	9.5%	13%	$0.17	Vectara HHEM LeaderboardGalileo Agent Leaderboard v2
#26	gpt-4o-2024-08-06 Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and DuckDB NSQL Leaderboard all_execution_accuracy	9.5%	17%	—	Vectara HHEM LeaderboardDuckDB NSQL Leaderboard
#27	grok-3 Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and SimpleQA Verified simpleqa_verified_score_pct	9.3%	13%	$6.00	Vectara HHEM LeaderboardSimpleQA Verified
#29	claude-opus-4-5-20251101-thinking Strong on Vals Finance Agent overall_accuracy_pct and Vals Mortgage Tax overall_accuracy_pct	9.1%	11%	—	Vals Finance AgentVals Mortgage Tax
#30	gpt-4.1 Strong on LanguageBench Grammar/Clarity Official (Split) grammar_clarity_score_pct and DuckDB NSQL Leaderboard all_execution_accuracy	8.9%	11%	$3.50	LanguageBench Grammar/Clarity Official (Split)DuckDB NSQL Leaderboard
#31	deepseek-v3 Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and Galileo Agent Leaderboard v2 Avg TSQ	8.8%	14%	—	Vectara HHEM LeaderboardGalileo Agent Leaderboard v2
#32	kimi-k2.5-thinking Strong on Vals Finance Agent overall_accuracy_pct and Vals Mortgage Tax overall_accuracy_pct	8.7%	13%	—	Vals Finance AgentVals Mortgage Tax

Compare Models

Select two different models above to compare their evidence side by side.

▶Ranking diagnostics & missing models

Source lift

Ranked

Sources

Quality

Low

Vals Mortgage Tax

36 rows · 1.0% avg lift

Vals Legal Bench

34 rows · 0.3% avg lift

Vals MedQA

33 rows · 0.3% avg lift

Vals Tax Eval v2

33 rows · 0.3% avg lift

Missing frontier models

No obvious gaps right now.

▶Taxonomy & task details

Core tasks

task.summarize_meeting_transcripttask.write_memo_brief

Required modes

mode.long_context

Domains

domain.general_business

Related in Productivity

Knowledge base Q&A (fast, no citations)

Answer KB questions grounded in retrieved text without citations.

Knowledge base Q&A (with citations)

Answer questions grounded in an internal KB, with evidence.

Document summarization

Summarize long business documents into scannable outputs.

Decision memo

Recommend a decision with options, constraints, and risks.