Candidate summary memo

Summarize a candidate profile into strengths, gaps, and questions.

task.summarize_doctask.write_memo_brief

Evidence quality is currently limited for this use case. Rankings below are useful for exploration, not a strong winner claim.

Provisional leader

gpt-4.1-20250414

Current leader based on limited benchmark evidence. Treat this ranking as directional until coverage improves.

23.1%

Best benchmark score

35.0%

Confidence

All ranked models — top 3

🥇

gpt-4.1-20250414

23.1%

🥈

claude-sonnet-4

20.0%

🥉

Grok-4-0709

16.9%

Ranked Models

Evidence Quality

79%

Evidence Points

Top Signal

MMLongBench-Doc Leaderboard: acc_score_pct

All Ranked Models

30 of 30 models

Rank	Model	Score	Confidence	Price / 1M	Evidence sources
🥇	gpt-4.1-20250414 Strong on MMLongBench-Doc Leaderboard acc_score_pct and Galileo Agent Leaderboard v2 Avg TSQ	23.1%	35%	—	MMLongBench-Doc LeaderboardGalileo Agent Leaderboard v2
🥈	claude-sonnet-4 Strong on Galileo Agent Leaderboard v2 Avg TSQ and LanguageBench overall:mean	20.0%	26%	$6.00	Galileo Agent Leaderboard v2LanguageBench
🥉	Grok-4-0709 Strong on Galileo Agent Leaderboard v2 Avg TSQ and Galileo Agent Leaderboard v2 Avg AC	16.9%	25%	—	Galileo Agent Leaderboard v2Galileo Agent Leaderboard v2
#4	gemini-2.5-pro Strong on Galileo Agent Leaderboard v2 Avg TSQ and Galileo Agent Leaderboard v2 Avg AC	16.8%	27%	$3.44	Galileo Agent Leaderboard v2Galileo Agent Leaderboard v2
#5	gemini-2.5-flash Strong on Galileo Agent Leaderboard v2 Avg TSQ and LanguageBench overall:mean	15.9%	22%	$0.17	Galileo Agent Leaderboard v2LanguageBench
#6	gpt-4o-20241120 Strong on MMLongBench-Doc Leaderboard acc_score_pct and DuckDB NSQL Leaderboard all_execution_accuracy	14.7%	24%	—	MMLongBench-Doc LeaderboardDuckDB NSQL Leaderboard
#7	gpt-5-2025-08-07 Strong on Aider Polyglot Leaderboard percent_correct_pct and Vals MedQA overall_accuracy_pct	14.6%	21%	—	Aider Polyglot LeaderboardVals MedQA
#8	qwen-2.5-72b-instruct Strong on Galileo Agent Leaderboard v2 Avg TSQ and DuckDB NSQL Leaderboard all_execution_accuracy	14.1%	20%	—	Galileo Agent Leaderboard v2DuckDB NSQL Leaderboard
#9	gemini-3.1-pro-preview Strong on Vals Mortgage Tax overall_accuracy_pct and Vals GPQA overall_accuracy_pct	13.6%	15%	$4.50	Vals Mortgage TaxVals GPQA
#11	gemini-3-pro-preview Strong on BFCL Multi-turn Official Multi Turn Acc and Vals Mortgage Tax overall_accuracy_pct	12.8%	17%	$4.50	BFCL Multi-turn OfficialVals Mortgage Tax
#13	gpt-5.2-2025-12-11 Strong on BFCL Multi-turn Official Multi Turn Acc and FACTS Benchmark Suite facts_grounding_score_pct	12.4%	15%	—	BFCL Multi-turn OfficialFACTS Benchmark Suite
#14	gpt-5-mini-2025-08-07 Strong on Vals MedQA overall_accuracy_pct and Vals LiveCodeBench overall_accuracy_pct	12.2%	18%	—	Vals MedQAVals LiveCodeBench
#16	o3-20250416 Strong on Aider Polyglot Leaderboard percent_correct_pct and Vals MedQA overall_accuracy_pct	11.8%	18%	$3.50	Aider Polyglot LeaderboardVals MedQA
#18	gpt-4o Strong on JSONSchemaBench Leaderboard medium_schema_compliance_pct and DuckDB NSQL Leaderboard all_execution_accuracy	11.1%	15%	$0.26	JSONSchemaBench LeaderboardDuckDB NSQL Leaderboard
#19	gemini-3-flash-preview Strong on Vals Legal Bench overall_accuracy_pct and Vals MedQA overall_accuracy_pct	11.0%	14%	$1.13	Vals Legal BenchVals MedQA
#20	gpt-5.4-2026-03-05 Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and Vals MedQA overall_accuracy_pct	10.4%	12%	—	Vectara HHEM LeaderboardVals MedQA
#22	gpt-5.1-2025-11-13 Strong on Vals Case Law v2 overall_accuracy_pct and Vals MedScribe overall_accuracy_pct	10.1%	13%	—	Vals Case Law v2Vals MedScribe
#23	gpt-4.1-mini-20250414 Strong on Galileo Agent Leaderboard v2 Avg TSQ and Galileo Agent Leaderboard v2 Avg AC	10.0%	14%	—	Galileo Agent Leaderboard v2Galileo Agent Leaderboard v2
#24	claude-sonnet-4.6 Strong on Vals Tax Eval v2 overall_accuracy_pct and Vals Finance Agent overall_accuracy_pct	10.0%	12%	$6.00	Vals Tax Eval v2Vals Finance Agent
#25	Kimi-K2-Instruct Strong on Galileo Agent Leaderboard v2 Avg TSQ and Galileo Agent Leaderboard v2 Avg AC	9.8%	12%	—	Galileo Agent Leaderboard v2Galileo Agent Leaderboard v2
#26	claude-opus-4-5-20251101 Strong on Vals Mortgage Tax overall_accuracy_pct and Vals MedQA overall_accuracy_pct	9.7%	15%	—	Vals Mortgage TaxVals MedQA
#27	grok-4-1-fast-reasoning Strong on BFCL Multi-turn Official Multi Turn Acc and Vals CorpFin v2 overall_accuracy_pct	9.6%	15%	$0.28	BFCL Multi-turn OfficialVals CorpFin v2
#28	gpt-4.1 Strong on DuckDB NSQL Leaderboard all_execution_accuracy and LanguageBench overall:mean	9.5%	12%	$3.50	DuckDB NSQL LeaderboardLanguageBench
#29	claude-opus-4-6-thinking Strong on Vals SWE-bench overall_accuracy_pct and Vals Mortgage Tax overall_accuracy_pct	9.5%	11%	—	Vals SWE-benchVals Mortgage Tax
#30	claude-opus-4-5-20251101-thinking Strong on Vals MedQA overall_accuracy_pct and Vals Mortgage Tax overall_accuracy_pct	9.2%	11%	—	Vals MedQAVals Mortgage Tax
#31	gemini-3.1-flash-lite-preview Strong on Vals Mortgage Tax overall_accuracy_pct and Vectara HHEM Leaderboard overall_hallucination_error_pct	9.2%	13%	$0.56	Vals Mortgage TaxVectara HHEM Leaderboard
#32	deepseek-r1 Strong on DuckDB NSQL Leaderboard all_execution_accuracy and Aider Polyglot Leaderboard percent_correct_pct	9.1%	15%	$0.27	DuckDB NSQL LeaderboardAider Polyglot Leaderboard
#34	o4-mini Strong on Aider Polyglot Leaderboard percent_correct_pct and Vals MedQA overall_accuracy_pct	8.9%	16%	$1.93	Aider Polyglot LeaderboardVals MedQA
#35	gpt-4o-mini-2024-07-18 Strong on DuckDB NSQL Leaderboard all_execution_accuracy and LLM Trustworthy Leaderboard privacy	8.7%	19%	—	DuckDB NSQL LeaderboardLLM Trustworthy Leaderboard
#36	kimi-k2.5-thinking Strong on Vals CorpFin v2 overall_accuracy_pct and Vals MedQA overall_accuracy_pct	8.7%	12%	—	Vals CorpFin v2Vals MedQA

Compare Models

Select two different models above to compare their evidence side by side.

▶Ranking diagnostics & missing models

Source lift

Ranked

Sources

Quality

Low

Vals Legal Bench

36 rows · 0.5% avg lift

Vals MedQA

35 rows · 0.5% avg lift

Vals Tax Eval v2

35 rows · 0.5% avg lift

Vals LiveCodeBench

35 rows · 0.4% avg lift

Missing frontier models

No obvious gaps right now.

▶Taxonomy & task details

Core tasks

task.summarize_doctask.write_memo_brief

Required modes

mode.long_context

Domains

domain.hr_recruiting

Related in HR

HR policy Q&A

Answer HR policy questions grounded in authoritative text.

Job description drafting

Draft job descriptions that match role requirements and tone.

Interview question bank

Generate structured interview questions and rubrics for a role.

Resume structuring

Extract structured candidate profiles from resumes.