Supply Chain

Tail spend categorization

Categorize tail spend purchases into taxonomy buckets for sourcing.

task.topic_taggingtask.intent_classification

Evidence quality is currently limited for this use case. Rankings below are useful for exploration, not a strong winner claim.

Provisional leader

claude-sonnet-4

Current leader based on limited benchmark evidence. Treat this ranking as directional until coverage improves.

20.3%

Best benchmark score

26.1%

Confidence

All ranked models — top 3

🥇

claude-sonnet-4

20.3%

🥈

gpt-4.1-20250414

19.8%

🥉

gemini-2.5-flash

17.6%

Ranked Models

Evidence Quality

79%

Evidence Points

Top Signal

Galileo Agent Leaderboard v2: Avg TSQ

All Ranked Models

30 of 30 models

Rank	Model	Score	Confidence	Price / 1M	Evidence sources
🥇	claude-sonnet-4 Strong on Galileo Agent Leaderboard v2 Avg TSQ and LanguageBench Grammar/Clarity Official (Split) grammar_clarity_score_pct	20.3%	26%	$6.00	Galileo Agent Leaderboard v2LanguageBench Grammar/Clarity Official (Split)
🥈	gpt-4.1-20250414 Strong on OpenVLM OCRBench Official ocrbench_score_pct and Galileo Agent Leaderboard v2 Avg TSQ	19.8%	30%	—	OpenVLM OCRBench OfficialGalileo Agent Leaderboard v2
🥉	gemini-2.5-flash Strong on Galileo Agent Leaderboard v2 Avg TSQ and LanguageBench Grammar/Clarity Official (Split) grammar_clarity_score_pct	17.6%	22%	$0.17	Galileo Agent Leaderboard v2LanguageBench Grammar/Clarity Official (Split)
#4	gemini-2.5-pro Strong on Galileo Agent Leaderboard v2 Avg TSQ and OpenVLM OCRBench Official ocrbench_score_pct	17.4%	32%	$3.44	Galileo Agent Leaderboard v2OpenVLM OCRBench Official
#5	Claude-3.5-Sonnet Strong on OpenVLM OCRBench Official ocrbench_score_pct and LanguageBench Grammar/Clarity Official (Split) grammar_clarity_score_pct	17.3%	20%	$6.00	OpenVLM OCRBench OfficialLanguageBench Grammar/Clarity Official (Split)
#6	gpt-5-2025-08-07 Strong on OpenVLM OCRBench Official ocrbench_score_pct and SciArena Leaderboard rating_elo	16.9%	23%	—	OpenVLM OCRBench OfficialSciArena Leaderboard
#7	gpt-4o Strong on MEGA-Bench app:Planning and OpenVLM OCRBench Official ocrbench_score_pct	16.6%	20%	$0.26	MEGA-BenchOpenVLM OCRBench Official
#9	gpt-4.1-mini-20250414 Strong on OpenVLM OCRBench Official ocrbench_score_pct and Galileo Agent Leaderboard v2 Avg TSQ	16.5%	25%	—	OpenVLM OCRBench OfficialGalileo Agent Leaderboard v2
#11	gpt-5-mini-2025-08-07 Strong on OpenVLM OCRBench Official ocrbench_score_pct and SciArena Leaderboard rating_elo	14.5%	22%	—	OpenVLM OCRBench OfficialSciArena Leaderboard
#12	gpt-4.1 Strong on DuckDB NSQL Leaderboard all_execution_accuracy and LanguageBench Grammar/Clarity Official (Split) grammar_clarity_score_pct	13.7%	17%	$3.50	DuckDB NSQL LeaderboardLanguageBench Grammar/Clarity Official (Split)
#18	qwen-2.5-72b-instruct Strong on Galileo Agent Leaderboard v2 Avg TSQ and DuckDB NSQL Leaderboard all_execution_accuracy	12.7%	19%	—	Galileo Agent Leaderboard v2DuckDB NSQL Leaderboard
#20	Grok-4-0709 Strong on Galileo Agent Leaderboard v2 Avg TSQ and Galileo Agent Leaderboard v2 Avg AC	12.3%	17%	—	Galileo Agent Leaderboard v2Galileo Agent Leaderboard v2
#21	gemini-2.0-flash-001 Strong on LanguageBench Grammar/Clarity Official (Split) grammar_clarity_score_pct and LanguageBench overall:mean	11.7%	16%	—	LanguageBench Grammar/Clarity Official (Split)LanguageBench
#25	deepseek-r1 Strong on DuckDB NSQL Leaderboard all_execution_accuracy and ContractEval Leaderboard contract_adherence_csr_pct	11.4%	22%	$0.27	DuckDB NSQL LeaderboardContractEval Leaderboard
#27	gemini-3.1-pro-preview Strong on Vals Mortgage Tax overall_accuracy_pct and Vals GPQA overall_accuracy_pct	11.2%	12%	$4.50	Vals Mortgage TaxVals GPQA
#28	o3-20250416 Strong on SciArena Leaderboard rating_elo and Aider Polyglot Leaderboard percent_correct_pct	11.1%	14%	$3.50	SciArena LeaderboardAider Polyglot Leaderboard
#37	gemini-3-pro-preview Strong on SciArena Leaderboard rating_elo and Vals Mortgage Tax overall_accuracy_pct	10.2%	13%	$4.50	SciArena LeaderboardVals Mortgage Tax
#50	Qwen3-Embedding-4B Strong on MTEB Retrieval and Rerank (Official) retrieval_score_pct and MTEB STS & Summarization Proxy Official sts_score_pct	9.3%	11%	—	MTEB Retrieval and Rerank (Official)MTEB STS & Summarization Proxy Official
#54	claude-sonnet-4.6 Strong on HalluHard Leaderboard overall_hallucination_error_pct and Vals Tax Eval v2 overall_accuracy_pct	9.2%	12%	$6.00	HalluHard LeaderboardVals Tax Eval v2
#58	gemini-3-flash-preview Strong on Vals Legal Bench overall_accuracy_pct and Vals MedQA overall_accuracy_pct	9.1%	12%	$1.13	Vals Legal BenchVals MedQA
#61	gpt-5.2-2025-12-11 Strong on FACTS Benchmark Suite facts_grounding_score_pct and Vals Tax Eval v2 overall_accuracy_pct	9.0%	10%	—	FACTS Benchmark SuiteVals Tax Eval v2
#77	gpt-5.4-2026-03-05 Strong on Vectara HHEM Leaderboard overall_hallucination_error_pct and Vals MedQA overall_accuracy_pct	8.6%	10%	—	Vectara HHEM LeaderboardVals MedQA
#82	gpt-5.1-2025-11-13 Strong on Vals Case Law v2 overall_accuracy_pct and Vals MedScribe overall_accuracy_pct	8.3%	10%	—	Vals Case Law v2Vals MedScribe
#95	o4-mini Strong on SciArena Leaderboard rating_elo and Aider Polyglot Leaderboard percent_correct_pct	8.0%	13%	$1.93	SciArena LeaderboardAider Polyglot Leaderboard
#104	GPT-4.1-nano-2025-04-14 Strong on OpenVLM OCRBench Official ocrbench_score_pct and OpenVLM MTVQA Official mtvqa_score_pct	7.8%	16%	—	OpenVLM OCRBench OfficialOpenVLM MTVQA Official
#106	Llama-3.3-70B-Instruct Strong on DuckDB NSQL Leaderboard all_execution_accuracy and LanguageBench Grammar/Clarity Official (Split) grammar_clarity_score_pct	7.8%	10%	—	DuckDB NSQL LeaderboardLanguageBench Grammar/Clarity Official (Split)
#121	gemini-3.1-flash-lite-preview Strong on Vals Mortgage Tax overall_accuracy_pct and Vectara HHEM Leaderboard overall_hallucination_error_pct	7.6%	11%	$0.56	Vals Mortgage TaxVectara HHEM Leaderboard
#122	Qwen-VL-Chat Strong on OpenVLM OCRVQA Education & Teaching Official ocrvqa_education_teaching_score_pct and OpenVLM TextVQA Official textvqa_score_pct	7.5%	14%	—	OpenVLM OCRVQA Education & Teaching OfficialOpenVLM TextVQA Official
#124	kimi-k2.5-thinking Strong on Vals CorpFin v2 overall_accuracy_pct and Vals MedQA overall_accuracy_pct	7.5%	12%	—	Vals CorpFin v2Vals MedQA
#152	grok-4-fast-reasoning Strong on Vals CorpFin v2 overall_accuracy_pct and Vals Tax Eval v2 overall_accuracy_pct	6.9%	12%	$0.28	Vals CorpFin v2Vals Tax Eval v2

Compare Models

Select two different models above to compare their evidence side by side.

▶Ranking diagnostics & missing models

Source lift

Ranked

Sources

Quality

Low

Vals MedQA

23 rows · 0.5% avg lift

Vals Legal Bench

23 rows · 0.5% avg lift

Vals Mortgage Tax

22 rows · 0.5% avg lift

Vals Tax Eval v2

22 rows · 0.5% avg lift

Missing frontier models

claude-opus-4-5-20251101

Thin evidence after weighting

Rank #10

18.6%

grok-4-1-fast-non-reasoning

Thin evidence after weighting

Rank #15

14.9%

▶Taxonomy & task details

Core tasks

task.topic_taggingtask.intent_classification

Required modes

mode.json_schema

Domains

domain.procurement_vendor_mgmt

Related in Supply Chain

Disruption monitoring brief

Summarize disruptions into risk, options, and recommendations.

Supplier risk monitoring

Track supplier risk signals from multi-source text and summarize actions.

Vendor contract summary (procurement)

Summarize vendor contracts into key terms, risks, and deviations.

Route alternatives planning

Propose alternative routes and tradeoffs under disruptions.