Biomedical

Protocol structuring

Convert protocols/methods text into structured steps and tables.

task.json_schema_fillingtask.timeline_extraction

Evidence quality is currently limited for this use case. Rankings below are useful for exploration, not a strong winner claim.

Provisional leader

gemini-3.1-pro-preview

Current leader based on limited benchmark evidence. Treat this ranking as directional until coverage improves.

22.7%

Best benchmark score

25.8%

Confidence

All ranked models — top 3

🥇

gemini-3.1-pro-preview

22.7%

🥈

gpt-5-2025-08-07

22.5%

🥉

gpt-4.1-20250414

20.8%

Ranked Models

Evidence Quality

81%

Evidence Points

Top Signal

Vals GPQA: overall_accuracy_pct

All Ranked Models

30 of 30 models

Rank	Model	Score	Confidence	Price / 1M	Evidence sources
🥇	gemini-3.1-pro-preview Strong on Vals GPQA overall_accuracy_pct and Vals MedCode overall_accuracy_pct	22.7%	26%	$4.50	Vals GPQAVals MedCode
🥈	gpt-5-2025-08-07 Strong on Vals GPQA overall_accuracy_pct and Vals MedCode overall_accuracy_pct	22.5%	33%	—	Vals GPQAVals MedCode
🥉	gpt-4.1-20250414 Strong on MMLongBench-Doc Leaderboard acc_score_pct and Vectara HHEM Leaderboard science_hallucination_error_pct	20.8%	31%	—	MMLongBench-Doc LeaderboardVectara HHEM Leaderboard
#4	gemini-3-pro-preview Strong on Vals GPQA overall_accuracy_pct and Vals MedCode overall_accuracy_pct	19.7%	25%	$4.50	Vals GPQAVals MedCode
#5	claude-sonnet-4 Strong on Vectara HHEM Leaderboard science_hallucination_error_pct and Vals GPQA overall_accuracy_pct	19.3%	32%	$6.00	Vectara HHEM LeaderboardVals GPQA
#6	gemini-2.5-pro Strong on Vals MedCode overall_accuracy_pct and Vectara HHEM Leaderboard science_hallucination_error_pct	18.4%	29%	$3.44	Vals MedCodeVectara HHEM Leaderboard
#7	gemini-3-flash-preview Strong on Vals GPQA overall_accuracy_pct and Vals MedCode overall_accuracy_pct	17.6%	23%	$1.13	Vals GPQAVals MedCode
#8	gpt-5-mini-2025-08-07 Strong on Vals GPQA overall_accuracy_pct and MathArena Models average_score_pct	17.5%	28%	—	Vals GPQAMathArena Models
#9	gpt-5.2-2025-12-11 Strong on Vals GPQA overall_accuracy_pct and Vals MedCode overall_accuracy_pct	17.3%	22%	—	Vals GPQAVals MedCode
#10	gemini-3.1-flash-lite-preview Strong on Vals GPQA overall_accuracy_pct and Vectara HHEM Leaderboard science_hallucination_error_pct	16.0%	25%	$0.56	Vals GPQAVectara HHEM Leaderboard
#11	gemini-2.5-flash Strong on Vectara HHEM Leaderboard science_hallucination_error_pct and FACTS Benchmark Suite facts_grounding_score_pct	15.8%	26%	$0.17	Vectara HHEM LeaderboardFACTS Benchmark Suite
#12	claude-opus-4-5-20251101 Strong on Vals GPQA overall_accuracy_pct and Vectara HHEM Leaderboard science_hallucination_error_pct	15.6%	23%	—	Vals GPQAVectara HHEM Leaderboard
#13	gpt-5.4-2026-03-05 Strong on Vals GPQA overall_accuracy_pct and Vectara HHEM Leaderboard science_hallucination_error_pct	15.3%	20%	—	Vals GPQAVectara HHEM Leaderboard
#14	Grok-4-0709 Strong on Vals GPQA overall_accuracy_pct and FACTS Benchmark Suite facts_grounding_score_pct	14.9%	23%	—	Vals GPQAFACTS Benchmark Suite
#15	o3-20250416 Strong on Vals GPQA overall_accuracy_pct and Vals MedCode overall_accuracy_pct	14.8%	23%	$3.50	Vals GPQAVals MedCode
#16	gpt-5.1-2025-11-13 Strong on Vals GPQA overall_accuracy_pct and Vals MedCode overall_accuracy_pct	13.3%	17%	—	Vals GPQAVals MedCode
#17	claude-sonnet-4.6 Strong on Vals GPQA overall_accuracy_pct and Vectara HHEM Leaderboard science_hallucination_error_pct	13.2%	17%	$6.00	Vals GPQAVectara HHEM Leaderboard
#18	kimi-k2.5-thinking Strong on Vals GPQA overall_accuracy_pct and MathArena Models average_score_pct	13.0%	20%	—	Vals GPQAMathArena Models
#19	claude-opus-4-5-20251101-thinking Strong on Vals GPQA overall_accuracy_pct and Vals MedCode overall_accuracy_pct	12.8%	16%	—	Vals GPQAVals MedCode
#20	claude-opus-4-1-20250805 Strong on Vectara HHEM Leaderboard science_hallucination_error_pct and Vals GPQA overall_accuracy_pct	12.3%	22%	—	Vectara HHEM LeaderboardVals GPQA
#21	claude-opus-4-6-thinking Strong on Vals GPQA overall_accuracy_pct and Vals MedCode overall_accuracy_pct	11.9%	14%	—	Vals GPQAVals MedCode
#22	grok-4-fast-reasoning Strong on Vals GPQA overall_accuracy_pct and Vals MedCode overall_accuracy_pct	11.9%	23%	$0.28	Vals GPQAVals MedCode
#23	gpt-4o-20241120 Strong on MMLongBench-Doc Leaderboard acc_score_pct and DuckDB NSQL Leaderboard all_execution_accuracy	11.9%	20%	—	MMLongBench-Doc LeaderboardDuckDB NSQL Leaderboard
#24	grok-4-1-fast-reasoning Strong on Vals GPQA overall_accuracy_pct and BFCL Multi-turn Official Multi Turn Acc	11.5%	22%	$0.28	Vals GPQABFCL Multi-turn Official
#26	gpt-4o Strong on MEGA-Bench overall_score and JSONSchemaBench Leaderboard medium_schema_compliance_pct	11.1%	16%	$0.26	MEGA-BenchJSONSchemaBench Leaderboard
#28	claude-sonnet-4-5-20250929-thinking Strong on Vals GPQA overall_accuracy_pct and Vals MedCode overall_accuracy_pct	10.9%	16%	—	Vals GPQAVals MedCode
#29	o4-mini Strong on Vals GPQA overall_accuracy_pct and MathArena Models average_score_pct	10.8%	22%	$1.93	Vals GPQAMathArena Models
#34	grok-3 Strong on Vectara HHEM Leaderboard science_hallucination_error_pct and Vals GPQA overall_accuracy_pct	10.2%	14%	$6.00	Vectara HHEM LeaderboardVals GPQA
#37	claude-opus-4 Strong on Vals GPQA overall_accuracy_pct and Vectara HHEM Leaderboard science_hallucination_error_pct	9.9%	16%	$10.00	Vals GPQAVectara HHEM Leaderboard
#38	claude-opus-4-6 Strong on MathArena Models average_score_pct and Vals MedCode overall_accuracy_pct	9.9%	14%	$10.00	MathArena ModelsVals MedCode

Compare Models

Select two different models above to compare their evidence side by side.

▶Ranking diagnostics & missing models

Source lift

Ranked

Sources

Quality

Low

Vals GPQA

44 rows · 2.0% avg lift

Vals Legal Bench

42 rows · 0.4% avg lift

Vals LiveCodeBench

41 rows · 0.3% avg lift

Vals Tax Eval v2

40 rows · 0.4% avg lift

Missing frontier models

No obvious gaps right now.

▶Taxonomy & task details

Core tasks

task.json_schema_fillingtask.timeline_extraction

Required modes

mode.json_schema

Domains

domain.biomed_literaturedomain.chemistry_materials

Related in Biomedical

Cross-paper contradiction analysis

Identify contradictions and uncertainty across papers with citations.

Literature synthesis with citations

Synthesize papers and guidelines with citations and uncertainty.