Security

Vulnerability-oriented code review

Review code for security vulnerabilities and propose mitigations.

task.code_reviewtask.risk_assessment

Evidence quality is currently limited for this use case. Rankings below are useful for exploration, not a strong winner claim.

Provisional leader

gemini-2.5-pro

Current leader based on limited benchmark evidence. Treat this ranking as directional until coverage improves.

28.6%

Best benchmark score

40.7%

Confidence

All ranked models — top 3

🥇

gemini-2.5-pro

28.6%

🥈

o3-20250416

26.2%

🥉

claude-sonnet-4

23.7%

Ranked Models

Evidence Quality

81%

Evidence Points

Top Signal

SWE-bench Verified Leaderboard: swe_verified_resolved_pct

All Ranked Models

30 of 30 models

Rank	Model	Score	Confidence	Price / 1M	Evidence sources
🥇	gemini-2.5-pro Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and VADER Leaderboard mean_score_pct	28.6%	41%	$3.44	SWE-bench Verified LeaderboardVADER Leaderboard
🥈	o3-20250416 Strong on VADER Leaderboard mean_score_pct and BaxBench Leaderboard average_secure_pass_1_pct	26.2%	34%	$3.50	VADER LeaderboardBaxBench Leaderboard
🥉	claude-sonnet-4 Strong on BaxBench Leaderboard average_secure_pass_1_pct and SWE-bench Verified Leaderboard swe_verified_resolved_pct	23.7%	36%	$6.00	BaxBench LeaderboardSWE-bench Verified Leaderboard
#4	gpt-5-2025-08-07 Strong on BaxBench Leaderboard average_secure_pass_1_pct and SWE-bench Verified Leaderboard swe_verified_resolved_pct	20.3%	26%	—	BaxBench LeaderboardSWE-bench Verified Leaderboard
#5	gpt-4o-2024-05-13 Strong on LLM Trustworthy Leaderboard privacy and RepoQA Official Results overall_average_pass_at_1_pct	17.1%	29%	—	LLM Trustworthy LeaderboardRepoQA Official Results
#6	Kimi K2 Thinking Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Sonar Java Quality Leaderboard functional_skill_pct	15.4%	21%	$1.07	SWE-bench Verified LeaderboardSonar Java Quality Leaderboard
#8	gpt-4.1 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and BaxBench Leaderboard average_secure_pass_1_pct	15.0%	30%	$3.50	SWE-bench Verified LeaderboardBaxBench Leaderboard
#9	Llama-2-7b-chat-hf Strong on LLM Trustworthy Leaderboard fairness and LLM Trustworthy Leaderboard privacy	14.3%	18%	—	LLM Trustworthy LeaderboardLLM Trustworthy Leaderboard
#10	gemini-3-pro-preview Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and BFCL Multi-turn Official Multi Turn Acc	14.2%	18%	$4.50	SWE-bench Verified LeaderboardBFCL Multi-turn Official
#11	gpt-4.1-20250414 Strong on MMLongBench-Doc Leaderboard acc_score_pct and Galileo Agent Leaderboard v2 Avg AC	14.0%	22%	—	MMLongBench-Doc LeaderboardGalileo Agent Leaderboard v2
#12	gpt-5.2-2025-12-11 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and BFCL Multi-turn Official Multi Turn Acc	13.9%	17%	—	SWE-bench Verified LeaderboardBFCL Multi-turn Official
#13	deepseek-r1 Strong on BaxBench Leaderboard average_secure_pass_1_pct and Sonar Java Quality Leaderboard functional_skill_pct	13.6%	27%	$0.27	BaxBench LeaderboardSonar Java Quality Leaderboard
#14	gpt-5-mini-2025-08-07 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Vals MedQA overall_accuracy_pct	13.0%	19%	—	SWE-bench Verified LeaderboardVals MedQA
#16	gpt-4o-mini-2024-07-18 Strong on LLM Trustworthy Leaderboard privacy and LLM Trustworthy Leaderboard adv	12.4%	23%	—	LLM Trustworthy LeaderboardLLM Trustworthy Leaderboard
#17	claude-opus-4-5-20251101 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Vals Mortgage Tax overall_accuracy_pct	12.3%	16%	—	SWE-bench Verified LeaderboardVals Mortgage Tax
#18	Meta-Llama-3-8B-Instruct Strong on LLM Trustworthy Leaderboard adv and LLM Trustworthy Leaderboard privacy	12.3%	19%	—	LLM Trustworthy LeaderboardLLM Trustworthy Leaderboard
#19	gemma-2b-it Strong on LLM Trustworthy Leaderboard fairness and LLM Trustworthy Leaderboard privacy	11.5%	18%	—	LLM Trustworthy LeaderboardLLM Trustworthy Leaderboard
#20	gemma-7b-it Strong on LLM Trustworthy Leaderboard fairness and LLM Trustworthy Leaderboard privacy	11.5%	19%	—	LLM Trustworthy LeaderboardLLM Trustworthy Leaderboard
#22	Grok-4-0709 Strong on Galileo Agent Leaderboard v2 Avg TSQ and Galileo Agent Leaderboard v2 Avg AC	11.1%	17%	—	Galileo Agent Leaderboard v2Galileo Agent Leaderboard v2
#23	glm-4.7 Strong on Sonar Java Quality Leaderboard functional_skill_pct and Sonar Java Quality Leaderboard issue_density_error_per_kloc	11.1%	17%	—	Sonar Java Quality LeaderboardSonar Java Quality Leaderboard
#24	gemini-3.1-pro-preview Strong on Vals Mortgage Tax overall_accuracy_pct and FACTS Benchmark Suite average_score_pct	11.0%	12%	$4.50	Vals Mortgage TaxFACTS Benchmark Suite
#26	falcon-7b-instruct Strong on LLM Trustworthy Leaderboard fairness and LLM Trustworthy Leaderboard privacy	10.7%	17%	—	LLM Trustworthy LeaderboardLLM Trustworthy Leaderboard
#27	minimax-m2.1 Strong on Sonar Java Quality Leaderboard functional_skill_pct and Sonar Java Quality Leaderboard bug_density_error_per_kloc	10.5%	17%	$0.53	Sonar Java Quality LeaderboardSonar Java Quality Leaderboard
#31	GLM-5 Strong on Sonar Java Quality Leaderboard issue_density_error_per_kloc and Sonar Java Quality Leaderboard functional_skill_pct	9.9%	11%	—	Sonar Java Quality LeaderboardSonar Java Quality Leaderboard
#32	zephyr-7b-beta Strong on LLM Trustworthy Leaderboard fairness and LLM Trustworthy Leaderboard privacy	9.9%	18%	—	LLM Trustworthy LeaderboardLLM Trustworthy Leaderboard
#34	o4-mini Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Aider Polyglot Leaderboard percent_correct_pct	9.7%	17%	$1.93	SWE-bench Verified LeaderboardAider Polyglot Leaderboard
#35	gpt-4o Strong on Sonar Java Quality Leaderboard functional_skill_pct and BaxBench Leaderboard average_secure_pass_1_pct	9.7%	26%	$0.26	Sonar Java Quality LeaderboardBaxBench Leaderboard
#36	gemini-3-flash-preview Strong on Vals Legal Bench overall_accuracy_pct and Vals MedQA overall_accuracy_pct	8.9%	11%	$1.13	Vals Legal BenchVals MedQA
#37	claude-opus-4 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Aider Polyglot Leaderboard percent_correct_pct	8.9%	11%	$10.00	SWE-bench Verified LeaderboardAider Polyglot Leaderboard
#39	Claude-3.5-Sonnet Strong on BaxBench Leaderboard average_secure_pass_1_pct and SWE-bench Verified Leaderboard swe_verified_resolved_pct	8.6%	16%	$6.00	BaxBench LeaderboardSWE-bench Verified Leaderboard

Compare Models

Select two different models above to compare their evidence side by side.

▶Ranking diagnostics & missing models

Source lift

Ranked

Sources

Quality

Low

Vals Legal Bench

25 rows · 0.4% avg lift

Vals GPQA

23 rows · 0.3% avg lift

Vals MedQA

22 rows · 0.4% avg lift

Vals Tax Eval v2

22 rows · 0.4% avg lift

Missing frontier models

claude-sonnet-4.6

Thin evidence after weighting

Rank #11

20.0%

▶Taxonomy & task details

Core tasks

task.code_reviewtask.risk_assessment

Required modes

none

Domains

domain.cybersecurity_defense

Related in Security

Security incident triage

Triage security incidents from alerts/logs into impact and next steps.

Malware analysis report (defensive)

Explain suspicious code and produce a defensive analysis report.

Threat Intelligence Analysis

Analyzing threat reports, CVEs, and security advisories to produce structured risk assessments.