Developer

Agentic bug fixing

Agentic loop that reproduces, fixes, and validates with tests.

task.agentic_multi_step_completiontask.debug_propose_fix

Evidence quality is currently limited for this use case. Rankings below are useful for exploration, not a strong winner claim.

Provisional leader

claude-opus-4-6

Current leader based on limited benchmark evidence. Treat this ranking as directional until coverage improves.

24.5%

Best benchmark score

27.4%

Confidence

All ranked models — top 3

🥇

claude-opus-4-6

24.5%

🥈

gpt-5-2025-08-07

20.9%

🥉

claude-sonnet-4

19.4%

Ranked Models

Evidence Quality

81%

Evidence Points

Top Signal

OpenHands Issue Resolution: issue_resolution_score_pct

All Ranked Models

30 of 30 models

Rank	Model	Score	Confidence	Price / 1M	Evidence sources
🥇	claude-opus-4-6 Strong on OpenHands Issue Resolution issue_resolution_score_pct and SWE-bench Verified Leaderboard swe_verified_resolved_pct	24.5%	27%	$10.00	OpenHands Issue ResolutionSWE-bench Verified Leaderboard
🥉	gpt-5-2025-08-07 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and GAIA Results Public score	20.9%	25%	—	SWE-bench Verified LeaderboardGAIA Results Public
#6	claude-sonnet-4 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Galileo Agent Leaderboard v2 Avg AC	19.4%	27%	$6.00	SWE-bench Verified LeaderboardGalileo Agent Leaderboard v2
#8	GLM-5 Strong on OpenHands Issue Resolution issue_resolution_score_pct and Sonar Java Quality Leaderboard functional_skill_pct	18.0%	29%	—	OpenHands Issue ResolutionSonar Java Quality Leaderboard
#9	claude-sonnet-4.6 Strong on OpenHands Issue Resolution issue_resolution_score_pct and OpenHands Index issue_resolution_score_pct	17.1%	30%	$6.00	OpenHands Issue ResolutionOpenHands Index
#11	Kimi K2 Thinking Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Sonar Java Quality Leaderboard functional_skill_pct	15.8%	40%	$1.07	SWE-bench Verified LeaderboardSonar Java Quality Leaderboard
#15	gemini-3-pro-preview Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and AppBench Leaderboard percentile_score_pct	13.6%	17%	$4.50	SWE-bench Verified LeaderboardAppBench Leaderboard
#16	kimi-k2.5-thinking Strong on Vals LiveCodeBench overall_accuracy_pct and Vals SWE-bench overall_accuracy_pct	13.1%	30%	—	Vals LiveCodeBenchVals SWE-bench
#17	gemini-2.5-pro Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Galileo Agent Leaderboard v2 Avg AC	12.7%	18%	$3.44	SWE-bench Verified LeaderboardGalileo Agent Leaderboard v2
#18	gpt-4.1-20250414 Strong on Galileo Agent Leaderboard v2 Avg AC and SWE-bench Verified Leaderboard swe_verified_resolved_pct	12.5%	19%	—	Galileo Agent Leaderboard v2SWE-bench Verified Leaderboard
#19	gpt-4o-2024-05-13 Strong on RepoQA Official Results overall_average_pass_at_1_pct and SWE-bench Verified Leaderboard swe_verified_resolved_pct	12.5%	17%	—	RepoQA Official ResultsSWE-bench Verified Leaderboard
#20	minimax-m2.1 Strong on Sonar Java Quality Leaderboard functional_skill_pct and Vals SWE-bench overall_accuracy_pct	12.1%	36%	$0.53	Sonar Java Quality LeaderboardVals SWE-bench
#22	gpt-5.2-2025-12-11 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Vals SWE-bench overall_accuracy_pct	11.6%	14%	—	SWE-bench Verified LeaderboardVals SWE-bench
#23	o3-20250416 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Aider Polyglot Leaderboard percent_correct_pct	11.0%	15%	$3.50	SWE-bench Verified LeaderboardAider Polyglot Leaderboard
#25	gpt-4o Strong on τ-bench Airline (Official README) tau_airline_pass1_pct and Sonar Java Quality Leaderboard functional_skill_pct	10.0%	20%	$0.26	τ-bench Airline (Official README)Sonar Java Quality Leaderboard
#26	gpt-4o-20241120 Strong on Aider Code Editing Leaderboard percent_correct_pct and BigCodeBench Official bigcodebench_complete_pct	10.0%	18%	—	Aider Code Editing LeaderboardBigCodeBench Official
#27	Grok-4-0709 Strong on Galileo Agent Leaderboard v2 Avg AC and Galileo Agent Leaderboard v2 Avg TSQ	10.0%	15%	—	Galileo Agent Leaderboard v2Galileo Agent Leaderboard v2
#28	gpt-5-mini-2025-08-07 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Vals LiveCodeBench overall_accuracy_pct	9.9%	15%	—	SWE-bench Verified LeaderboardVals LiveCodeBench
#29	qwen-2.5-72b-instruct Strong on Galileo Agent Leaderboard v2 Avg AC and Aider Code Editing Leaderboard percent_correct_pct	9.8%	14%	—	Galileo Agent Leaderboard v2Aider Code Editing Leaderboard
#30	glm-4.7 Strong on Sonar Java Quality Leaderboard functional_skill_pct and Vals LiveCodeBench overall_accuracy_pct	9.8%	14%	—	Sonar Java Quality LeaderboardVals LiveCodeBench
#31	claude-opus-4-5-20251101 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Vals Terminal-Bench 2 overall_accuracy_pct	9.7%	12%	—	SWE-bench Verified LeaderboardVals Terminal-Bench 2
#32	deepseek-r1 Strong on Sonar Java Quality Leaderboard functional_skill_pct and Aider Polyglot Leaderboard percent_correct_pct	9.6%	14%	$0.27	Sonar Java Quality LeaderboardAider Polyglot Leaderboard
#36	claude-opus-4 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Aider Polyglot Leaderboard percent_correct_pct	8.7%	11%	$10.00	SWE-bench Verified LeaderboardAider Polyglot Leaderboard
#39	o4-mini Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Aider Polyglot Leaderboard percent_correct_pct	8.6%	14%	$1.93	SWE-bench Verified LeaderboardAider Polyglot Leaderboard
#41	GLM-4.7 Strong on OpenHands Issue Resolution issue_resolution_score_pct and OpenHands Index issue_resolution_score_pct	8.0%	21%	—	OpenHands Issue ResolutionOpenHands Index
#43	gpt-4.1-mini-20250414 Strong on Galileo Agent Leaderboard v2 Avg AC and SWE-bench Verified Leaderboard swe_verified_resolved_pct	7.9%	13%	—	Galileo Agent Leaderboard v2SWE-bench Verified Leaderboard
#44	gpt-4o-2024-08-06 Strong on Aider Code Editing Leaderboard percent_correct_pct and GAIA Results Public score	7.8%	15%	—	Aider Code Editing LeaderboardGAIA Results Public
#45	Kimi-K2-Instruct Strong on Galileo Agent Leaderboard v2 Avg AC and SWE-bench Verified Leaderboard swe_verified_resolved_pct	7.8%	11%	—	Galileo Agent Leaderboard v2SWE-bench Verified Leaderboard
#50	gemini-2.5-flash Strong on Galileo Agent Leaderboard v2 Avg AC and Galileo Agent Leaderboard v2 Avg TSQ	6.1%	10%	$0.17	Galileo Agent Leaderboard v2Galileo Agent Leaderboard v2
#51	gpt-4o-mini-2024-07-18 Strong on BigCodeBench Official bigcodebench_complete_pct and BigCodeBench Official bigcodebench_instruct_pct	5.3%	14%	—	BigCodeBench OfficialBigCodeBench Official

Compare Models

Select two different models above to compare their evidence side by side.

▶Ranking diagnostics & missing models

Source lift

Ranked

Sources

Quality

Low

Vals LiveCodeBench

19 rows · 0.9% avg lift

SWE-bench Verified Leaderboard

18 rows · 2.1% avg lift

Vals MedQA

14 rows · 0.2% avg lift

Vals Terminal-Bench 2

13 rows · 0.6% avg lift

Missing frontier models

grok-4-1-fast-reasoning

Thin evidence after weighting

Rank #12

19.5%

grok-4-1-fast-non-reasoning

Thin evidence after weighting

Rank #15

14.9%

▶Taxonomy & task details

Core tasks

task.agentic_multi_step_completiontask.debug_propose_fix

Required modes

mode.tool_calling

Domains

domain.software_engineering

Related in Developer

Codebase onboarding brief

Summarize a repository's architecture, modules, and conventions.

Autonomous Coding Agent

End-to-end autonomous software engineering: reading issues, writing code, running tests, submitting PRs.

Code generation

Generate correct, secure code from requirements.

Refactoring assistant

Refactor code safely while preserving behavior and improving clarity.