Unit test generation
Generate meaningful unit tests and edge cases.
Provisional leader
gpt-4o-2024-05-13
Current leader based on limited benchmark evidence. Treat this ranking as directional until coverage improves.
24.2%
Best benchmark score
30.8%
Confidence
All ranked models — top 3
Ranked Models
30
Evidence Quality
81%
Evidence Points
13
Top Signal
RepoQA Official Results: overall_average_pass_at_1_pct
All Ranked Models
| Rank | Model | Score |
|---|---|---|
| 🥈 | gpt-4o-2024-05-13 Strong on RepoQA Official Results overall_average_pass_at_1_pct and RepoQA Official Results all_average_pass_at_1_pct | 24.2% |
| 🥉 | gpt-5-2025-08-07 Strong on Aider Polyglot Leaderboard percent_correct_pct and SWE-bench Verified Leaderboard swe_verified_resolved_pct | 23.2% |
| #4 | claude-sonnet-4 Strong on Galileo Agent Leaderboard v2 Avg AC and SWE-bench Verified Leaderboard swe_verified_resolved_pct | 23.2% |
| #7 | gemini-3-pro-preview Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Vals SWE-bench overall_accuracy_pct | 18.6% |
| #8 | gpt-5.2-2025-12-11 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Vals SWE-bench overall_accuracy_pct | 18.1% |
| #10 | gpt-4o-20241120 Strong on Aider Code Editing Leaderboard percent_correct_pct and BigCodeBench Official bigcodebench_complete_pct | 17.5% |
| #11 | gpt-4.1-20250414 Strong on Galileo Agent Leaderboard v2 Avg AC and MMLongBench-Doc Leaderboard acc_score_pct | 17.5% |
| #12 | o3-20250416 Strong on Aider Polyglot Leaderboard percent_correct_pct and SWE-bench Verified Leaderboard swe_verified_resolved_pct | 17.0% |
| #13 | gemini-2.5-pro Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Galileo Agent Leaderboard v2 Avg AC | 16.8% |
| #14 | gemini-3.1-pro-preview Strong on Vals SWE-bench overall_accuracy_pct and Vals LiveCodeBench overall_accuracy_pct | 16.5% |
| #16 | Kimi K2 Thinking Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Sonar Java Quality Leaderboard functional_skill_pct | 15.7% |
| #17 | gpt-5-mini-2025-08-07 Strong on Vals LiveCodeBench overall_accuracy_pct and SWE-bench Verified Leaderboard swe_verified_resolved_pct | 15.7% |
| #20 | Grok-4-0709 Strong on Vals LiveCodeBench overall_accuracy_pct and Galileo Agent Leaderboard v2 Avg AC | 15.4% |
| #21 | qwen-2.5-72b-instruct Strong on Galileo Agent Leaderboard v2 Avg AC and Aider Code Editing Leaderboard percent_correct_pct | 15.0% |
| #22 | claude-opus-4-5-20251101 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Vals Terminal-Bench 2 overall_accuracy_pct | 14.2% |
| #25 | gemini-3-flash-preview Strong on Vals SWE-bench overall_accuracy_pct and Vals LiveCodeBench overall_accuracy_pct | 13.9% |
| #26 | o4-mini Strong on Aider Polyglot Leaderboard percent_correct_pct and Vals LiveCodeBench overall_accuracy_pct | 13.8% |
| #27 | gpt-5.4-2026-03-05 Strong on Vals SWE-bench overall_accuracy_pct and Vals LiveCodeBench overall_accuracy_pct | 13.8% |
| #29 | claude-sonnet-4.6 Strong on Vals SWE-bench overall_accuracy_pct and Vals LiveCodeBench overall_accuracy_pct | 13.5% |
| #30 | claude-opus-4-6-thinking Strong on Vals SWE-bench overall_accuracy_pct and Vals LiveCodeBench overall_accuracy_pct | 13.2% |
| #31 | glm-4.7 Strong on Vals LiveCodeBench overall_accuracy_pct and Vals SWE-bench overall_accuracy_pct | 13.2% |
| #32 | minimax-m2.1 Strong on Vals SWE-bench overall_accuracy_pct and Vals LiveCodeBench overall_accuracy_pct | 13.2% |
| #35 | deepseek-r1 Strong on Aider Polyglot Leaderboard percent_correct_pct and Sonar Java Quality Leaderboard functional_skill_pct | 12.9% |
| #36 | claude-opus-4 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Aider Polyglot Leaderboard percent_correct_pct | 12.9% |
| #37 | gpt-5.1-2025-11-13 Strong on Vals LiveCodeBench overall_accuracy_pct and Vals SWE-bench overall_accuracy_pct | 12.7% |
| #38 | claude-opus-4-5-20251101-thinking Strong on Vals SWE-bench overall_accuracy_pct and Vals LiveCodeBench overall_accuracy_pct | 12.7% |
| #39 | Phi-3-medium-128k-instruct Strong on RepoQA Official Results overall_average_pass_at_1_pct and RepoQA Official Results all_average_pass_at_1_pct | 12.3% |
| #43 | gpt-4o-2024-08-06 Strong on Aider Code Editing Leaderboard percent_correct_pct and Aider Code Editing Leaderboard correct_edit_format_pct | 11.7% |
| #44 | kimi-k2.5-thinking Strong on Vals LiveCodeBench overall_accuracy_pct and Vals SWE-bench overall_accuracy_pct | 11.5% |
| #46 | gpt-4.1 Strong on SWE-bench Verified Leaderboard swe_verified_resolved_pct and Aider Polyglot Leaderboard percent_correct_pct | 11.4% |
Compare Models
▶Ranking diagnostics & missing models
Source lift
Ranked
56
Sources
8
Quality
Low
Vals LiveCodeBench
Vals Legal Bench
Vals Tax Eval v2
Vals MedQA
Missing frontier models
No obvious gaps right now.
▶Taxonomy & task details
Core tasks
Required modes
Domains
Related in Developer
Codebase onboarding brief
Summarize a repository's architecture, modules, and conventions.
Autonomous Coding Agent
End-to-end autonomous software engineering: reading issues, writing code, running tests, submitting PRs.
Code generation
Generate correct, secure code from requirements.
Refactoring assistant
Refactor code safely while preserving behavior and improving clarity.