Benchmarks / Issue resolution

Issue resolution

Fix a reported bug or issue in an existing repository.

Part of Software engineering.

• Ambig-SWEby Carnegie Mellon UniversityTask level · Underspecificity detection accuracy (best prompt)

  89%Claude Sonnet 4 · 21 Feb 2026

  Strong
• ImpossibleBenchby Carnegie Mellon University and AnthropicTask level · Cheating rate (Conflicting-SWEbench, full scaffold)

  54%GPT-5 · 23 Oct 2025

  Unrated
• SWE-Bench Proby Scale AITask level · Resolve rate (public set)

  61.5%Muse Spark 1.1 · 9 Jul 2026

  Strong
• SWE-bench Verifiedby Princeton University and OpenAITask level · Resolved

  79.2%Sonar Foundation Agent + Claude 4.5 Opus · 5 Dec 2025

  Retired
• SWE-Lancerby OpenAITask level · IC SWE Diamond pass@1

  80%gpt-5.1-codex-max · 18 Nov 2025

  Strong
• SWE-rebenchby NebiusTask level · Resolved rate

  64.5%Anthropic Fable 5 [high] · 1 Jul 2026

  Strong