Benchmarks / Terminal operations

Terminal operations

Multi-step work in a shell, for example build, configure, and administer systems.

Part of DevOps, SRE, and IT operations.

• AIOpsLabby Microsoft Research, UIUC, UC Berkeley and IIScTask level · Accuracy

  59.3%Flash · 12 Jan 2025

  Emerging
• SREGymby UIUC and University of TorontoTask level · End-to-end success rate

  72.2%GitHub Copilot (GPT-5.6 Sol, max) · 2026

  Strong
• Terminal-Bench 2.0by Laude Institute and Stanford UniversityTask level · Resolution rate

  84.7%NexAU-AHE + GPT-5.5 · 23 Apr 2026

  Retired