AIOpsLab

Built by Microsoft Research, UIUC, UC Berkeley and IISc · released 12 Jan 2025

A framework from Microsoft Research and university partners that deploys microservice applications on Kubernetes, injects faults, generates load, and exposes telemetry so that AI agents can be tested on cloud operations tasks. The paper ships a suite of 48 problems.

Frontier

59.3%

Accuracy

Flash

12 Jan 2025 · Source: Microsoft Research (benchmark maintainers)

Best accuracy in the paper. Flash is the authors' simplified re-implementation of a workflow agent with hindsight generation; the paper does not state its backbone model in the results table.

Emerging

The best result is at 20–60% of the ceiling, or at 50–100% of human parity.

Dots are recorded results; the line is the best result so far; the red dot is the current frontier. Sources for every point are in the table below.

Embed this card

The image names the institutions behind the numbers and links back to this page.

HTML:

<a href="https://canagentswork.com/benchmarks/aiopslab/"><img src="https://canagentswork.com/og/benchmarks-aiopslab.png" width="600" height="315" alt="AIOpsLab: the best result is 59.3% (Flash, 12 Jan 2025)." loading="lazy"></a>

Markdown:

[![AIOpsLab: the best result is 59.3% (Flash, 12 Jan 2025).](https://canagentswork.com/og/benchmarks-aiopslab.png)](https://canagentswork.com/benchmarks/aiopslab/)

What it measures

Whether an agent answers each operations problem correctly. Problems sit at four levels: detect that something is wrong, localize the faulty service, analyze the root cause, and mitigate the fault in the live cluster. The headline is accuracy across all 48 problems. The framework also records time, steps, and tokens.

Accuracy: Share of the 48 problems that the agent solves correctly across detection, localization, root cause analysis, and mitigation. Higher is better.

Status compares the frontier with a ceiling of 100%.

Facts

Grain: Task level: bounded tasks with a clear spec
Environment: live-system, cli, api-tools
Grading: state-check, automated-tests
Tasks: 48
Reuse: MIT (GitHub repository license). (open-mit)

Limits to keep in mind

Published results cover four agents built on GPT-4-turbo and GPT-3.5-turbo from early 2025. There is no public leaderboard, so newer models are not tracked here. Source
The problem set in the paper is 48 problems from a small number of faults across a few applications. The repository has since added more problems. Source
SREGym, which ports the AIOpsLab problems, reports that strong agents now mitigate more than 80% of these ported problems. Source

Recorded results

A selection that shows the frontier over time. The full leaderboard is at the source.

System	Accuracy	Date	Source
GPT-3.5-w-Shell harness: shell-only baseline	15.3%	12 Jan 2025	Microsoft Research · primary
GPT-4-w-Shell harness: shell-only baseline	49.1%	12 Jan 2025	Microsoft Research · primary
ReAct	55.9%	12 Jan 2025	Microsoft Research · primary
Flash · frontier	59.3%	12 Jan 2025	Microsoft Research · primary

Timeline

12 Jan 2025 — AIOpsLab paper released with 48 cloud operations problems. Source

Where it sits in the atlas

DevOps, SRE, and IT operations

Incident response Terminal operations

Go to the source

Paper arxiv.org
Code github.com

Last checked 23 Sep 2026 against 3 primary sources. See an error? Tell us.

How to cite

Credit the original work first: AIOpsLab by Microsoft Research, UIUC, UC Berkeley, and IISc (https://arxiv.org/abs/2501.06706).

Then, if you used this page:

Can Agents Work. "AIOpsLab: frontier results and sources." https://canagentswork.com/benchmarks/aiopslab/ (accessed 2026-09-24). CC BY 4.0.

@misc{caw-aiopslab,
  title        = {{AIOpsLab: frontier results and sources}},
  author       = {{Can Agents Work}},
  year         = {2026},
  howpublished = {\url{https://canagentswork.com/benchmarks/aiopslab/}},
  note         = {Accessed 2026-09-24. CC BY 4.0}
}