📊 OpenSRE Bench Explorer
Explore evaluation runs for AI Site Reliability Engineering agents — the artifacts written by the
OpenSRE benchmark framework
(report.json, provenance.json, cases/*.json).
The question these runs exist to answer: does wrapping an agent framework around an LLM actually beat the LLM on its own at diagnosing production incidents — and what does the difference cost in tokens, latency and tool calls.
Run provenance
Per mode / model
Median metric per mode/llm cell. Strata come from the run: all,
seen-shape/unseen-shape, and the held-out generalization split.
Agent lift
opensre+llm minus the model-alone baseline, per model per metric. Positive is better for every
metric except invalid_action_count, where lower is better.
Metric means over the filtered cells
report.json aggregates each stratum by median, which collapses binary metrics to 0 / 0.5 / 1.
These are means — “out of N cases, how often”.
Success rate by fault category
Cost vs. latency per mode / model (means)
Cells
One cell’s tool-call trajectory and final diagnosis — what the agent actually looked at before it answered.