How to Check an Agent's Diagnosis Before It Touches Production

TL;DR

When teams get ready to add agents to their on-call workflows, the hard part is deciding when to let those agents act on production versus just advise. A root cause claim only earns that trust if you can verify what the agent ruled out. Causely's causal model now exposes that reasoning directly to agents: which alternative diagnoses were considered for a symptom, what evidence ruled out those alternatives, and how far the failure would have spread if left unaddressed.

Acting vs. Advising

A recent r/kubernetes thread separated a remediation agent into two parts: an LLM proposes actions such as scaling, rolling back, cordoning, or draining, and a separate deterministic layer vetoes anything unsafe against the live cluster state before it executes. The design assumes a proposal and the decision to act on it are two different things.

The top reply doesn't argue with the architecture. It points to the actual gap: confidence in the cause and the safety to act are two different things, and they often get conflated. The suggested fix is to force the agent to explain why this diagnosis, specifically, before it hands over a proposed action. That way a human or a policy engine has something to review, not just a recommendation to accept or reject. Another engineer in the same thread pushes further: a veto needs memory. Blast-radius budgets per namespace, cooldowns, and a rule to halt and page a human if the expected effect doesn't show up. And the veto must check real cluster state itself, not trust what the agent reports about that state.

Platform teams everywhere are running into the same challenge: when is an agent's diagnostic capability good enough to let it act in production?

Why isn't "the agent is confident" the same as "the agent is right"?

An agent proposing an action and an agent explaining its diagnosis are two different things. Only one of them is checkable before anyone touches production. A confidence score is an assertion. A list of possible explanations for the symptom, with the causal chain behind each, is something a reviewer — human or automated — can inspect.

That's the same problem the thread's second engineer described: a veto that trusts the agent's self-report is a second opinion, not a check. A veto that can see what the model considered and rejected, and how large the blast radius of the winning diagnosis is, has something real to validate with.

What does it mean to ask an agent "why this and not that"?

When it comes to understanding cause and effect, most people familiar with the domain don't usually ask, "why did X happen?" They tend to ask "why X, rather than Y," a finding from explainable-AI research on contrastive explanation: an explanation is only satisfying relative to some alternative that didn't occur, even when that alternative is never stated out loud. A 2025 formalization of the problem frames it directly as "why P but not Q" and treats computing the difference between the two as the actual explanation task, not as an afterthought.

Causely's MCP server now exposes two tools designed specifically for this question. get_potential_diagnoses tells you what could explain a given symptom, not just the one the causal model selected, and shows the causal chain behind each. get_signal_potential_diagnoses runs the same question in the other direction: given an observed signal, what could explain it, and what's the causal chain connecting each candidate to that signal? Both draw on the same causal model that produces the primary diagnosis: the same reasoning, made visible instead of collapsed into a single answer.

Here is an example

Take a familiar incident: elevated latency and timeouts on a checkout service, several layers downstream from a shared database connection pool. An LLM-only agent traces this from the symptom alone. It queries telemetry, traces back from the checkout service, and finds a CPU spike on an adjacent service along the way. It diagnoses the CPU spike as the cause and restarts that service. The dashboard improves. The agent treats the incident as resolved.

The real cause, connection exhaustion on the shared database connection pool upstream, provides no direct signal at that layer, so the agent never sees it. The restart bought time by clearing the CPU spike, not by fixing the underlying problem. The incident resurfaces when the connection limit is reached again, and the agent repeats this loop.

Calling get_signal_potential_diagnoses on the original symptom surfaces the connection-exhaustion diagnosis alongside the CPU-spike alternative, each with the causal chain showing what it explains and what it fails to account for. The CPU spike doesn't explain the downstream timeout pattern; the connection exhaustion does. That comparison is what makes the diagnosis reviewable before a remediation agent acts on it, not just the final answer.

How far would this failure have spread if it wasn't caught?

A diagnosis with a small, well-understood blast radius and a diagnosis with a large, uncertain one are different risk decisions, even if a model is equally confident in both. rank_entities and get_diagnosis_observable_signals surface which entities the failure would affect and which signals to expect downstream before any action is taken.

This is the concrete version of the veto idea from the above Kubernetes thread: a blast-radius budget is only useful if something can compute the blast radius first. Ranking entities by dependency exposure is what gives a policy engine, or a human running the on-call rotation, a number to set that budget against instead of a guess.

Where can causal explanations go further?

Today's tools answer "why this and not that" and "how far would it spread" against the current state of the causal model. The next layer is forward simulation: "what would break if this component degraded," asked before any symptom appears, not after. That requires modeling failure modes as first-class entities with their own preconditions and blast radius, rather than only reasoning from observed anomalies backward. It's a natural extension of the same causal model: the same questions, asked earlier instead of after the fact.

FAQ

What is a contrastive explanation in root cause analysis? Contrastive explanation answers "why this cause, not that one" instead of just "why did this happen." Research on explainable AI has found that people evaluate explanations relative to an alternative that didn't occur, even if that alternative is unstated. In root cause analysis, this means showing which diagnoses were considered and ruled out, not only the one that was selected.

How is this different from a deterministic safety veto layer? A safety veto checks whether a proposed action is safe to execute against live system state. Contrastive diagnosis explanation checks whether the reasoning behind the diagnosis holds up, and which alternatives were ruled out and on what evidence. The two are complementary: a veto layer is more useful with a diagnosis it can inspect, rather than a bare confidence score.

How do I use the new MCP tools to check an agent's diagnosis before it acts? Call get_potential_diagnoses or get_signal_potential_diagnoses against the symptom an agent is reasoning to see what could explain it and the causal chain behind each candidate. Pair this with rank_entities or get_diagnosis_observable_signals to see the blast radius of the leading diagnosis before approving any remediation action.

Doesn't surfacing alternative diagnoses just mean the model is less confident? No. Showing what could explain a symptom and the causal chain behind each candidate makes a diagnosis checkable, not uncertain. A model that only reports its top answer is asking to be trusted. A model that shows what else it considered and why each was ruled out is being reviewed. The second is the stronger claim.

What to do next

• Read How Causal Reasoning Addresses the Limitations of LLMs in Observability for the causal model this builds on.
• Read Beyond the Blast Radius for more on dependency-aware impact analysis.
• Learn more about Causely MCP tools