Your Agents Are Observable. That Doesn't Mean They Work.

Unpopular opinion: most of the money going into “AI observability” right now is making teams feel safer without making their agents any better. The dashboards are real, the traces are detailed, the latency histograms are beautiful. And none of it answers the only question that matters when an agent is making decisions on your behalf: is it actually getting the answer right?

LangChain’s State of Agent Engineering 2025 report, drawn from more than 1,300 practitioners surveyed over two weeks in late November and early December, put a number on the gap. Eighty-nine percent of respondents had implemented some form of observability for their agents. Just 52.4 percent were running offline evaluations against a test set. That is a spread of nearly 37 points between teams that can watch their agents and teams that can tell whether their agents work.

That spread is not a curiosity. In June 2025, Gartner predicted that over 40 percent of agentic AI projects would be canceled by the end of 2027, and named three drivers: escalating costs, unclear business value, and inadequate risk controls. Read those three again with the survey in mind. Every one of them is a measurement failure. A team that cannot measure whether an agent produces correct, valuable output cannot defend its cost, cannot demonstrate its value, and cannot bound its risk. The cancellation wave Gartner is forecasting is, in large part, an evaluation wave that never happened.

What observability tells you, and the three questions it cannot answer

Observability and evaluation get talked about as if they were two names for the same maturity, and the tooling vendors are happy to let that confusion stand. They are not the same discipline. They answer different questions, at different times, for different reasons.

Observability is the practice of making a running system’s behavior legible: traces of each step, the tool calls an agent made, the tokens it consumed, the latency of each hop, the errors it threw. It is descriptive and it is retrospective. It tells you, with precision, what happened.

Evaluation is the practice of judging whether what happened was any good. It compares the agent’s output against a standard, whether that standard is a known-correct answer, a rubric, a policy, or a human’s judgment. It is normative. It tells you whether the behavior was right.

The distinction becomes concrete the moment you write down what observability structurally cannot tell you, no matter how rich the traces get. There are three such questions, and they are the ones that decide whether an agent program survives contact with a finance review.

The first is correctness. A trace shows that the agent called the refund tool with a specific amount. It cannot tell you that the amount was wrong. The call succeeded, the span is green, the latency was fine, and the customer was refunded twice. Observability recorded a healthy transaction. Only an evaluation, checking the decision against what the decision should have been, surfaces the defect.

The second is direction: is the agent getting better or worse? Observability shows you today’s behavior and yesterday’s behavior, but it has no notion of correct, so it cannot tell you whether a prompt change, a model upgrade, or a new tool improved quality or quietly regressed it. Teams discover regressions the way they discover everything without evals: a customer complains.

The third is worth. Observability can tell you an agent cost eleven cents and made nine tool calls. It cannot tell you whether those nine calls produced an outcome worth eleven cents. Cost per token is an observability metric. Cost per correct outcome is an evaluation metric, and it is the only one a CFO cares about.

A green trace tells you the agent did something. It says nothing about whether the something was the right thing. That difference is the entire distance between a demo and a product.

Why the gap exists

The gap is not a mystery, and naming why it opens is the first step to closing it. There is a recurring shape to it across the teams we work with, and it usually comes down to three habits that each feel reasonable in isolation.

Ship and watch. Observability tooling matured first and installs in an afternoon: drop in an SDK, point it at a collector, and the traces start flowing. Evaluation asks for something the SDK cannot provide, which is a definition of what correct means for your task. So teams ship the agent, wire up the traces, and tell themselves they will “watch it in production.” Watching is not evaluating. The symptom is a team that can produce a detailed timeline of any individual agent run on request but cannot tell you its task success rate over the last thousand runs.

The demo-passed fallacy. The agent worked in the demo. It handled the three test cases the team tried by hand, and everyone saw it. That success gets silently promoted into evidence of production quality, when all it established was that the agent can succeed, not how often it does. The symptom is confidence that scales with how impressive the demo was rather than with how much of the input distribution has actually been measured.

Dashboard comfort. A wall of green panels is genuinely reassuring, and reassurance is psychologically hard to distinguish from information. A team with a beautiful observability setup feels like a team in control. The symptom is that nobody can answer “what is our agent’s accuracy on the cases that matter?” but everybody can pull up the latency dashboard from memory.

Which evaluation a given behavior actually needs

The reason many teams stall is that “add evals” sounds like one large, undifferentiated project, and the perception of scale is itself the blocker. It is not one project. Different agent behaviors need different evaluation modes, and most of them are cheaper than the team fears. The decision turns on two properties of the output: whether correctness is mechanically checkable, and how much a wrong answer costs.

flowchart TD
    A[Agent behavior to evaluate] --> B{Is correctness<br/>mechanically checkable?}
    B -->|Yes: exact match, schema,<br/>tool result, assertion| C[Offline test set in CI<br/>Run on every change]
    B -->|No: subjective quality| D{Is a wrong answer<br/>high stakes or irreversible?}
    D -->|Yes: money, deletion,<br/>external message| E[Human review gate<br/>before the action<br/>+ continuous sample audit]
    D -->|No: low stakes| F[LLM-as-judge on a rubric<br/>for breadth<br/>+ periodic human spot-check]
    C --> G[Track success rate<br/>and cost per correct outcome]
    E --> G
    F --> G

The left branch is where teams should start, because it is the cheapest and it is more of their workload than they expect. A surprising share of agent behavior is mechanically checkable: did the agent produce valid JSON against the schema, did it call the right tool with arguments in range, did the extracted invoice total match the known value, did the SQL it generated return the expected rows. For all of these you do not need an LLM to judge anything. You need a set of inputs with known correct outputs and a few assertions, run in continuous integration on every prompt or model change. This is the single highest-leverage evaluation a team can build, and it is ordinary software testing pointed at a probabilistic component.

Here is the shape of that offline evaluation, deliberately minimal, so the point is unmistakable: it is a test loop over a golden dataset, not a platform.

# A golden dataset: inputs paired with their known-correct outcomes.
# This is the artifact most teams skip, and the one that makes everything else possible.
GOLDEN = [
    {"input": "Refund order 4021, damaged on arrival", "expect_tool": "issue_refund", "expect_amount": 49.99},
    {"input": "Where is my order 4021?",                "expect_tool": "get_status",   "expect_amount": None},
    # ... grow this from real, anonymized production traffic, especially the cases that failed
]

def evaluate(agent, dataset):
    correct = 0
    for case in dataset:
        result = agent.run(case["input"])          # observability captures HOW this ran
        # evaluation asks the question a trace cannot: was the decision right?
        tool_ok   = result.tool_called == case["expect_tool"]
        amount_ok = result.amount == case["expect_amount"]
        if tool_ok and amount_ok:
            correct += 1
        else:
            log_failure(case, result)              # failures become tomorrow's regression tests
    return correct / len(dataset)                  # the number a demo can never give you

The trace tells you the agent called issue_refund. This loop tells you it called issue_refund when the customer only asked where their order was. Same run, two entirely different questions answered.

The trade-offs, stated plainly

The three modes in the decision tree are not a ladder you climb; they are tools you match to the behavior, and each has a cost that determines when it earns its place.

An offline test set is cheap to run and unambiguous, but it only covers behaviors with a checkable answer and it is only as good as the dataset behind it. Its real cost is curation: someone has to assemble and maintain inputs paired with correct outputs, and that dataset has to grow from production or it goes stale. Choose it, and start with it, whenever correctness is mechanical.

LLM-as-judge scales to subjective quality that no assertion can capture, tone, helpfulness, faithfulness to a source, and it runs at machine speed across thousands of cases. The cost is that the judge is itself a probabilistic system with its own biases, and an uncalibrated judge gives you confident numbers that are quietly wrong. Choose it for breadth on lower-stakes subjective output, and keep it honest with periodic human spot-checks. Never let it grade itself into a metric nobody has audited.

Human review is the most trustworthy and the least scalable. It is the right and sometimes the only acceptable gate when a wrong answer moves money, deletes data, or sends an irreversible message to a customer. The cost is throughput and headcount, so you reserve it for the high-stakes tail and use sampling, not full coverage, everywhere else.

The goal is not to evaluate everything the same way. It is to stop evaluating nothing and calling the dashboard a safety net.

There is also an honest case for observability alone, and pretending otherwise would undercut the argument. If an agent is genuinely low stakes, a human is in the loop on every output before it has any effect, and the cost of a wrong answer is trivial and reversible, then rich traces plus that human may be all the evaluation the situation warrants. The failure is not skipping formal evals in that narrow case. The failure is assuming your case is that case without checking, which is exactly the assumption the demo-passed fallacy encourages.

Closing the gap without boiling the ocean

None of this requires a platform migration or a new team. The teams that close the gap fastest treat it as a sequence of small, concrete steps, each of which produces a number they did not have before.

Notice that observability is not the enemy in any of this. It is the raw material. Your traces are where the golden dataset comes from, your failed runs are your best test cases, and your production monitoring is what catches the drift your offline suite has not learned about yet. The argument is not that teams over-invested in observability. It is that they stopped one discipline too early and mistook the instrument panel for the flight.

The teams whose agent programs will still be running in 2027, when Gartner expects nearly half of them to be gone, will not be the ones with the most detailed traces. They will be the ones who can put a defensible number next to the sentence “our agent gets this right X percent of the time,” and who watched that number move in the right direction every time they changed something. Observability gets you the data. Evaluation gets you the number. Only one of them keeps the project funded.

If you are running agents in production on observability alone and you are being asked, or are about to be asked, to justify what they cost, that is the conversation worth having now rather than after the budget review. We have helped teams stand up exactly this kind of evaluation layer on top of the observability they already have, and we are glad to talk through what the first offline eval should cover in your case. The weakest link in this argument is that a genuinely low-stakes, human-in-the-loop agent may not need formal evals at all; if you have scaled an agent to production on observability alone and it held up under real load, we would genuinely like to hear how you knew it was working.