Agent observability: how to see what your AI agent actually did

A conventional business application fails in ways engineers know how to debug: an error code, a stack trace, a log line. An AI agent fails differently. It returns a confident, well-written answer that is wrong, or it quietly takes six steps where two would do, and nothing in a standard server log explains why.

Observability is the practice of recording enough about each run that you can reconstruct what happened afterward. For agents, that record is called a trace.

Why an agent is harder to debug than software

Anthropic's engineering team, writing about its multi-agent research system, put the core problem plainly: "Agents make dynamic decisions and are non-deterministic between runs, even with identical prompts" ^[1]. Its users reported agents "not finding obvious information," and the team could not tell why until it added full production tracing, which let it "diagnose why agents failed and fix issues systematically" ^[1].

Cost behaves differently too. In Anthropic's data, agents typically use about four times more tokens than chat interactions, and multi-agent systems about fifteen times more ^[1]. A loop that retries a failing tool silently multiplies your bill.

What a trace contains

A trace is one end-to-end run. Inside it, individual steps are recorded as spans. OpenAI's Agents SDK, a widely used open-source framework, shows a typical default set: a trace wraps each run, with spans for agents, model turns, LLM generations, function tool calls, guardrail checks and handoffs between agents. Generation spans capture model inputs and outputs, and function spans capture tool-call inputs and outputs ^[2].

That is the minimum worth demanding from any agent platform, whether you build or buy it. With it, a bad answer can be traced back to the specific step where things went wrong: the search that returned nothing, the tool that errored, the handoff that dropped context.

Standards and tooling

Observability tools come from many vendors, and the OpenTelemetry project, the common open standard for application telemetry, has argued for shared conventions so that teams are not locked into one vendor's format ^[3]. Its GenAI work distinguishes between frameworks that bake instrumentation in, which is simple to adopt but can lag behind the standard, and external instrumentation libraries, which keep the framework lean but risk fragmentation ^[3]. The OpenAI documentation lists more than two dozen third-party integrations that can receive its traces, which shows how crowded the tooling market already is ^[2].

Practically, ask any vendor whether traces can be exported in an open format to the monitoring stack you already run.

Privacy: the setting people miss

Detailed traces contain customer data. In the OpenAI Agents SDK, the option that stores model and tool inputs and outputs in spans defaults to on, and can be switched off by configuration ^[2]. Anthropic describes taking a different route for its research system: monitoring "agent decision patterns and interaction structures" without reading individual conversation contents, to protect user privacy ^[1].

Neither approach is universally right. The point is that retention and access to traces need an explicit decision, because a trace can hold exactly the personal or confidential data you were trying to protect.

From traces to evaluation

Traces are most valuable when paired with tests. Anthropic's team began evaluating with about 20 queries that reflected real usage, noting that small samples work when early improvements are large ^[1]. For agents that change state in other systems, it also judged whether the agent reached the correct final state rather than checking every step ^[1].

A sensible starting set: keep traces for every production run for a fixed retention period, sample failed or escalated runs for weekly review, and turn each recurring failure into a permanent test case.

Practical implication: Before putting an agent in front of customers, confirm you can open any single run and see every model call, tool call and result, and decide how long that record is kept and who can read it.