Trace agent work without mistaking telemetry for evaluation

A trace answers a narrower question

On October 14, 2026, Claude Platform announced beta OpenTelemetry export for Messages API requests and Managed Agents sessions, with SDK spans available in several languages, including C# (Claude Platform release notes). That creates a useful design question: how do I make an agent’s work visible in the operational trace for the application around it?

My answer starts with a boundary. A trace can help explain what happened during a request: which steps ran, where time was spent, and where an error surfaced. It cannot establish that the agent’s answer was correct, safe, or useful. Those are different questions and need different evidence.

I find it tempting to treat a detailed trace as a kind of record of truth. It is not. A span records an instrumented operation and its attributes; it does not independently validate the operation’s result. A successful model request can produce a wrong answer. A tool call can complete without doing what the user intended. A trace that ends without an error says something about execution, not quality.

That distinction matters when people use observability dashboards to make decisions about an AI feature. If I want to diagnose a timeout or a failed tool call, spans are a good place to start. If I want to know whether the system followed a policy or answered correctly, I need an evaluation process that checks outputs against explicit criteria. Mixing those goals into one “agent health” signal risks making operational success look like product success.

Join the agent to the application trace

Agent activity is rarely the whole request. A user may arrive through an API, pass through application logic, trigger a Managed Agents session, and then wait for the result to be stored or returned. Looking only at the agent’s spans leaves the caller’s work outside the picture; looking only at the application span hides what happened inside the agent.

Claude Platform’s OpenTelemetry API guide describes connecting API spans to application traces: call the SDK inside an application span, propagate trace context, and send the spans to the same observability backend. The Claude SDK tracing guide also describes SDK spans nested in application traces. I would use that as the target shape: the application operation provides the context, and the provider and tool activity appears beneath it where propagation works.

That last qualification is important. I would not assume every operation automatically joins the same trace just because all the data reaches one backend. Trace context needs to make it across the boundary. For a non-SDK client, the API guide shows copying the active context into request headers; it also notes that W3C trace context should be available when an application uses a different propagator. I would test the trace shape with a real request path before relying on it in a dashboard. A disconnected span may still be useful, but it is not the end-to-end view I intended.

For a Managed Agents session that continues across separate requests, I would also keep the application’s own correlation model clear. The Claude Platform OpenTelemetry reference describes a conversation identifier that can group Managed Agents spans across traces. That is useful for finding related work, but I would not confuse a conversation grouping with a single end-to-end trace. A trace answers “what happened in this operation?” A conversation identifier can help answer “which operations belong to this interaction?” They are related links, not interchangeable concepts.

Record deliberately, not exhaustively

The first question I would ask before enabling a new telemetry destination is not “Can I capture everything?” It is “Which failure do I need to diagnose, and what is the least data that lets me do it?” A trace can reveal more than timing and status if attributes, prompts, tool inputs, or outputs are attached. That may make debugging easier, but it can also put sensitive user or business data in a system with a different audience and retention policy than the application itself.

I would define a collection policy before turning on broad capture. Start with operation names, durations, status, and carefully selected identifiers that help connect work. Decide separately whether any prompt or result content is necessary. If content is not needed to answer a known diagnostic question, do not record it by default. If it is needed for a narrow investigation, I would treat that as a deliberate exception, with access and retention reviewed rather than assuming that the observability platform is an appropriate place for unrestricted conversation data.

The same care applies to baggage and propagated context. Context helps join spans, but I would avoid casually placing user-supplied values or secrets into fields that travel between services. I would keep identifiers opaque where possible and consider whether the people who can search the trace backend should be allowed to see the underlying interaction. These are design choices about data exposure, not small implementation details.

I would also keep the exporter boundary visible in the architecture. Claude Platform’s OpenTelemetry setup guide describes configuring a destination in the Claude Console for API spans. The application’s SDK spans and provider-generated spans may therefore have different configuration points, even when they eventually appear together. I would document who can change each destination, which environment it applies to, and how to disable or restrict export if the data policy changes. A trace is only useful operationally if its route and audience are understood.

Keep evaluation as a separate system

I would not put correctness into the trace by attaching a vague “good answer” label and calling that evaluation. Operational telemetry should help me see the path taken and investigate its failures. Evaluation should define what counts as an acceptable result, how that judgment is made, and how the result informs a release or a product decision.

Those systems can be connected without being conflated. A trace identifier can help an investigator find the execution behind an evaluation result, provided the identifier is handled safely and retained appropriately. Evaluation data can tell me that a class of outputs needs attention; the trace can help explain how one example was produced. Neither replaces the other. The trace does not supply a correctness label, and the evaluation does not necessarily explain a timeout, a broken dependency, or a failed tool operation.

I am still exploring how best to evaluate managed-agent behavior in real systems, so I would be cautious about prescribing a universal evaluation design. The operational boundary seems clearer: instrument the request path, connect spans where context is available, and keep the collected data limited to what the diagnostic use requires. Then define output evaluation on its own terms, rather than treating a clean trace as evidence that the agent did good work.

That separation gives traces a valuable but honest role. They make execution legible; they do not certify the result.