AI Agent Observability: A Practical Guide for 2026

Your agent failed at step 14. Which step 14?

A single LLM call is easy to debug. One prompt in, one completion out — log both, look at the pair, find the problem.

An agent is a different animal. One user request fans out into a planning call, three tool calls, a retrieval step, two more model calls to synthesize, and a retry somewhere in the middle because a tool returned malformed JSON. When the final answer is wrong — or the run costs 40x what you expected — "we log our prompts" tells you almost nothing. Agent observability is the discipline of capturing that entire execution tree so you can answer three questions: what did the agent actually do, what did it cost, and where did it go wrong?

This guide covers what to capture, the three instrumentation approaches available in 2026 (SDK, OpenTelemetry, gateway), and the two failure modes most teams discover late: runaway costs and traces full of PII.

What is agent observability?

Agent observability extends LLM observability from single request/response pairs to full multi-step executions. Instead of a flat log of API calls, you get a trace: a tree of spans representing every step the agent took, in order, with timing, token counts, and inputs/outputs attached.

A useful trace for one agent run looks like this:

run: "refund request for order #8812"          6.2s   $0.0841
├── llm: plan next action (gpt-5.6)            1.1s   1,204 tok
├── tool: lookup_order(order_id="8812")        0.3s
├── llm: plan next action (gpt-5.6)            0.9s   1,876 tok
├── tool: refund_policy_search(query=...)      0.8s
├── llm: plan next action (gpt-5.6)            1.0s   2,341 tok
│   └── retry: tool args failed validation     +0.4s
├── tool: issue_refund(amount=49.00)           0.6s
└── llm: compose reply (gpt-5.6-luna)          1.1s   2,090 tok

The difference from classic LLM logging is structural. You're no longer asking "what was the prompt?" — you're asking "why did the third planning call decide to issue a refund, and what context had accumulated by then?"

Concretely, agent observability needs to capture:

• +The step sequence — every model call and tool call, parent-child relationships, and ordering
• +Tool calls with arguments and results — the agent's decisions are only explainable if you can see what its tools returned
• +Per-step tokens and cost — agents multiply spend; a trace that can't attribute cost per step can't explain a $600 day
• +Latency per step — agents feel slow as a whole; the trace tells you whether it's the model, a tool, or retry loops
• +Errors and retries — failed tool calls, schema validation failures, and the model's recovery attempts
• +Run-level metadata — user, session, agent version, prompt version, environment

Why single-call LLM observability isn't enough

Most teams start with request logging — every OpenAI or Anthropic call captured somewhere. Then they ship an agent and discover four problems at once.

Cost stops being linear. A chat feature costs roughly the same per message. An agent's cost per run varies by an order of magnitude depending on how many steps it takes. Loops are the pathological case: an agent that keeps calling a failing tool, apologizing, and trying again can burn thousands of calls before anyone notices. Flat call logs show volume going up; they don't show which runs looped or why.

Failures are mid-trajectory. In production, agents rarely fail with an exception. They fail by taking a wrong turn at step 6 and confidently producing a wrong answer at step 14. Without the trace, your debugging session starts from a user complaint and a pile of disconnected API logs.

Nondeterminism compounds. One model call is nondeterministic. Chain nine of them with tool calls in between, and the same input can produce structurally different executions. You can't reason about "the" behavior of an agent — you can only look at distributions across real traces, which is also why running evals on real traffic beats synthetic test sets for agents.

The blast radius is bigger. Agents move a lot more data than a chat box does. Security researchers estimate AI agents move roughly 16x more data than human users — every tool result, retrieved document, and database row flows through the model context. That has both a security implication (covered in our post on AI agent data leaks) and an observability implication we'll get to below: your traces are now a copy of all that data.

Three ways to instrument agents

In 2026 there are three real approaches, and most vendors are built around one of them.

The honest summary: SDK instrumentation gives you the deepest single-app traces, gateway capture gives you the widest coverage for the least work, and OTel is the interchange format connecting them. LangSmith now supports end-to-end OpenTelemetry, and the OTel GenAI semantic conventions are the closest thing the industry has to an agreed schema for LLM spans.

Why the gateway angle matters for agents specifically

Here's the property that makes gateway-level capture unusually good for agents: every step an agent takes that costs money or makes a decision is an LLM API call, and every one of those calls passes through the same egress point.

Tool execution happens in your code, but tool selection — the arguments the model chose, the results fed back into context — is visible in the request/response bodies of the model calls themselves. Which means a gateway sees the agent's full decision sequence without a single line of instrumentation:

• +The planning call and its output
• +Every tool call the model requested, with arguments
• +Every tool result, because it appears in the next request's messages
• +Tokens, cost, latency, and errors for each step
• +All of it grouped into a conversation by message-history continuity

This is also the only approach that survives organizational reality. Your company doesn't run one agent — it runs a LangGraph service, a CrewAI prototype, three internal scripts, and developers using coding agents. Instrumenting each one with an SDK is a project per codebase. Pointing their base URL at a gateway is a config change per codebase, and suddenly the finance question ("what did agents cost us this week, by team?") has one answer in one place. We've written before about what an AI gateway is if the concept is new.

The right architecture for most teams is hybrid: gateway capture as the always-on baseline across everything, plus OTel/SDK spans in the one or two agents complex enough to need custom instrumentation.

The metrics that actually matter

Once traces are flowing, resist the urge to dashboard everything. Five metrics explain most agent behavior in production:

• +Cost per run (p50 / p95 / max) — the p95-to-p50 ratio is your loop detector. A healthy agent sits around 2–3x; a drifting prompt or flaky tool pushes it past 10x before total spend looks alarming.
• +Steps per run — the same signal in structural form. Alert on the distribution shifting, not on a fixed threshold.
• +Tool error rate per tool — agents mask failing tools by retrying and improvising. A tool at a 30% failure rate can hide behind a "working" agent for weeks while tripling cost.
• +Retry rate on model calls — schema validation failures and malformed tool arguments concentrate here after model or prompt version changes.
• +Trace completion rate — the fraction of runs that reach a terminal state versus hitting step limits, timeouts, or budget caps. This is your closest proxy for "the agent actually finished the job."

Note what's absent: aggregate token counts and total spend. They matter for the invoice, but they're trailing indicators. Distribution shifts in the five metrics above are the leading ones.

The two problems agent traces create

Observability is supposed to reduce risk. Agent traces, done naively, add two new risks.

1. Your traces are now a PII warehouse

Remember the 16x data movement. Agent traces capture tool results — CRM records, support tickets, retrieved documents, database rows. All of it lands, verbatim, in your observability store. If that store is a third-party SaaS, you've created a shadow copy of your most sensitive data in a system your DPIA probably doesn't mention.

This is the observability paradox: the more complete your traces, the bigger your data-protection problem. We covered the mechanics in Your LLM observability tool is logging PII; with agents, the volume and sensitivity are strictly worse.

The fix is structural, not procedural: redact PII before it's stored — ideally at the same choke point where traces are captured — so traces stay debuggable (consistent placeholders like [PERSON_1], [EMAIL_1] preserve referential structure) without being a breach waiting to happen.

2. Observability without limits is a dashboard for disasters

A trace that shows you the agent looped 400 times is useful. A gateway that stopped it at 50 calls is better. Agent observability matures into agent control:

• +Budgets and hard spend caps per agent, team, or API key, so a runaway loop hits a wall instead of your invoice — see hard spend caps
• +Tool-call restrictions, so an agent that was prompt-injected into calling delete_records gets blocked at the egress point — see restricting agent tool calls
• +Alerting on trace anomalies — step counts, retry rates, and cost-per-run distributions shifting are your earliest signal that a prompt change broke something

If your observability layer and your control layer are the same layer, the loop from "detected" to "prevented" is one policy, not a cross-team project.

How Grepture helps

Grepture is an AI gateway, which makes it the choke-point approach described above — with two modes depending on how much control you want in the hot path.

Proxy mode routes agent traffic through Grepture: full traces grouped by conversation, per-step tokens/cost/latency, plus enforcement — PII redaction before anything is stored or sent to a provider, budgets with hard caps, and tool-call restrictions.

Trace mode is for latency-sensitive agents: requests go directly to your provider, and the SDK captures metadata asynchronously — same dashboard, zero added latency in the agent loop.

Either way, you get agent-shaped answers without instrumenting every framework: which runs looped, what each step cost, what the model saw when it made the wrong call — with PII already masked in the trace. Setup is a base-URL change; the monitoring and logging guide walks through it.

Key takeaways

• +Agent observability means traces, not logs — capture the full execution tree (model calls, tool calls, retries) with per-step cost and latency, or you can't debug mid-trajectory failures.
• +Single-call logging breaks down on agents because cost is nonlinear, failures are silent wrong turns, and behavior only makes sense as a distribution across runs.
• +Three instrumentation options exist — SDK (deepest, most work), OpenTelemetry (the emerging interchange standard), and gateway capture (widest coverage, zero code changes). Most teams should run gateway capture as the baseline and add OTel spans where depth is needed.
• +Agent traces are a PII liability by default — redact at the capture point, not after storage.
• +Pair observability with control: budgets, spend caps, and tool-call restrictions turn "we saw the runaway loop" into "the runaway loop got stopped."