Observability
What are the best practices for AI agent observability?
Give every agent run an ID, log each step's input, tool call, decision and result in order, and record how the run ended. Then watch three signals: error rate, volume against a baseline, and silence. Review a summary every week.
AgentTracer stores the events you send. It does not score output quality, it has no loop detector, and it does not replay a run. A repeated tool name on one run is on how do I catch an AI agent stuck in a loop?
The record of each action, kept outside the agent, is on how do you create an audit trail for AI agent actions? The four alert kinds are written out on how to monitor AI agents.
Steps
- Give the run an id and a final status. On POST /api/v1/events, run.id is 1 to 128 characters. run.status may be running, completed, failed, or cancelled. Without a run id, events from two conversations land in one pile. The project timeline prints that id under the event name when you send it. The status is stored on the run. The timeline does not show it as its own column.
- Log the input, not only the tool name. Put the argument that explains the step in the payload. A name alone does not show what was asked. A name is optional. AgentTracer stores the JSON you send.
- Record decisions and errors as events. Recommended types are tool_call, decision, message, error, and span. Other type strings are stored. An event counts toward the error rate only when its type is error, any case. Up to 100 events can ride in one request.
- Keep payloads small and leave secrets out. Include the argument that explains the step, not the whole document. Do not put secrets, raw credentials, or personal data in the payload. AgentTracer does not redact it. The request body must stay under 1,000,000 characters.
- Alert on error rate, volume, and silence. A check around 15:00 UTC compares the last 24 hours with the previous 7 windows. It flags an error-rate spike, a volume spike, a volume drop, or an agent that went silent. It has no loop detector. It does not score output quality, and it does not replay a run. A project can change the thresholds.
- Review the weekly summary. The Monday digest, around 14:00 UTC, covers the trailing 7 days: event counts, top tool names, recent errors, and anomalies. The same summary is in the dashboard. You can turn the email off.
The example below is one JSON POST to /api/v1/events. OpenTelemetry is not required. There is no official SDK. Any language that can send HTTP can record a run.
curl -sS -X POST https://agenttracer.net/api/v1/events \
-H "Authorization: Bearer $AGENTTRACER_API_KEY" \
-H "Content-Type: application/json" \
-d '{"agent":{"id":"support-bot"},"run":{"id":"run_01h","status":"failed"},"events":[{"type":"tool_call","name":"search_docs","timestamp":"2026-10-29T00:12:00.120Z","payload":{"query":"refund policy"}},{"type":"decision","name":"cite_policy","timestamp":"2026-10-29T00:12:00.880Z","payload":{"choice":"cite_policy"}},{"type":"error","name":"empty_result","timestamp":"2026-10-29T00:12:01.200Z","payload":{"reason":"no hits"}}]}'Checklist
| Signal | What to log | Why it matters | Where it shows in AgentTracer |
|---|---|---|---|
| Error rate | type error, with a name and a small payload. | A jump against the recent baseline is the failure signal you can alert on. | Daily error-rate spike. The latest window needs at least 20 events, an error rate of at least 20%, and a rate at least 2× the baseline. |
| Volume | One event per step, each with a timestamp. | Volume against a baseline shows a burst or a drop without a price list. | Daily volume spike and volume drop. A spike needs at least 20 events and at least 3× the baseline. A drop needs a baseline of at least 30 events and a latest window at or below 25% of that average. |
| Silence | An agent id on the events you send. | A busy agent that then sends nothing is a different failure from a bad step. | Silence alert. The agent needs at least 5 events in the previous 24 hours and zero in the last 24 hours. |
| One run | run.id, and run.status when the run ends. | Steps from one conversation stay together, in the order of their timestamps. | Project timeline of the newest 100 events. The run id is under the name. Status is stored on the run and is not its own column. |
| A decision | type decision, the choice, and the input that led to it. | The next tool call is easier to judge when the choice is on the timeline. | The decision is a row on the project timeline, with its payload. |
Questions
Do I need OpenTelemetry to record an agent run?
No. One JSON POST to /api/v1/events is enough. OpenTelemetry is not required, and there is no official SDK. Any language that can make an HTTP request can record a run.
What should I not log?
Do not send secrets, raw credentials, or personal data in the payload. AgentTracer stores that JSON and does not redact it. It does not pull prompts out as their own fields.
How long does AgentTracer keep events?
Free keeps events for 14 days, Pro for 30 days, and Team for 90 days. A daily job deletes events older than that window. The windows are on the pricing page.
Does AgentTracer score output quality?
No. It does not score output quality and it does not run evals. It stores the events you send and alerts on error rate, volume, and silence. It has no loop detector, and it does not replay a run. A repeated tool name is on How do I catch an AI agent stuck in a loop?