Error-rate spike
The latest window has at least 20 events, the error rate is at least 20%, and the rate is at least 2× the baseline rate.
Production
Monitor AI agents in production by recording each run's actions and alerting when the error rate, the volume, or the silence changes. The final answer is not enough. A tool loop and a stopped agent both look quiet in a chat log.
The steps below are the practice. The numbers in the alert section are AgentTracer's defaults, from the plan and threshold tables in this repo, and a project can change them. If you need token cost or a self-hosted eval stack, start with AI agent observability tools instead of forcing that job onto a timeline.
AI agent observability is the ordered record of a run, kept long enough to debug one failure and to notice when the pattern changes. Agent tracing is that ordered record. Monitoring is the note that the error rate, the volume, or the silence moved. A definition of the model-call record, which is a different question, is on what is LLM observability?
AgentTracer checks once a day, around 15:00 UTC. The default window is the last 24 hours, compared with the previous 7 windows. An event counts as an error when its type is error, any case. You can replace these numbers per project.
The latest window has at least 20 events, the error rate is at least 20%, and the rate is at least 2× the baseline rate.
The baseline average is above zero, the latest window has at least 20 events, and volume is at least 3× that average. This is the closest signal to a cost spike when you are not pricing tokens.
The baseline averaged at least 30 events a window, and the latest window is at or below 25% of that average.
An agent sent at least 5 events in the previous 24 hours and zero events in the last 24 hours. The missing timeline is the alert.
What those four look like in the product is on AI agent monitoring. The check does not assign a risk score and does not price tokens. A risky tool shows up only if you recorded that call.
Alerts catch a break. A weekly read catches the slow drift. AgentTracer emails a digest on Monday around 14:00 UTC for the trailing 7 days: event counts, top tool names, recent errors, and anomalies. The same summary is in the dashboard. Projects with nothing to say are counted as quiet and left out of the body. You can turn the email off. The daily alert email is separate, and you can turn that off too.
Agent tracing is the ordered events of one run. Five types cover a production agent: tool_call, decision, message, error, and span. Other type strings are stored. A name is optional. A timestamp is required. Up to 100 events can ride in one request. The body must stay under 1,000,000 characters.
Create an account, open the project, and copy the API key. The key is shown once. Send it as Authorization: Bearer. There is no official SDK. The quick start on the homepage is the same request.
Free is $0 and includes 50,000 events a month, then ingest stops. Pro is $39 a month for 500,000 events. Retention is 14 days on Free, 30 days on Pro, and 90 days on Team. AgentTracer is hosted only.
curl -sS -X POST https://agenttracer.net/api/v1/events \
-H "Authorization: Bearer $AGENTTRACER_API_KEY" \
-H "Content-Type: application/json" \
-d '{"agent":{"id":"support-bot"},"run":{"id":"run_01h"},"events":[{"type":"tool_call","name":"search_docs","timestamp":"2026-10-29T00:12:00.120Z","payload":{"query":"refund policy"}}]}'Give each run an id, record every tool call, decision, message, and error with a timestamp, and alert when the error rate, the volume, or the silence changes against a recent baseline. The final answer is not the monitor. A tool loop and a stopped agent both look quiet in a chat log.
AI agent observability is the ordered record of what an agent did on a run, kept so you can debug one failure and notice when that pattern changes. Agent tracing is the ordered record. Monitoring is the alert when the record's shape changes.
Agent tracing is the ordered events of one run: tool calls, decisions, messages, errors, and spans. A trace without timestamps is a bag of logs. A trace that only stores the last message hides the step that failed.
Four kinds, on a daily check around 15:00 UTC: an error-rate spike, a volume spike, a volume drop, and an agent that went silent. The defaults compare the last 24 hours with the previous 7 windows. Each project can change the thresholds. A weekly digest goes out on Monday around 14:00 UTC.
No, not if you send it in the background. AgentTracer isn't a proxy, so your model calls never pass through us. Each event is a separate small HTTP request. Send it without waiting for the reply, or give it a short timeout, and your agent never waits on us.