AI Trace

Origin & evolution

Tracing is old; the application to language models is recent. The shape borrows directly from distributed tracing, where a request fans out into nested spans, each timed and attributed. The OpenTelemetry GenAI Special Interest Group, formed in April 2024 under the Cloud Native Computing Foundation, standardised that vocabulary for AI: how to name a span, what attributes a tool call carries, which span kind marks an agent. Before it, tools like LangSmith, Langfuse, and Traceloop each used incompatible proprietary formats, which locked teams into one vendor's view of their own system. The convention is now adopted across Datadog, Google Cloud, AWS, and Azure.

How it works in practice

A support agent answers a refund question. The top-level trace captures the user message, the final reply, 4,200 prompt tokens, 380 completion tokens, and 2.1 seconds of latency. Inside it sit three child spans: a retrieval call against the knowledge base, a lookup_order tool call that returns the order status, and the final generation. A week later the same flow starts returning wrong refund amounts. The engineer opens one failing trace, sees the lookup_order span returned a stale total, and fixes the tool rather than the prompt. The trace turned "the AI is wrong sometimes" into a precise, located fault.

AI Trace vs. its neighbours

• AI Model🤖AI ModelAI & Machine LearningAn AI or ML model used within the productView reference → is the thing being observed; the trace is the observation👁️ObservationUser ResearchA specific behaviour or statement observedView reference →. One model produces thousands of traces, each a separate execution with its own inputs and timing.
• Agent Session🧑‍💻Agent SessionWorkflows & AgentsAn agent working sessionView reference → groups many traces into one coherent user interaction or task☑️TaskProduct SpecificationA unit of work within a story or epicView reference →. A session is the conversation; a trace is a single turn or call within it.
• AI Cost Tracker💸AI Cost TrackerAI & Machine LearningA tracker for AI inference costsView reference → aggregates the token and latency figures that traces emit. The trace is the raw record of one call; the cost tracker is the running total across many.

In the graph

In the Unified Product Graph, an AI Trace sits in the AI region as the observability record of model execution. The model that ran it connects through ai_model_produces_ai_traceAI ModelproducesAI Tracecausal, and nested child spans connect through ai_trace_spawns_ai_traceAI TracespawnsAI Tracehierarchy, which lets a parent agent call hold its retrieval and tool sub-calls as queryable children. That self-referencing edge matters because agent behaviour is recursive: debugging means walking from the symptom⚠️SymptomEngineeringA symptom of a problemView reference → at the top down to the span that actually failed.