Engineering note
Tracing AI workflows across the whole product
Carry one run identity across model calls, workflow stages, tools, application logs, and provider callbacks. Each telemetry system can keep its specialty while the shared identity reconstructs one product outcome.
By Raphael Mansueto · Reviewed
Start with the product run, not the model call
A model trace explains only one part of a production workflow. Ample News crossed grounded research, topic generation, interview preparation, voice and video providers, social scheduling, and delivery callbacks. A tracked workflow run supplied the product-level identity needed to relate those stages.
Langfuse captured model, agent, tool, token, latency, and cost behavior. OpenTelemetry connected those spans to the surrounding workflow. Structured application logs and Sentry covered operational context and failures outside the model boundary.
Let each signal answer a different question
Forcing every detail into one observability product usually weakens the result. The useful design is correlation, not duplication.
- AI traces explain prompts, tools, model responses, tokens, and cost
- Distributed traces explain stage and provider latency across service boundaries
- Application logs explain domain state and recovery decisions
- Error monitoring explains exceptions, releases, and affected execution context
Control telemetry volume deliberately
Long-running workflows can produce more spans than operators can use. Low-value infrastructure spans should be filtered while agent, workflow, tool, and model events remain available. Sampling and retention are product-operability decisions because they affect both investigation quality and cost.
Observability makes behavior inspectable, not automatically correct
Correlation shortens the path from a failed product outcome to the responsible stage, but it does not replace typed boundaries, retries, idempotency, or human decisions. Private prompts, customer data, raw traces, and provider identifiers also require disclosure controls before any evidence is made public.