Tracing a request through an AI feature when it spans four services

A director of platform at a fintech asked me to sit in on an incident review. A support agent had escalated a case where the company’s AI assistant gave a customer a wrong account status. The team had good instincts. They pulled logs. Then they hit a wall.

The feature was not one service. A request came into an orchestrator. The orchestrator called a retrieval service to fetch account context. It called a model provider to generate the answer. It called a post-processor to format and check the response. Then it wrote a record to the system of record. Four hops, four sets of logs, four teams.

Each service had logs. None of them shared an ID. So the retrieval logs and the model logs and the post-processor logs all existed, but nobody could prove which retrieval belonged to which answer. They were staring at four piles of true statements that would not connect.

The lesson: propagate one trace ID through every hop, including the model call

Distributed tracing is old news for normal microservices. AI features get treated as an exception, usually because the model call feels like a black box you hand off to a vendor. That instinct is exactly backwards. The model call is the hop where the most can go wrong and where you have the least native visibility, so it needs the trace ID more than any other hop, not less.

Generate a trace ID at the edge, the moment a request enters the feature. Then carry it everywhere:

Pass it as a header or field into the retrieval service and log it there.
Attach it to the metadata you keep about the model call, alongside the prompt and version.
Pass it into the post-processor so its decisions are tied to the same request.
Write it onto the record in the system of record.

When every hop stamps the same ID, an incident becomes a single query. You pull one trace and see the whole path: what came in, what got retrieved, what the model produced, what the post-processor changed, what got written. The fintech’s wrong-status bug turned out to be a stale cache in retrieval. The model answered correctly given the context it was handed. Without the shared ID, they would have blamed the model for a week.

What good teams do

Create the trace ID once, at the entry point, and never regenerate it midway.
Include it in structured logs at every service, as a first-class field you can filter on.
Store it with the AI call envelope so model behavior joins cleanly to everything else.
Propagate it across async boundaries too, including queues and background jobs, where IDs quietly get dropped.
Make “no trace ID” a failing check in code review for any service in the path.

How we approach it at Density Labs

Teams rarely skip tracing on purpose. It erodes at the seams, where one service was written by a different team, or where the model call was bolted on later and never wired into the existing tracing setup. The result is a feature that works fine until the day it does not, and then nobody can follow a single request end to end.

Our AI Opportunity Assessment is a fixed two-week engagement for 2,500 dollars that scopes the real production work before you build. For a multi-service AI feature, part of that work is walking one real request through every hop and confirming a single ID survives the whole trip, model call included. Where it breaks, we show you exactly where and what it costs you during an incident.

An AI feature is a distributed system. Trace it like one, or accept that some incidents will never be solved.