Do my LLM traces record token usage and user IDs?

fleetdiff inspect reads a local OTLP capture and reports which usage and identity fields are present, missing or zero.

In this synthetic capture, all four model attempts have user IDs, two have session IDs, and one is missing token fields. Another has input and output recorded as zero, with no declared origin.

Check a capture

Requires Git, curl, gh, shasum, tar, and Go 1.26.9 or later. Linux or macOS, amd64 or arm64. Go generates the synthetic capture from the released tag; inspection runs with the verified v0.6.0 binary.

git clone --depth 1 --branch v0.6.0 https://github.com/llm-measurement/fleetdiff.git fleetdiff-traces-usage
cd fleetdiff-traces-usage
sh scripts/install.sh v0.6.0 ./release
GOWORK=off go run ./examples/inspect > synthetic-traces.json
./release/fleetdiff inspect synthetic-traces.json

The report starts with readiness and the next step:

Your traces can fully answer 2 of 7 questions.
Next: Record both input and output usage; resolve invalid, conflicting, or subset-violating values. Declare unavailable usage instead of filling in zeros.

Usage fields

4 model attempts across 6 captured spans; 170 reported tokens.
Usage: 3 complete, 1 missing either field.
Origin unknown: input 4, output 4 attempts.

The zero-valued fields are present, so they are included in the three complete observations. The origin check keeps their source unknown.

User and session fields

user: ~4 distinct; identity present on 4/4 attempts

session: ~2 distinct; identity present on 2/4 attempts

The report also lists standard attribute names and uses local aliases for individual users and sessions.

Check your own traces

Use the capture recipes for a collector file exporter, a Python SDK or LiteLLM, then pass your local capture to fleetdiff inspect. Keep the capture local and share the default report instead of the raw file. The inspect guide covers accepted formats and field mappings.

Source and reproduction: released synthetic spans and OTLP capture generator. Verification record.

Readiness describes fields in the captured model attempts; zeros without declared usage origin remain unknown.
Measurement notes