Which model actually served your LiteLLM route?

fleetdiff reads your LiteLLM spend logs and shows which models actually served each route, how that mix changed between the last two weeks, and what the router's own classifier calls cost.

Audit your LiteLLM database

1. Install. Linux or macOS; needs curl, gh, shasum and tar. The installer checks the release before unpacking it.

curl -fsSLo install-fleetdiff.sh https://raw.githubusercontent.com/llm-measurement/fleetdiff/6089061c931fa19fac876e40b3809fdea4792ab8/scripts/install.sh
sh install-fleetdiff.sh v0.12.0 ./bin

2. Run it with the connection file from the LiteLLM spend guide:

./bin/fleetdiff investigate --litellm-db --database-url-file ~/.config/litellm-db-url --routing-audit

For each original route, you get each model's share of requests, tokens and recorded spend in both weeks, the classifier's calls and cost on a separate line, and the requests with no recorded agent or route. Add --routing-by route-agent to split each route by agent.

Look at one route or agent

Set BEFORE and AFTER to comparison dates and ROUTE to your route.

./bin/fleetdiff investigate --litellm-db \
  --database-url-file ~/.config/litellm-db-url \
  --before-period "$BEFORE" \
  --after-period "$AFTER" \
  --scope-route "$ROUTE" --routing-audit

Use --scope-agent for one agent instead. Reports show aliases. Add --show-labels for a local copy with the real names.

Split routes by agent as JSON

Set BEFORE and AFTER to comparison dates.

./bin/fleetdiff investigate --litellm-db \
  --database-url-file ~/.config/litellm-db-url \
  --before-period "$BEFORE" \
  --after-period "$AFTER" \
  --routing-audit --routing-by route-agent --format json

Already have an export file?

Set BEFORE and AFTER to comparison dates.

./bin/fleetdiff investigate --litellm-spend ./spend.csv \
  --before-period "$BEFORE" \
  --after-period "$AFTER" \
  --routing-audit

Export with psql instead

Set BEFORE and AFTER to RFC3339 UTC bounds covering both periods.

set -eu
umask 077
# Set BEFORE and AFTER to RFC3339 UTC bounds covering both periods.
# AFTER is exclusive; for example 2026-10-01T00:00:00Z to 2026-10-03T00:00:00Z.
curl -fsSLo export.sql https://raw.githubusercontent.com/llm-measurement/fleetdiff/v0.12.0/examples/litellm-spend/export.sql
mkdir ./routing-export
psql 'service=litellm_readonly' -Xq -v ON_ERROR_STOP=1 \
  -v start_utc="$BEFORE" \
  -v end_utc="$AFTER" \
  -v routing_details=true -v csv_file='./routing-export/spend.csv' -f export.sql \
  >./routing-export/private-psql.out 2>./routing-export/private-psql.err
printf 'Exported routing details to ./routing-export/spend.csv\n'

Set BEFORE and AFTER to comparison dates.

./bin/fleetdiff investigate --litellm-spend ./routing-export/spend.csv \
  --before-period "$BEFORE" \
  --after-period "$AFTER" \
  --routing-audit

Security and access

fleetdiff reads one read-only snapshot. It never writes to your database, and it doesn't read prompts or responses.
Remote databases must use sslmode=verify-full. Only a local socket may skip TLS.
The connection URL comes from an owner-only file or the FLEETDIFF_LITELLM_DATABASE_URL environment variable, never from the command line.
Analysis happens on your machine. Nothing is sent anywhere else. Reports use aliases instead of raw keys.

Full details: security review.

Try the sample

Linux or macOS; needs curl, gh, shasum and tar. The installer checks the release before unpacking it.

set -eu
umask 077
mkdir fleetdiff-routing-sample
cd fleetdiff-routing-sample
curl --proto '=https' --tlsv1.2 -fsSLo install-fleetdiff.sh \
  https://raw.githubusercontent.com/llm-measurement/fleetdiff/6089061c931fa19fac876e40b3809fdea4792ab8/scripts/install.sh
sh install-fleetdiff.sh v0.12.0 ./release
curl --proto '=https' --tlsv1.2 -fsSLo spend.csv \
  https://raw.githubusercontent.com/llm-measurement/fleetdiff/v0.12.0/examples/routing/synthetic.csv
./release/fleetdiff investigate --litellm-spend spend.csv \
  --before-period 2026-10-01 --after-period 2026-10-02 --routing-audit

What you'll see

These excerpts are exact output from the released binary and this synthetic sample. Across both routes, model-request counts and recorded tokens stay the same.

  Model-request rows: 3 -> 3 requests; 360 -> 360 recorded tokens; recorded USD 0.0017 -> 0.0016.

The first route has two requests in each period. In the second period, one is recorded under Sonnet and one under Sol, so Sonnet's request share falls from all requests to half.

    claude-sonnet-5-5: requests 2 -> 1; tokens 240 -> 120; recorded USD 0.0012 -> 0.0006.
      Cohort shares: requests 100% -> 50%; tokens 100% -> 50%; known recorded spend 100% -> 54.55%.
    gpt-6.1-sol: requests 0 -> 1; tokens 0 -> 120; recorded USD 0 -> 0.0005.
      Cohort shares: requests 0% -> 50%; tokens 0% -> 50%; known recorded spend 0% -> 45.45%.

Classifier calls are counted separately from model requests. Recorded spend comes from the source, not a recalculated bill.

  Classifier routing overhead: 1 -> 1 rows; raw recorded total tokens 21 -> 21; recorded USD 0.00005 -> 0.00005.

Coverage is broken down by call type. Here the Responses rows have no recorded agent, so they cannot be assigned to a known agent.

    aresponses:
      agent known: 0 -> 0; unknown: 1 -> 1 (column absent 0 -> 0; null/empty 1 -> 1; invalid 0 -> 0).

Source and reproduction: eight-row synthetic CSV and released routing guide.

What the numbers mean

The model shown is the one the gateway recorded, not a confirmation from the provider.
A change in the model mix shows what changed, not why.
Requests with an unknown model stay in the totals.
Only rows LiteLLM marks as classifier calls count as classifier cost.
Measurement notes