What would switching models have cost?

Re-price recorded usage under a different model mix, with explicit cache and workload assumptions.

Run it on your data

Set up a LiteLLM read-only login and connection file.

Linux or macOS; needs curl, gh, shasum and tar. The installer checks the release before unpacking it.

umask 077
curl --proto '=https' --tlsv1.2 -fsSLo install-fleetdiff.sh \
  https://raw.githubusercontent.com/llm-measurement/fleetdiff/6089061c931fa19fac876e40b3809fdea4792ab8/scripts/install.sh
sh install-fleetdiff.sh v0.12.0 ./bin

Set PERIOD to a completed date, such as 2026-10-07. Choose models from the price table.

./bin/fleetdiff what-if --litellm-db --database-url-file ~/.config/litellm-db-url \
  --period "$PERIOD" \
  --from claude-sonnet-5-5 --to gpt-6.1-sol --fraction 0.5

0.5 moves half the covered token volume.

Use an existing export

Set PERIOD to a completed UTC date and use spend.csv from the export recipe.

./bin/fleetdiff what-if --litellm-spend ./spend.csv \
  --period "$PERIOD" \
  --from claude-sonnet-5-5 --to gpt-6.1-sol --fraction 0.5

Use your workload mix

Assign keys or teams to QA, extraction or coding.

Add a workload map

Save this as workload-map.json, replacing the example IDs with those in your export.

{
  "version": 1,
  "mappings": [
    {"selector": {"kind": "key", "value": "your-coding-key-id"}, "workload": "coding"},
    {"selector": {"kind": "team", "value": "your-extraction-team-id"}, "workload": "extraction"}
  ]
}

Set PERIOD to a completed UTC date.

./bin/fleetdiff what-if --litellm-spend ./spend.csv \
  --period "$PERIOD" \
  --from claude-sonnet-5-5 --to gpt-6.1-sol --fraction 0.5 \
  --workload-map ./workload-map.json --format json

Mapped records use their assigned workload; unmapped records remain under all three measured assumptions. Conflicting key/team mappings are rejected. A model pair without a measured output factor stays at level 1; no reverse or transitive factor is invented.

Security

See the connection reference.

Try the sample

Linux or macOS; needs curl, gh, shasum and tar. The installer checks the release before unpacking it.

set -eu
umask 077
mkdir fleetdiff-what-if-sample
cd fleetdiff-what-if-sample
curl --proto '=https' --tlsv1.2 -fsSLo install-fleetdiff.sh \
  https://raw.githubusercontent.com/llm-measurement/fleetdiff/6089061c931fa19fac876e40b3809fdea4792ab8/scripts/install.sh
sh install-fleetdiff.sh v0.12.0 ./release
curl --proto '=https' --tlsv1.2 -fsSLo spend.csv \
  https://raw.githubusercontent.com/llm-measurement/fleetdiff/v0.12.0/examples/what-if/spend.csv
./release/fleetdiff what-if --litellm-spend spend.csv \
  --period 2026-10-07 --from claude-sonnet-5-5 --to gpt-6.1-sol --fraction 0.5

What you'll see

Checked synthetic example: moving half of the covered claude-sonnet-5-5 usage to gpt-6.1-sol reprices the period from $0.154400 to $0.109398 to $0.147974, depending on workload assumptions.

This is exact output from the released binary and this synthetic sample. The amounts cover the whole priced period, including traffic that does not move.

Moving 50% of claude-sonnet-5-5 traffic to gpt-6.1-sol (covered usage): $0.154400 -> $0.109398 to $0.147974, depending on workload assumptions.
Period: Wed 2026-10-07 00:00:00 UTC -> Thu 2026-10-08 00:00:00 UTC (end exclusive). Prices: 2026-10-14 (bundled); measured factors: 2026-10-14.

Level 1, same work and output verbosity: $0.148703
Level 2, qa: $0.112531 (propagated mean-interval endpoints $0.109398 to $0.116140)
Level 2, extraction: $0.122062 (propagated mean-interval endpoints $0.117196 to $0.127759)
Level 2, coding: $0.140323 (propagated mean-interval endpoints $0.133659 to $0.147974)

Level 1 keeps output tokens unchanged, applies the measured input tokenizer factor and preserves cache shares. Level 2 also applies the measured output-verbosity factor for QA, extraction or coding. Without a workload map, the report shows all three assumptions rather than choosing one for you.

move-1: 50% of claude-sonnet-5-5 -> gpt-6.1-sol
  Moved traffic, level 1: $0.077200 -> $0.071503; input tokenizer factor 0.668804955
  Output share of moved target cost at level 1: 83.91%

The move detail separates the moved portion from the whole period. Here, output is 83.91% of that portion's target cost at level 1, so the output-verbosity assumption affects most of its cost.

Coverage: 3 observations priced; missing splits are excluded, never zero.
  missing split: 1
  3 moved observations have no workload mapping; all three workload assumptions are shown where measured.

Three sample records have complete component splits. The fourth lacks cache details and is excluded, not priced as zero. Source and proposed costs both use the dated price table, not the export's recorded spend.

Source and reproduction: released synthetic CSV, full checked text output and JSON output.

What the numbers mean

This re-prices recorded usage under explicit model, cache and workload assumptions. It is not a forecast or an invoice.
The factors are geometric-mean paired ratios measured on public benchmark tasks; your prompts, workload and model behavior may differ.
Level 2 endpoints show the uncertainty of the measured average output-verbosity factor, not a range for your traffic. In a fresh calibration on 20 batches of 30 tasks, they contained the measured cost of the moved traffic in 19 of 20 batches for Sol to Luna QA and 17 of 20 for Sol to Luna extraction, but in only 2 to 14 of 20 for the other six completed cases. Coding factors fit on HumanEval did not transfer to MBPP, and an earlier per-request test also failed. See the calibration.
No quality equivalence is claimed.

The released benchmark evidence records model IDs, fit counts, dates, dataset and license pins, estimator and price sources; the factor table links each measured pair to that provenance.

This synthetic example checks repricing arithmetic for recorded model attempts with complete token splits, not completed-task costs.
Measurement notes