Budgets
A budget is a dollar or token cap that the router checks before it contacts an upstream, and reconciles against real usage afterwards. When a cap binds, the caller gets a 402 and no provider is ever called, so the money you did not authorize is money you never spend.
What a cap actually promises
Every dispatched leg passes through a ledger operation called a reservation. The router estimates what the leg will cost, and admits it only if this holds, atomically, for every scope the leg touches:
spent + reserved + estimate <= cap
That invariant is what makes concurrent requests unable to each individually "fit" and collectively blow past a cap: an in-flight leg's estimate occupies the cap for as long as it is in flight, and is replaced by the real cost once the leg settles.
Warning
A cap is not a hard ceiling on realized spend. It is a hard gate on
new dispatch. Overshoot is bounded by the sum, over in-flight legs, of
max(0, actualCost - reservedEstimate). That is nonzero whenever true
tokenization exceeds the estimator's heuristic (dense code and CJK text can approach one
token per character), whenever completionReserveFraction is below
1.0, or whenever a configured price is simply wrong. Size your cap with
that headroom in mind rather than treating it as a fuse.
Note
The whole budget block is optional. Omit it and the router does no
accounting at all: no caps, no estimation, no 402, and GET /budget reports
an empty scope list.
Scopes
Caps are declared at three scopes, each of which becomes a ledger key. All three are independent accounts; a leg reserves against all of them at once.
| Config | Scope key | Bounds |
|---|---|---|
| budget.global | global | The whole process, all traffic. |
| budget.perRoute.<name> | route:<name> | Everything matched by one named route. |
| budget.perUpstream.<name> | upstream:<name> | Everything dispatched to one upstream, from any route. |
The reserve step walks the scopes in a fixed order (global, then
route:<routeName>, then upstream:<legUpstream>), and
a refusal names the first binding scope in that order. Scopes with no
configured cap are skipped entirely, so a deployment that only sets
budget.global pays for exactly one account lookup per leg.
Reporting uses a deliberately narrower precedence. The
x-switchback-budget-remaining header and the budget block of a
routing report pick a single binding scope: route, then upstream, then global,
taking the first of those that has a configured cap. So a request can be gated by
global while its header reports headroom on the tighter
route: account. Read GET /budget when you want all scopes at
once.
Windows
Every cap has a window, and every boundary is UTC. There are five, not three, and the
default when window is omitted is monthly.
| Window | Rolls at | Notes |
|---|---|---|
| hourly | Top of the UTC hour | |
| daily | UTC midnight | |
| weekly | UTC midnight, Monday-anchored | Not "seven days ago"; a fixed calendar week. |
| monthly | First of the UTC month | The default. |
| total | Never |
A non-rolling lifetime cap anchored at the epoch. Its resetsAt is
reported as 9999-12-31T23:59:59.999Z.
|
What a boundary does
Nothing is prorated and nothing carries over. When a scope's window rolls, that scope's
spend and its outstanding reservations are both reset to zero and the new
window starts clean. A request refused at 23:59:59 UTC under a daily cap
succeeds a second later, at the full cap, with no back-pressure and no queue. There is no
soft landing as a window fills, either: legs are admitted at full speed until one no
longer fits, and then that one is refused.
Because the boundary discards in-flight reservations, a leg that spans a boundary is committed against the new window when it settles. Windows shorter than your slowest request will therefore attribute a little spend to the window after the one that authorized it.
Warning
A total window genuinely never resets, including across restarts, provided
the backend persists its state. On the default in-memory backend with no
persistPath, a restart silently returns a lifetime cap to zero spent. If
you mean "lifetime", pair total with a persistent backend.
Cap kinds: dollars, tokens, or both
A cap object is { dollars?, tokens?, window? }. You may set either dimension,
both, or neither. Both dimensions are enforced independently, so whichever runs out first
is the one that refuses the request.
A fixed-point decimal string, never a JSON number. Money is never a float
anywhere in the router: dollars are parsed at the config boundary and carried
internally as micro-dollar (1e-6) integers. The grammar is digits with an optional
decimal point and at most six fractional digits. A leading
-, a non-numeric string, "Infinity", "NaN",
scientific notation such as "1e-6", or a seventh fractional digit are
each rejected at load, so a malformed cap fails startup instead of silently becoming
zero or infinity.
A non-negative integer counting prompt plus completion tokens together. A token cap needs no pricing at all, which makes it the enforcement you can always have, including for models the price catalog has never heard of.
One of hourly, daily, weekly,
monthly, total. Anything else is refused at load.
Reserve, then commit or release
The ledger exposes four operations (reserve, commit,
release, read), and the router uses them in that shape around
every leg.
Reserve is two-phase, and side-effect-free on refusal
Phase one evaluates the invariant for every applicable scope against a read-only view of the ledger. If any scope fails, nothing is mutated (no partial reservation is left behind on the scopes that would have fit) and a structured refusal is returned. Phase two runs only after every scope passes, materializing the accounts and recording the reservation under each. There is no window in which a refused request has reserved half a cap.
The refusal carries exactly these fields, and they are what the 402 body is built from:
| Field | Meaning |
|---|---|
| scope | First binding scope key. |
| window | That scope's window. |
| limitKind | Which dimension bound: "dollars" or "tokens". |
| limit | The configured cap, echoed verbatim. |
| spent | Spend plus outstanding reservations at the moment of refusal. |
| resetsAt | ISO-8601 UTC end of the window. |
Reservations expire
A reservation has a TTL of 60 000 ms. Expired reservations are pruned on
every ledger operation, and the in-memory backend additionally sweeps on a timer (60 s by
default), so a crashed or abandoned request cannot hold a cap hostage forever. If a
commit arrives for a reservation that was already pruned, it degrades
gracefully: the real cost is still booked, as direct spend rather than as the settlement
of a reservation. Nothing is lost from the ledger, but a request slower than the TTL does
briefly stop occupying the cap, which is one of the ways two very slow concurrent legs can
overshoot.
Which failures still cost money
After the leg finishes, its verdict selects a settlement. The rule is deliberately conservative: a leg the provider plausibly executed is charged, because a retry storm must not multiply unaccounted spend. Only failures that provably never reached a model release their reservation.
| Leg verdict | Settlement | Charged? |
|---|---|---|
| success | commit actuals (reported usage, priced) |
Yes |
| success |
commit relayed_tokens when the stream ended abnormally after commit
and carried no usage frame. Never a plain release after commit.
|
Yes |
| rate_limited | release | No |
| auth_budget | release | No |
| connect_error | release | No |
| timeout_connect | release | No |
| server_error |
commit: error_usage if the error body carried usage, else
prompt_estimate
|
Yes |
| silent_empty | commit, same rule | Yes |
| timeout_ttft | commit, same rule | Yes |
| stream_error_pre_token | commit, same rule | Yes |
| translated_bad_request | release if the leg was dispatched, else none |
Only if dispatched |
| bad_request | release | No |
| content_policy | commit, same rule as server_error |
Yes |
| budget_exhausted |
none: the refusal happened at reserve time, so there is nothing to
settle
|
No |
| no_candidates | none |
No |
The settlement chosen for each leg is visible per-leg in the routing report as
budget: { settlement, amount }, using the labels released,
committed_actuals, committed_prompt_estimate,
committed_error_usage, committed_relayed_tokens and
none. See Observability for the full report
shape, and Routing for where those verdicts come from.
Note
At settle time the router prefers the upstream's own reported cost when it is a
non-negative finite number; failing that it prices the reported token counts from the
catalog or your overlay; failing that it keeps the estimate. The chosen source is
stamped on the report as cost: { amount, source }, so you can tell a
provider-reported figure from a locally priced one.
How a leg is estimated
A reservation needs a number before a single token exists, so the router estimates one. Two different precision regimes meet here on purpose: token counts are a heuristic, widened by a safety factor; money is exact, computed in fixed-point integers and narrowed to micro-dollars exactly once.
estimate = promptEstimate × promptPrice
+ imageCount × imagePrice
+ completionCeiling × completionPrice
promptEstimate = ceil(ceil(serializedChars / 4) × promptSafetyFactor)
completionCeiling = ceil((max_tokens ?? 4096) × completionReserveFraction)
| Constant | Default |
|---|---|
| promptSafetyFactor | 2.0 |
| completionReserveFraction | 1.0 |
| catalog max completion fallback | 4096 |
| per-image price | "0.001275" |
serializedChars counts everything the upstream will tokenize, not just user
text: all message content including the system message, each message's
name and tool_call_id, the serialized
tool_calls history, and the serialized tools schemas. Image
parts are excluded from the character count and priced per image instead. Caller-supplied
numbers are clamped before they reach the money math: a missing or non-finite
max_tokens falls back to the catalog default, a finite negative floors to
zero, and token counts coerce to non-negative integers.
Warning
promptSafetyFactor and completionReserveFraction are the two
knobs that decide how much a cap can be overshot. Lowering either buys you throughput
near a full cap and gives back precision: at
completionReserveFraction: 0.25 the router reserves a quarter of the
requested max_tokens, so four maximally verbose concurrent completions can
settle for more than was ever reserved. Raise promptSafetyFactor instead if
your traffic is dense code or non-Latin script, where chars / 4
under-counts badly.
For /v1/embeddings, completionReserveFraction is forced to
0. There is no completion to reserve, so estimation is prompt-side only.
Pricing: the catalog and the priceMap overlay
A dollar cap can only bind if the leg has a price. Prices come from two places: a signed
catalog artifact bundled with the router, and your own priceMap overlay,
which always wins.
The catalog is verified as a detached Ed25519 signature before it is parsed, at
every load, so a tampered catalog never partial-loads. Pricing is a money surface and is
treated as one. Catalog rows are keyed canonically, like openai/gpt-4o, with
aliases such as gpt-4o; both resolve, and canonical keys win over aliases.
Danger
priceMap keys are <upstreamModelId>@<upstream>:
the id the upstream knows, not the slug your clients send.
A key that does not match produces no error and no warning: the overlay
is silently ignored, the leg falls through to the catalog, and your price override
quietly never applies. If a dollar cap is configured, that near-miss key is the
difference between a free local model and a leg refused as unpriceable. Where the
catalog does have a row, it is the difference between your negotiated rate and a list
price. Check every key you add against the right-hand side of your
models mapping.
{
"models": {
// clients ask for this slug …
"llama-3.3-70b": {
// … and this upstream calls it this.
"ollama-local": "llama3.3:70b"
}
},
"budget": {
"priceMap": {
// RIGHT: upstreamModelId@upstream
"llama3.3:70b@ollama-local": { "prompt": "0", "completion": "0" },
// WRONG: the client slug. Never matches. Never warns.
"llama-3.3-70b@ollama-local": { "prompt": "0", "completion": "0" }
}
}
}
An overlay entry accepts prompt, completion, and optionally
cache_read and image. All of them are plain non-negative decimal
strings under the same grammar as a dollar cap; scientific notation is rejected at load
with an explicit message.
A malformed price literal is never treated as $0. It is either refused at
load or leaves the model unpriced.
The router never guesses a price
If a leg has no price, and a dollar cap applies to it, and
allowUnpriced is false (the default), the leg is refused with
budget_unpriceable, and the error names the two remedies: set
allowUnpriced: true, or add a priceMap entry. With no dollar
cap, or with allowUnpriced: true, an unpriced leg proceeds normally and any
token cap still applies to it.
Danger
The bundled catalog prices only the models it lists. A daily job
rebuilds it from OpenRouter and vendor price pages and opens a reviewed pull request, so
your release prices the models in its last merged refresh. A model listed at $0 prompt
and $0 completion counts as unpriced unless its id ends in :free. With a
dollar cap configured and allowUnpriced left at false,
every unpriced model refuses at reserve time with
budget_unpriceable until you give it a priceMap entry. This is
the single biggest practical limit of budgets today. Plan for it: either price every
model you route to, or lean on tokens caps, which need no prices at all.
Setting allowUnpriced: true to make the errors go away means unpriced legs
draw down nothing from your dollar cap. The cap stops binding for exactly the
traffic you could not price.
A budget block you can paste
This is the whole feature surface in one object, nested under the same root as the rest of your configuration.
{
"budget": {
"backend": { "kind": "memory", "persistPath": "./.switchback/budget.json" },
"global": { "dollars": "500.00", "tokens": 50000000, "window": "monthly" },
"perRoute": { "chat-default": { "dollars": "50.00", "window": "daily" } },
"perUpstream": { "openrouter-fallback": { "dollars": "25.00", "window": "monthly" } },
"priceMap": {
"llama3.3:70b@ollama-local": { "prompt": "0", "completion": "0" }
},
"promptSafetyFactor": 2.0,
"completionReserveFraction": 1.0,
"allowUnpriced": false,
"overflowFallback": false,
"onStoreError": "fail-closed"
}
}
Note
On SIGHUP the config file is re-read and budget caps, the
priceMap overlay and the cap gauges are rebuilt in place. The budget
backend is not: changing backend.kind, its url or its
persistPath logs a loud "restart required" warning and keeps running the
old store. Cap changes are hot; topology changes are not.
Backends
The ledger is pluggable because the correct answer depends entirely on how many processes share a cap. Pick by topology, not by preference.
| backend.kind | Use when | What you are trading |
|---|---|---|
| memory default | One process owns the cap. |
Atomicity is free: the critical section is single-threaded with no
await in it. Optional persistPath snapshot, written
every 30 000 ms and again on shutdown, so a crash can lose up to that window of
spend.
|
| file | Several processes on one host share a cap. |
Correct but low-throughput: an advisory lockfile plus read-modify-write plus a
temp-file rename on every operation. Lock timeout 5 000 ms, stale-lock steal after
10 000 ms. persistPath is required; the CLI refuses
to start without it.
|
| redis | A fleet of replicas shares one cap. |
The only fleet-shared component in the router. reserve,
commit and release are each a single Lua script, money
crosses the wire as micro-dollar integers, and reservation expiry is enforced
inside the script rather than by key TTL.
|
A per-replica memory backend in a fleet does not enforce your cap; it
enforces N copies of your cap. That is the one substitution that silently
multiplies spend, and it is why Deploy treats the budget backend as
a topology decision.
State files are refused if they are readable by anyone else
A budget snapshot records your spend, so both the memory and
file backends harden it. A state file that is group- or other-accessible
(mode & 0o077) is refused at load, with the offending
octal mode in the message. A corrupt or wrong-shaped snapshot also refuses to boot rather
than silently starting from a zeroed ledger. A budget store that quietly forgets its spend
is worse than one that will not start. Snapshots are written 0600 with an
explicit chmod.
Redis specifics
Keys are prefixed {switchback-budget}. The braces are a Cluster hash tag, so
every scope key lands in one slot.
Cluster MOVED and ASK redirects are not followed, so supported topologies are standalone, Sentinel behind a proxy, or Cluster behind a
single-endpoint proxy. The connection string goes in backend.url as a
redis:// or rediss:// URL; keep a real credential out of any
config file you commit.
Danger
onStoreError: "fail-open" means that when the store is unreachable,
caps are not enforced and requests keep flowing. It is loud about it:
the log line reads [switchback][budget] fail-open: cap NOT enforced. But
/ready still answers ready, with "budgetStore":"degraded". The
default, "fail-closed", instead refuses traffic and takes
/ready to 503 so your orchestrator stops sending it work. Choose
deliberately: fail-open trades a cap for availability, and a Redis outage during a
runaway workload is exactly when the cap mattered.
At the cap: the 402
Refusal happens at reserve time, which is
before any upstream is contacted. The status is always
402 Payment Required for budget_exhausted, on both the native
and the OpenRouter-shaped surface.
HTTP/1.1 402 Payment Required
content-type: application/json
x-switchback-budget-remaining: global=0.000001
{
"error": {
"message": "budget cap reached for scope \"global\" (monthly); limit 0.000001, spent 0.000000, resets 2026-08-01T00:00:00.000Z",
"type": "budget_exhausted",
"code": 402,
"metadata": {
"reason": "budget_exhausted",
"scope": "global",
"window": "monthly",
"limit_kind": "dollars",
"limit": "0.000001",
"spent": "0.000000",
"remaining": "0.000001",
"resets_at": "2026-08-01T00:00:00.000Z"
}
}
}
The same request against the /api/v1 surface returns the OpenRouter-shaped
error envelope carrying the identical metadata object, so a client
written against either shape can branch on metadata.reason and back off until
metadata.resets_at. The metadata keys are exactly reason,
scope, window, limit_kind, limit,
spent, remaining, resets_at.
Note that spent is spend plus outstanding reservations, which is why
a 402 can report a spent lower than the limit: the difference is
in-flight work that has already claimed the remaining headroom.
Warning
An in-flight stream is never cut off when a cap is crossed mid-stream.
The enforcement point is reserve, before dispatch, so caps gate new
dispatch only. A completion that was admitted runs to its end and is charged in full.
Likewise, listing budget_exhausted in a route's advanceOn does
nothing: it is ignored there, and governed by overflowFallback instead.
overflowFallback
By default a budget refusal is terminal: the 402 surfaces immediately and the chain stops. That is usually what you want, because the alternative spends money you were trying not to spend.
Set overflowFallback: true and the engine instead advances to the
next leg, whose own estimate is checked against whatever headroom remains. A leg reached
this way is annotated budget_overflow_fallback: true in the report and the
response carries x-switchback-budget-overflow: true. If nothing in the chain
fits, the last refusal surfaces as the 402. This is how you express "when the expensive
upstream's per-upstream cap is spent, fall through to the cheap one": the caps do the
routing. It only helps if the next leg is cheaper or sits under a different scope; if the
binding scope is global, no leg can fit, and the overflow just walks the
chain to the same refusal.
The remaining-budget header
Successful responses can carry x-switchback-budget-remaining, formatted
<scope>=<amount> in six-decimal fixed point, for example
x-switchback-budget-remaining: global=4.999977. It is a convenience for
clients that want to self-throttle, and it is honest about its own limits:
- It is omitted entirely when the binding scope has no dollar cap. There is no sentinel value and it never reports spend as though it were remaining. Absence means unlimited, not zero.
- It is best-effort and bounded by a 2 000 ms store read timeout. A hung store makes the header disappear rather than withhold a completion you have already paid for.
-
On streaming responses it is the pre-dispatch value, read before the
relay begins, because the winning leg's per-upstream scope is not known until commit. A
stream can therefore report
global=5.000000and then spend money. The settled figure lands in the routing report'sbudgetblock. - Under keepalive mode the response head commits before those headers can be set, so attribution rides an in-stream object on the final usage-bearing chunk instead.
Reading current spend
GET /budget is the authoritative view: read straight from the store,
credential-scrubbed on the way out, listing only the scopes your config declares, ordered
global then upstream:* then route:*.
$ curl -s http://127.0.0.1:4011/budget
{
"budgetBackend": "memory",
"scopes": [
{
"scope": "global",
"window": "monthly",
"windowStart": "2026-07-01T00:00:00.000Z",
"spentTokens": 0,
"spentDollars": "0.000000",
"reservedTokens": 0,
"reservedDollars": "0.000000",
"limit": { "dollars": "5.00" },
"remaining": { "dollars": "5.000000" }
}
]
}
remaining.dollars is limit − (spent + reserved), so it is what a
new request would actually be measured against rather than a pure spend figure. A store
read failure answers 503 budget store is unavailable, never a raw 500. An
uncapped deployment answers with an empty scopes array.
For scraping, /metrics exposes switchback_budget_cap and
switchback_budget_spent gauges, each labelled with scope and
nothing else. They emit only when a cap lookup is available and the binding scope's
remaining is non-null, and they do correctly report a scope that has gone over its cap.
The per-request settlement trail lives in Observability.
Known limits
Everything here is current behavior, not a roadmap. Read it before you rely on a cap for anything that matters.
-
Two priced models out of the box. Under a dollar cap with
allowUnpriced: false, anything outside the bundled catalog needs apriceMapentry or it is refused. -
A wrong
priceMapkey fails silently. The key is<upstreamModelId>@<upstream>. There is no warning for a near-miss. - Caps bound new dispatch, not realized spend. Overshoot is bounded, not zero.
-
A fail-open store means no enforcement. Loud in logs,
degradedon/ready, but not enforcing. -
Two deferred defects worth knowing. A malformed numeric field read back
from Redis throws past
onStoreError(producing a 500 even in fail-open mode), and the file-store's lock rename-back has a narrow re-clobber race. Both are tracked as known hardening work. -
/budgetonly shows declared scopes. An empty list means no caps are configured, not that nothing has been spent.
Next: the budget block field-by-field in
Configuration, which backend suits your process topology in
Deploy, and where settlements, gauges and reports surface in
Observability.