switchback — part of the aimock family private · pre-launch

Observability

Four GET endpoints, five response headers, one per-request routing report, and one CLI summarizer. Between them they answer the two questions you actually have at 3am: which upstreams did this request try, and where did the dollar go.

The operator surface, in one table

All of these are served by the router's own HTTP frontend, on the same port as /v1/chat/completions. There is no separate admin listener and no second port to open. Every example on this page assumes the router is on 127.0.0.1:4011.

Endpoint Returns Use it for
GET /health 200 {"status":"ok"} Static liveness. It does not touch the budget store.
GET /ready 200 with budgetBackend, or 503 Readiness that reflects the budget store. See below for the full matrix.
GET /metrics Prometheus text exposition, content-type: text/plain; version=0.0.4 Counters and gauges for the whole process. Scrape target.
GET /budget Per-scope spend snapshot, JSON Current spend and remaining headroom against each declared cap.
GET /route/reports {"reports":[…]}, newest first The per-request decision trace. This is the incident endpoint.
GET /route/reports/<id> One report, or a structured 404 Pulling up the exact request a client complained about, by the id in its report.

Anything not on this list (and not one of the routed endpoints) is a structured 404 from the router itself. An unknown path is never proxied upstream, so you cannot accidentally turn this process into an open relay by guessing a URL.

the whole surface, by hand shell
# liveness and readiness
curl -s http://127.0.0.1:4011/health
curl -s http://127.0.0.1:4011/ready

# metrics, spend, and the last N routing decisions
curl -s http://127.0.0.1:4011/metrics
curl -s http://127.0.0.1:4011/budget
curl -s http://127.0.0.1:4011/route/reports

# one specific decision, by the id from a report
curl -s http://127.0.0.1:4011/route/reports/rt-e0d276c4f3ee

Note

The router performs no authentication of its own, on these endpoints or any other. If the port is reachable, so is your spend data. The deployment contract is loopback or a private network behind your own ingress. See Security and Deploy.

Response headers: the decision, attached to the answer

Every chat and embeddings response carries what the router decided, so a client that captured a response already holds most of the story without a second call. All header names are lower-case.

Header When it is set Value
x-switchback-upstream A leg committed The winning upstream's name, as you declared it in config.
x-switchback-dialect A leg committed The winning leg's wire dialect.
x-switchback-attempts A leg committed The sum of per-leg attempts across the whole request. Not the number of legs, and not attempts on the winner alone.
x-switchback-budget-overflow Any leg was reached via a budget-overflow advance "true"
x-switchback-budget-remaining The binding scope has a dollar cap <scope>=<amount>, six-decimal fixed point. Omitted entirely when the binding scope is uncapped.

A verified non-streaming 200:

response headers http
x-switchback-upstream: stub
x-switchback-dialect: openai-compat
x-switchback-attempts: 1
x-switchback-budget-remaining: global=4.999977

x-switchback-attempts: 1 with no x-switchback-budget-overflow is the shortest possible story: one leg, one attempt, no overflow advance, and the binding scope still has headroom.

Streaming, and the keepalive exception

On a stream the head is deferred (nothing is written until a leg commits), so the same attribution headers are present on a streamed response. Verified:

streamed response head http
content-type: text/event-stream; charset=utf-8
cache-control: no-cache
connection: keep-alive
x-switchback-upstream: stub
x-switchback-dialect: openai-compat
x-switchback-attempts: 1
x-switchback-budget-remaining: global=5.000000

Two caveats that will cost you an hour if you do not know them. First, under keepalive mode the head commits early, before a leg is chosen, so those headers cannot be set at all; attribution rides an in-stream x_switchback object on the final usage-bearing chunk instead, carrying upstream, dialect, translated and attempts. Second, on any stream x-switchback-budget-remaining is the pre-dispatch value: it is read before the relay starts, because the winner's per-upstream scope is not known until commit. The example above reads global=5.000000 on a stream that then went on to spend money. The settled figure exists only in the routing report.

Prometheus metrics at /metrics

GET /metrics returns Prometheus text exposition with content-type: text/plain; version=0.0.4. The scrape-time gauges are recomputed immediately before serialization, so a gauge is never stale by a scrape interval. If no metrics registry is wired the endpoint honestly answers 501 rather than pretending to be empty, but the CLI always wires one, so a served router always has metrics.

A verified scrape, exact names and labels:

GET /metrics prometheus
# TYPE switchback_leg_attempts_total counter
switchback_leg_attempts_total{reason="",upstream="stub",verdict="success"} 2
# TYPE switchback_requests_total counter
switchback_requests_total{outcome="success"} 2
# TYPE switchback_budget_cap gauge
switchback_budget_cap{scope="global"} 5
# TYPE switchback_budget_spent gauge
switchback_budget_spent{scope="global"} 0.000046
# TYPE switchback_catalog_age_seconds gauge
switchback_catalog_age_seconds{} 215741
# TYPE switchback_holdback_bytes gauge
switchback_holdback_bytes{} 65536

Every series this router exports

This is the complete list. If a series is not here, this router does not export it, whatever an older document told you.

Series Type Labels Notes
switchback_leg_attempts_total counter upstream, verdict, reason verdict is success, retryable or fatal; reason is empty on success. This is the failover series.
switchback_requests_total counter outcome outcome is one of success, error, aborted. Per request, not per leg.
switchback_ttft_ms histogram upstream Observed only when a leg reports a time-to-first-token, i.e. streaming legs. absent on non-streaming workloads
switchback_budget_spent gauge scope scope only. There is no kind label.
switchback_budget_cap gauge scope scope only. Pair it with the series above.
switchback_catalog_age_seconds gauge none Age of the loaded price catalog.
switchback_holdback_bytes gauge none A gauge, not a histogram. Default value 65536 (64 KiB).

The two budget gauges emit only when a cap lookup is available and the request's binding-scope remaining is non-null; spent is computed as cap − remaining in micro-dollars and exposed as a float. They do emit correctly for a scope that is over its cap, so a negative headroom is visible rather than clamped away.

Scraping it

The registry is minted per server instance. There is no global singleton and no aggregation across processes, so every replica is its own scrape target. A fleet scraped through a single load-balanced address will show you one arbitrary replica's counters per scrape, which is worse than useless during an incident.

one replica, right now shell
# everything
curl -s http://127.0.0.1:4011/metrics

# just the failover series and the budget gauges
curl -s http://127.0.0.1:4011/metrics | grep -E '^switchback_(leg_attempts_total|budget_)'
one target per process yaml
scrape_configs:
  - job_name: switchback
    metrics_path: /metrics
    static_configs:
      - targets:
          - "10.0.1.11:4011"
          - "10.0.1.12:4011"
        # list every replica individually: /metrics is per-process
three queries worth saving promql
# which upstreams are producing retryable legs, by reason
sum by (upstream, reason) (rate(switchback_leg_attempts_total{verdict="retryable"}[5m]))

# fraction of each declared cap consumed
switchback_budget_spent / switchback_budget_cap

# share of requests the router itself could not serve
sum(rate(switchback_requests_total{outcome="error"}[5m]))
  / sum(rate(switchback_requests_total[5m]))

What the metrics will not tell you

They are counters and gauges, and they are aggregate by construction. They will tell you that openai-primary started producing rate_limited legs eight minutes ago. They cannot tell you what one named request did. For that you need the routing report, next.

There is also no OpenTelemetry integration, no distributed tracing, and no log shipping in this router. The complete set of sinks is: stdout and stderr, an in-memory journal, the routing-report ring buffer, /metrics, and, optionally, recorded fixtures on disk. If you want traces or a dashboard, they are built on top of these, by you.

The routing report

One report per request, retained in a bounded FIFO of 1 000 reports; the newest evicts the oldest. Ids have the form rt- followed by twelve hex characters. The report is filled through an observer hook that runs on request completion, so a dropped or failed report can never affect a live request, and every report passes through the redaction chokepoint before it lands in the buffer rather than on the way out.

Report shape

id string

rt-<12 hex>. Use it against /route/reports/<id> while it is still in the buffer.

surface string

The endpoint path that was actually called, which distinguishes an OpenRouter-shaped /api/v1 caller from a plain /v1 one.

requested_model string

The model slug the client asked for, before any expansion.

candidates string[]

The candidate models the route expanded to, in order. Compare this against legs to see what was skipped without being dispatched.

legs object[]

One entry per leg the engine actually walked, in the order it walked them. Fields below.

winner object optional

{candidate, upstream, provider}. Absent when nothing committed: the chain was exhausted, or a fatal verdict surfaced.

budget object optional

{scope, remaining} for the binding scope. remaining is null when that scope has no dollar cap, and a six-decimal string amount otherwise. Unlike the header on a stream, this is the settled figure.

client object

{aborted, committed_at_ms?}. aborted: true means the caller hung up; if you are chasing phantom failures, check this before blaming an upstream.

Leg fields

Only set keys are emitted, which matters: an absent key is information. The full set is candidate, upstream, dialect, translated (only when true), verdict, status, retry_after_ms, attempts (only when > 1), upstream_error {message, status?, code?, metadata?}, connect_ms, ttft_ms, total_ms, dropped_params, clamped_params [{param, from, to}], tool_ids, unsupported_content, usage {prompt_tokens, completion_tokens}, budget {settlement, amount?}, budget_overflow_fallback (only when true), and cost {amount, source}.

verdict is a single string, either "success" or a slash form: "retryable/<reason>" or "fatal/<reason>". The reason vocabulary is the routing verdict taxonomy; Routing lists every name and what advances a chain. The settlement vocabulary is the budget one, and Budgets has the table of which failures still cost money.

One report, verified

The simplest possible trace: one candidate, one leg, committed on the first attempt. This is real output, not a sketch.

GET /route/reports → one element of reports[] json
{
  "id": "rt-e0d276c4f3ee",
  "surface": "/api/v1/chat/completions",
  "requested_model": "gpt-4o",
  "candidates": ["gpt-4o"],
  "legs": [{
    "candidate": "gpt-4o",
    "upstream": "stub",
    "dialect": "openai-compat",
    "verdict": "success",
    "status": 200,
    "connect_ms": 1,
    "usage": { "prompt_tokens": 5, "completion_tokens": 1 },
    "budget": { "settlement": "committed_actuals", "amount": "0.000023" },
    "cost": { "amount": "0.000023", "source": "catalog" }
  }],
  "client": { "aborted": false, "committed_at_ms": 1 },
  "winner": { "candidate": "gpt-4o", "upstream": "stub", "provider": "openai" },
  "budget": { "scope": "global", "remaining": "4.999954" }
}

Once an id ages out of the ring buffer, /route/reports/<id> answers a structured 404 reading no such routing report: <id>. That is the expected outcome for anything older than the last thousand requests on that process, not a bug.

Walkthrough: which legs were tried, and why

Here is the report you actually care about: a request that failed over. The shape is exactly as documented above; the upstream names and values are illustrative, because a real one from your incident is the only interesting one.

a three-leg report, illustrative values json
{
  "id": "rt-4b19c7a0de52",
  "surface": "/v1/chat/completions",
  "requested_model": "mux/fast",
  "candidates": ["gpt-4o", "claude-sonnet-4-5"],
  "legs": [
    {
      "candidate": "gpt-4o",
      "upstream": "openai-primary",
      "dialect": "openai-compat",
      "verdict": "retryable/rate_limited",
      "status": 429,
      "retry_after_ms": 45000,
      "budget": { "settlement": "released" }
    },
    {
      "candidate": "gpt-4o",
      "upstream": "openrouter-fallback",
      "dialect": "openai-compat",
      "verdict": "retryable/server_error",
      "status": 503,
      "attempts": 2,
      "budget": { "settlement": "committed_prompt_estimate", "amount": "0.004120" }
    },
    {
      "candidate": "claude-sonnet-4-5",
      "upstream": "anthropic-main",
      "dialect": "anthropic-messages",
      "translated": true,
      "verdict": "success",
      "status": 200,
      "dropped_params": ["logit_bias"],
      "usage": { "prompt_tokens": 812, "completion_tokens": 240 },
      "budget": { "settlement": "committed_actuals", "amount": "0.006036" },
      "cost": { "amount": "0.006036", "source": "catalog" }
    }
  ],
  "client": { "aborted": false },
  "winner": { "candidate": "claude-sonnet-4-5", "upstream": "anthropic-main", "provider": "anthropic" },
  "budget": { "scope": "route:chat-default", "remaining": "41.238904" }
}

Read it in this order.

  1. candidates versus legs. Two candidates, three legs, so one candidate was tried on two different upstreams. Going the other way, a candidate that appears in candidates with no corresponding leg was skipped without a dispatch: a no_route skip leaves no leg entry at all, which is why a report can look shorter than the chain you configured.
  2. Legs are in walk order. Leg 1 is the first thing tried, and the last leg is where it landed. winner just restates the last leg's identity plus the resolved provider.
  3. Each verdict says both severity and cause. retryable/rate_limited on leg 1 means the engine was allowed to move on; retry_after_ms: 45000 is the upstream's own hint. A Retry-After above the route's cap causes an immediate advance rather than a sleep, so a large value here plus an immediate next leg is the system working as configured.
  4. attempts: 2 on leg 2 is a same-leg retry. The key only appears when it is greater than one, so its absence on legs 1 and 3 means exactly one attempt each. Total attempts across the request is 4, which is what x-switchback-attempts: 4 on the response reported, even though only three legs exist. That mismatch is the single most common misreading of that header.
  5. translated: true on leg 3 means the request was rebuilt in another dialect: openai-compat in, anthropic-messages out. dropped_params tells you exactly what could not be expressed in that dialect and was left behind; if a client complains that behavior changed during a failover, this is the field that explains it. Providers has the per-dialect matrix.
  6. Now check what advanced the chain. A retryable reason only advances if it is listed in that route's advanceOn. A reason not listed surfaces to the client immediately, which is what you are looking at when a report has one leg and a retryable verdict. See Routing.

Two more fields earn their keep during an incident. client.aborted: true means the caller went away before commit, and the last real verdict surfaced. Nothing upstream was necessarily wrong. And a leg annotated budget_overflow_fallback: true was only reached because a budget refusal was allowed to advance the chain, which is the same event the x-switchback-budget-overflow header reports.

Walkthrough: where the dollar came from

Take the verified single-leg report above. It cost 0.000023. Four fields establish that, and each one answers a different objection.

Field Value What it settles
usage 5 prompt / 1 completion The token counts the upstream actually reported for this leg.
cost.source catalog Where the price came from. Cost prefers the upstream's own reported usage.cost when it is a non-negative finite number, then catalog or priceMap pricing times the reported tokens, and only then the pre-dispatch estimate.
budget.settlement committed_actuals How the reservation was resolved. released means that leg cost nothing; a committed_* value means it did, and the suffix says on what basis.
budget.remaining 4.999954 Post-settlement headroom on the binding scope, six-decimal fixed point.

The failover report above shows the part that surprises people: leg 2 cost money even though it failed. Its settlement is committed_prompt_estimate, because a 503 is a leg the provider plausibly ran, and a retry storm must not multiply unaccounted spend. Leg 1's 429, by contrast, is released. Sum the amount across every leg and you have the true cost of the request, which is strictly more than the winner's cost whenever failover happened. Budgets has the full verdict-to-settlement table.

Note

Three "remaining" numbers exist and they legitimately disagree. x-switchback-budget-remaining is limit − spent, and on a stream it is the pre-dispatch value. The report's budget.remaining is the settled figure. /budget's remaining.dollars is limit − (spent + reserved), so it is the strictest of the three and will read lower while requests are in flight.

Spend and liveness: /budget, /health, /ready

GET /budget is a per-scope snapshot read straight from the store and credential-scrubbed on the way out. Only scopes your config actually declares appear, in the order global, then upstream:*, then route:*. An uncapped deployment returns an empty scopes array rather than inventing rows. A store read failure answers 503 with budget store is unavailable, never a bare 500.

GET /budget json
{"budgetBackend":"memory","scopes":[{"scope":"global","window":"monthly",
"windowStart":"2026-07-01T00:00:00.000Z","spentTokens":0,"spentDollars":"0.000000",
"reservedTokens":0,"reservedDollars":"0.000000","limit":{"dollars":"5.00"},
"remaining":{"dollars":"5.000000"}}]}

/health is static liveness and always answers 200 {"status":"ok"}. /ready is the one that can say no, and it says it for exactly one reason, the budget store:

Condition Response
Backend is not redis 200 {"status":"ready","budgetBackend":"memory"} (always ready, echoing the backend)
Backend is redis, reachable 200 {"status":"ready","budgetBackend":"redis"}
Redis unreachable, onStoreError is fail-closed (the default) 503: budget store (redis) is unreachable — fail-closed, not ready
Redis unreachable, fail-open 200 {"status":"ready","budgetBackend":"redis","budgetStore":"degraded"} (ready, and caps are not being enforced)

Warning

That last row is the one to alert on. "budgetStore":"degraded" is a ready router with no working spend ceiling. It is loud in the logs, but a readiness probe alone will not catch it. Match on the field, not the status code.

The journal, and the other sinks

Three observers are composed onto the same completion-hook seam: a journal recorder, the metrics recorder, and the report ring-buffer recorder. You have met the last two.

The journal is an in-memory Journal consumed as a library type from aimock. It is per-instance and has no HTTP endpoint of its own in this router. It is not scrapeable, not dumpable, and not something you can point a log collector at. Treat it as internal request bookkeeping, not an operator surface. If you need per-request history you can query, that is /route/reports.

Recorded fixtures are the one sink that outlives the process. --record, or route.record.enabled in config, persists routed legs as replayable fixtures under route.record.dir. Recording is fire-and-forget by design: a persistence failure never affects a response that has already been sent, so turning it on cannot break serving. See Configuration for the record block.

What is redacted before it reaches any sink

One module owns this, and everything that writes headers or bodies to any sink routes through it: the journal, routing reports, recorded fixtures, debug logs, and the error bodies returned to clients. So: do not expect to find a credential in a report, and do not use the journal or a fixture as a way to recover one.

The mechanism, the header list, and the credential patterns are documented on Security.

Summarizing a window with switchback report

/route/reports gives you a thousand JSON objects. The report subcommand turns a set of them into the shape of the window. It accepts files, globs, or a /route/reports dump, and it recognizes four payload forms in this order: the dump object {reports:[…]}, a bare JSON array, a single report object, or NDJSON.

dump, then summarize shell
curl -s http://127.0.0.1:4011/route/reports > reports.json
node dist/switchback-cli.js report reports.json

# globs work, and --json emits the structured summary
node dist/switchback-cli.js report './reports/*.json' --json

Verified text output:

node dist/switchback-cli.js report reports.json output
switchback report — window of 5 reports

Routed where (upstream: wins / legs):
  stub: 3 win(s) / 4 leg(s)

Failover (4 legs total): 3 success, 1 retryable, 0 fatal
  0 of 5 requests failed over (0.0%)
  by reason:
    silent_empty: 1

Spend by upstream (total 0.000069):
  stub: 0.000069

Estimated savings vs all-OpenRouter baseline:
  baseline (all-OpenRouter): 0.000069
  actual spend:              0.000069
    winner-actual:           0.000069
    failover waste:          0.000000
  price savings:             0.000000
  net savings:               0.000000
  (3 priced, 0 unpriceable)

--json emits the same thing structurally, with top-level keys windowReportCount, routing (byUpstream with {wins, legs}, plus byWinnerModel, byRequestedModel, byWinnerProvider), failover (totalLegs, successLegs, retryableLegs, fatalLegs, byReason, requestsWithFailover, requestFailoverRate), spend (byUpstream, total) and savings. Full flag reference on the CLI page.

How to read the savings block honestly

It is an arithmetic comparison, not a claim. The baseline prices each served request's actual winner token usage at the catalog's OpenRouter list price for the winning model. Actual is the sum of committed costs across every leg. From there, priceSavings = baseline − winnerActual and netSavings = baseline − totalActual, which goes negative when failover waste outweighs the price advantage, and is meant to. Requests with no winner, no usage, or no catalog price are excluded and counted as unpriceable; no dollar is ever invented for them. The baseline deliberately does not apply your priceMap overlay, so overriding a price to zero does not flatter the comparison.

Honest limits

Every one of these has bitten someone reading a report at the wrong moment.

The report buffer is in-memory and process-wide 1 000 reports

A restart loses all history. In a fleet, each process serves only its own reports, so the request you want may be on a replica you are not querying. /route/reports/<id> 404s once the id ages out.

/metrics is per-process no aggregation

The registry is minted per server instance; there is no global singleton. Scrape every replica separately.

switchback_ttft_ms can be entirely absent histogram

It is observed only when a leg reports a finite time-to-first-token. On a non-streaming-only workload the series simply does not exist. That is not a broken exporter.

Absent leg keys carry meaning report

attempts appears only when greater than 1, so its absence means exactly one attempt. A no_route skip leaves no leg entry at all, so a report can be shorter than the chain you configured.

Stream headers lag the report budget

On a stream x-switchback-budget-remaining is the pre-dispatch value, and under keepalive mode the x-switchback-* headers cannot be set at all. The settled figures live in the report.

No tracing, no log shipping, no bundled dashboard by omission not supported

There is no OpenTelemetry integration and no exporter beyond /metrics. The complete sink list is stdout and stderr, the in-memory journal, the report ring buffer, /metrics, and optional recorded fixtures. Anything richer is yours to build on top.

Where to go next

The verdict and reason names you just read in a report are defined in Routing, and the settlement names in Budgets. For wiring /health and /ready into a supervisor or orchestrator, and for the per-replica scrape posture, see Deploy. For every flag report accepts, see the CLI reference.