Deploy
One stateless Node process, its config read once at startup, its keys held by reference, and exactly one piece of state that matters. This page is the operator's side of that: what to bind, what to probe, what a bad config does, and how to put the thing in a container you build yourself.
The shape of the process
The router is a single Node process on its own http.createServer. It is not
embedded in a framework, it does not fork children, and it does not daemonize. It reads
the config file once at startup, materializes the provider keys into memory, and
dispatches. Node >= 20.17.0 is the floor (see
Install for why).
There is no PID file, no log rotation, and no built-in TLS termination. Logs go to stdout and stderr and nowhere else. Whatever supervises your other services (systemd, an orchestrator, a process manager) supervises this the same way, and whatever terminates TLS for the rest of your estate terminates it here.
Only one thing in the process is correctness-bearing state: the budget ledger. The routing-report ring buffer (capacity 1 000, in memory) and the journal are observability: losing them loses history, never money and never a routing decision. Everything else is derived from the config file.
Danger
The router authenticates no inbound request. Anyone who can reach the port can spend
your provider keys and your entire budget, and can read
/route/reports for other callers' routing history. A private network or an
authenticating ingress in front is part of the deployment contract, not a hardening
nicety. See Security.
Bind host and port
route.listen.port is required in the config file. There is
no port default and no environment variable for it.
route.listen.host defaults to 127.0.0.1, so a fresh config is
loopback-only until you say otherwise.
Two equivalent ways to accept off-host traffic: set listen.host in the
config, or pass --host 0.0.0.0 on the command line. Either path logs the same
exposure warning, because the check is against the bound host (127.0.0.1,
localhost, ::1) and not against how it got set.
--host and --port simply override the config values.
[switchback] WARNING: frontend binding non-loopback host "0.0.0.0" — the router holds all
configured provider keys; exposing it on the network is a deliberate choice (§17.2).
Ensure upstream auth and network access are locked down.
listen.port: 0 is legal and binds an OS-assigned port. The port it actually
got is in the startup line, which is the only place it appears:
[switchback] router listening on http://<host>:<port>. That is
fine for a sidecar registered by discovery, and a trap for anything that needs a fixed
port in a probe.
Request bodies are capped at 8 MiB, which is the only condition that produces a
413. An unknown path is a structured 404 and is never proxied
upstream, so a misrouted health check cannot become a request against a provider.
Config and secrets at runtime
One JSON file describes the whole edge, and it never contains a credential. Each
upstream's auth is a reference: { "env": "VAR_NAME" } or
{ "file": "path" }. An inline secret is refused at load, and the refusal
never echoes the value. A key file must be mode 0400 or 0600;
any looser mode is a hard error that names the actual octal mode.
Env references are read once, at load. A variable that is unset or empty is a startup failure naming the variable. That is deliberate: the alternative is an upstream that is silently keyless and 401s during an incident. So the deployment shape is the ordinary one: config as a file (a config map, a mounted file, a path on disk), secrets as environment variables injected by whatever already holds them.
# secrets come from your secret store into the environment;
# the config file only ever names the variable.
export OPENAI_API_KEY="$(your-secret-tool read openai/router-key)"
node dist/switchback-cli.js --config /etc/switchback/switchback.config.json --redact-env
Validate before you roll
config check loads the config and resolves every referenced environment
variable, then exits without serving and without dispatching anything. It prints variable
names only, never values, which makes it safe to run in a pipeline log.
Run it as the pre-flight step of your rollout, in the same environment the router will
get.
$ node dist/switchback-cli.js config check --config /etc/switchback/switchback.config.json
Config check passed: /etc/switchback/switchback.config.json is valid and all referenced env vars are set.
Exit 0 means the config parses, every schema rule holds, and every referenced
variable is present. Exit 1 means it told you which one is not. A config with
no route section is also exit 1:
has no "route" section — nothing to serve.
--redact-env, and what it costs
With --redact-env, the referenced key variables are deleted from
process.env after the keys are materialized into the isolated auth store.
Dispatch keeps working, because the router reads from the store and never from the
environment, so what you gain is that a diagnostic dump, or anything that later reads the
environment, finds nothing to read.
Warning
The cost is key rotation. With the variables gone there is nothing left for
SIGHUP to re-read, so rotating a provider key requires a restart. On a
reload, key material carries forward only for auth references that did
not change; a re-pointed reference, a changed file path, or a changed auth kind
each warn loudly and leave that upstream keyless until restart. Pick
per deployment: rotate-by-signal, or scrub-the-environment. You do not get both.
The startup line tells you which mode you are in, every time:
# default
[switchback] auth: default mode — keys held in memory; SIGHUP re-reads keys from env
# with --redact-env
[switchback] auth: redact-env mode — provider key vars scrubbed from process.env after load;
SIGHUP reloads non-secret config only (key rotation requires restart)
[switchback] redact-env: scrubbed 1 provider key var(s) from the environment after load
Fail fast: what a bad config does
It exits 1 before it serves anything. That is the whole design, and it is
worth knowing exactly how far startup gets before each refusal, because the message you
find in the logs tells you which gate you hit.
| Gate | Message shape | Exit |
|---|---|---|
| Config unreadable or not JSON | Failed to load config from <path>: <msg> | 1 |
Schema violation, inline secret, unset auth.env |
Invalid route config in <path>: <msg> | 1 |
No route section |
has no "route" section — nothing to serve. | 1 |
backend.kind: "file" with no persistPath |
budget.backend.kind is "file" but budget.backend.persistPath is not set | 1 |
Bad catalog.refresh URL (checked before any timer arms) |
Invalid catalog.refresh URL in <path>: <msg> | 1 |
| Port already in use, or any bind failure | Failed to start switchback frontend: <msg> | 1 |
Two more refusals come from the budget state file specifically, and both will surprise an operator who has not met them. A state file that is group- or other-accessible is refused with its octal mode in the message. A corrupt or wrong-shaped snapshot is also refused, rather than booting from a zeroed ledger that would read every cap as fully available, which is the failure mode that quietly spends real money.
Nothing here degrades into a running-but-wrong process. The one exception is
SIGHUP: a reload that throws is swallowed, the last-good config stays in
force, and the process survives. See below.
Probes: /health and /ready
Wire liveness to /health and readiness to /ready. They are not
interchangeable, and the difference is load-bearing.
| Probe | Endpoint | Behavior |
|---|---|---|
| Liveness | GET /health |
Static 200 {"status":"ok"}. It answers if the event loop is turning.
It says nothing about dependencies, which is exactly what you want from a restart
signal.
|
| Readiness | GET /ready |
200 {"status":"ready","budgetBackend":"memory"} normally.
503 when the backend is redis, unreachable, and
onStoreError is fail-closed.
|
That 503 is the point of having a separate readiness probe. On a Redis outage with the
default fail-closed posture, the router will refuse requests anyway, so it
reports itself not-ready and your load balancer drains it, instead of leaving a
healthy-looking replica in rotation to serve nothing but refusals. Under
fail-open, /ready answers
200 {"status":"ready","budgetBackend":"redis","budgetStore":"degraded"};
alert on that degraded field, because it means caps are not being enforced.
/metrics is Prometheus text exposition and it is
per-process: the registry is minted per server instance and there is no
shared singleton. Scrape every replica. The same is true of /route/reports:
each process serves only its own, in memory, newest first, and a restart loses the window.
Details and the exact series names are in Observability.
Signals and lifecycle
| Signal | What happens |
|---|---|
| SIGTERM |
Graceful: deregister handlers, stop the catalog refresher, log
[switchback] shutting down…, close the server, exit
0 (exit 1 only if the close itself errors).
|
| SIGINT | |
| SIGHUP |
Re-read the config file and hot-swap the reloadable derived state, then log
[switchback] SIGHUP: re-read config and reloaded derived runtime state. A reload that throws logs retaining last-good config and the
process survives.
|
Warning
Streaming responses are held open by the router for as long as the generation runs. Set
your orchestrator's termination grace period above your longest expected
stream, or a rolling deploy will cut live generations mid-token. The same number applies
to your ingress and load-balancer idle timeouts. The router's own upstream deadlines are
separate and per-upstream: timeouts.connectMs,
timeouts.ttftMs, timeouts.totalMs.
What SIGHUP actually reloads
The whole file is re-read, but most of it is not reloadable. Anything declined is declined loudly, with a restart-required warning, so a reload never silently diverges from the file on disk.
| Section | Reloaded on SIGHUP | Notes |
|---|---|---|
Budget caps (global, per-upstream, per-route) |
Yes | Rebuilt in place, with accrued spend preserved. |
budget.priceMap overlay |
Yes | The price source is re-pointed at the reloaded map. |
| Budget-cap metric gauges | Yes | Re-set from the new caps. |
| Auth registry | Default mode only |
Keys re-read from the environment. Under --redact-env there is
nothing left to re-read.
|
Budget backend (kind, url, persistPath,
onStoreError)
|
No | Restart to change the backend or its store-error posture. |
catalog (refresh, url, interval) |
No | An armed refresher keeps its startup URL. |
Observer hooks and the shadow recorder (record) |
No | Restart required. |
listen host and port |
No | The socket is bound at startup. The original bind is retained and the served config is restored so it cannot misreport. |
--host, --port and --record are re-applied after
every reload, so the flags in force never drift from the served config. Full flag and
exit-code detail is in the CLI reference.
State: pick a budget backend
This is the one decision that drives your whole topology, so make it first. The question is not "which is best", it is "how many processes have to agree about how much has been spent".
| Processes | backend.kind |
Why |
|---|---|---|
| One | memory | Atomicity is free: a reservation is a synchronous read-modify-write with nothing awaited in the critical section. This is the default. |
| Several, one host | file | An advisory lockfile plus a whole-ledger rewrite through a temp file and rename. Correct, and deliberately low-throughput. |
| A fleet | redis | The only fleet-shared component in the system. Reserve, commit and release are each a single atomic Lua script across the scope keys. |
Horizontally, the router scales as identical independent processes behind any TCP load
balancer. There is no leader, no gossip, and no shared cache. Point them all at one Redis
and the caps become fleet-wide; leave them on memory and each replica
enforces its own copy of the cap, which means N replicas can spend N times the ceiling.
{
"backend": {
"kind": "memory",
"persistPath": "/var/lib/switchback/budget.json"
}
}
{
"backend": {
"kind": "file",
"persistPath": "/var/lib/switchback/budget.json"
}
}
{
"backend": {
"kind": "redis",
"url": "rediss://budget.internal:6379"
},
"onStoreError": "fail-closed"
}
memory, with a snapshot
persistPath is optional on memory and it bounds your loss
window: the ledger is loaded on boot, written every 30 000 ms, and written again
on a clean close. Snapshots are written 0600 with an explicit
chmod. Without it, a restart starts from zero spend, which for a monthly cap
means the month effectively restarts too.
file, for a handful of workers on one box
persistPath is required here; omitting it is a startup
failure. Every operation takes an advisory lock and rewrites the whole JSON ledger, so
this is for a few workers sharing a local filesystem, not for a fleet. The lock timeout is
5 000 ms and a lockfile older than 10 000 ms is treated as stale and
stolen. Make sure the ledger and its .lock and .tmp siblings
live on a local filesystem all the workers share, at mode 0600.
redis, for a fleet
The client is a hand-rolled minimal RESP2 implementation, part of the project's
one-runtime-dependency posture, and worth knowing before you plan around a client library
feature. Keys are prefixed {switchback-budget}, and the braces are a Cluster
hash tag, so every scope key lands in a single slot. Reservation expiry is enforced inside
the Lua scripts rather than by key TTL, so a multi-scope reservation expires atomically
and a process that dies mid-leg cannot leak reserved budget forever.
Warning
MOVED and ASK redirects are not followed. The
supported shapes are standalone, Sentinel behind a proxy, or Cluster behind a
single-endpoint proxy. Pointing this at a multi-endpoint Cluster will not work,
and the way you will find out is a budget store that fails during traffic rather than at
startup.
onStoreError decides what an unreachable store means, and it is the most
consequential single word in your config:
fail-closed refuses requests and returns 503 from
/ready, so the replica drains. fail-open keeps routing
uncapped and says so on every occurrence ([switchback][budget] fail-open: cap NOT enforced), while /ready answers 200 with budgetStore: "degraded".
It trades your bill for availability. Choose deliberately, and alert on both signals
if you choose it.
A fleet config, end to end
Redis-backed caps, a public bind behind your own ingress, fail-closed. This is the shape most multi-replica deployments want.
{
"route": {
"listen": { "host": "0.0.0.0", "port": 4011 },
"upstreams": {
"openai-direct": {
"baseURL": "https://api.openai.com/v1",
"dialect": "openai-compat",
"profile": "openai",
"auth": { "env": "OPENAI_API_KEY" },
"timeouts": { "connectMs": 2000, "ttftMs": 15000, "totalMs": 120000 }
}
},
"models": { "gpt-4o": { "openai-direct": "gpt-4o" } },
"routes": {
"chat-default": {
"match": {},
"chain": ["gpt-4o"],
"policy": {
"advanceOn": ["rate_limited", "server_error", "timeout_ttft"],
"maxTotalAttempts": 4
}
}
},
"budget": {
"backend": { "kind": "redis", "url": "rediss://budget.internal:6379" },
"global": { "dollars": "5000.00", "window": "monthly" },
"onStoreError": "fail-closed"
}
}
}
Field-by-field meaning lives in Configuration; the failover
semantics of chain and advanceOn are in
Routing; cap scopes and windows are in
Budgets.
Containers: build your own image
Warning
There is no published container image. Nothing has been pushed to any
registry, so there is no image to pull. The publish workflow is gated off behind a repo
variable that is unset, and it skips on every build. There is also no Helm chart and no
Kubernetes manifest anywhere in the repository. What the repository does have is a
Dockerfile and a .dockerignore, which means you can build your
own image from a clone. That is the only container path today.
What the image actually is
The Dockerfile is a four-stage build: a toolchain stage that activates the
pinned pnpm through Corepack, a build stage that runs pnpm build, a separate
production-only dependency install, and a runtime stage that copies just those two results
in. No source, no dev toolchain, no tests, no docs.
| Property | Value | Why it is that |
|---|---|---|
| Base image | node:22-alpine, pinned by digest |
The tag is documentation; the digest is the pin. Node 22 is the active LTS line
and clears the >= 20.17.0 floor. The pinned digest currently
resolves to Node 22.23.1 on Alpine 3.24.1 and covers linux/amd64 and
linux/arm64.
|
| Alpine is safe here | no native addons | The one production dependency is dependency-free pure JS, and the Redis backend speaks RESP2 over a hand-rolled socket client, so there is no native module to mis-link against musl. |
| User | node (uid/gid 1000) |
Non-root, reusing the user the official image already has rather than inventing
one. The payload stays root-owned and world-readable; the router never writes into
/app.
|
| Exposed port | 4011 |
The image's documented port, not a code default.
route.listen.port is still required in your config.
EXPOSE and the healthcheck are kept in sync with it through one build
arg.
|
| Healthcheck | GET /health |
Every 30s, 5s timeout, 10s start period, 3 retries, run with plain
node so the image needs no curl or wget.
|
| Entrypoint | exec form, node is PID 1 |
No shell wrapper, so the router receives SIGTERM directly and its
exit status is the container's exit status.
|
| Default command | --config /app/switchback.config.json --host 0.0.0.0 |
Replaced wholesale by any arguments you pass to docker run. The
0.0.0.0 bind is required inside a container; loopback would be
unreachable from outside it.
|
| No init process | no tini, no --init baked in | The router spawns no child processes, so there are no orphans to reap and nothing should sit between the runtime and the router's exit code. |
Two assertions in the build are worth calling out because they exist to turn a 3am
production mystery into a red build. The build stage fails unless
dist/switchback-cli.js is executable and the signed price-catalog
snapshot (dist/switchback/catalog/snapshot/catalog.json and its
.sig) actually landed in dist/. The catalog loader reads those
two files from disk and verifies the signature before parsing, so an image without them is
a router whose every priced lookup fails. This is the classic
runtime-image-missing-its-data-files failure, and it is asserted at build time instead.
Warning
The entrypoint must be the .js entry. The CLI only self-runs when the
basename of its script path is switchback,
switchback-cli.js or switchback-cli.ts, so the CJS twin,
dist/switchback-cli.cjs, starts nothing, exits 0, and looks
exactly like a healthy no-op. If you rewrite the entrypoint, keep the .js.
Build it
Ordinary docker build from a clone of the repository. Nothing is fetched from
a package registry for this project itself, because the router is built from the source
you cloned.
$ docker build -t switchback:local .
# the two build args, if you need to move off the defaults
$ docker build \
--build-arg SWITCHBACK_PORT=8787 \
--build-arg NODE_IMAGE=node:22-alpine@sha256:<digest> \
-t switchback:local .
Base image for every stage. Override it with a digest, not a floating tag, or you lose the reproducibility the default is there to provide.
Drives EXPOSE and the healthcheck's target port. It does
not configure the router. That is still
route.listen.port in your config file.
Read only by the container healthcheck. If you change the router's port at
run time (a different config, or --port), set this to match and the
probe follows, with no rebuild.
Run it
The config is a read-only mount and the keys are environment pass-through. Neither is ever
baked into a layer, and the .dockerignore enforces the config half of that:
switchback.config.json and switchback.config.*.json are excluded
from the build context outright, along with .env files, .git,
and the host's dist/, so a stale local build cannot silently become your
release.
$ docker run --rm \
-p 127.0.0.1:4011:4011 \
-v /etc/switchback/switchback.config.json:/app/switchback.config.json:ro \
--env OPENAI_API_KEY \
switchback:local
# or hand the whole set over from your secret tooling
$ docker run --rm \
-p 127.0.0.1:4011:4011 \
-v /etc/switchback/switchback.config.json:/app/switchback.config.json:ro \
--env-file /run/secrets/switchback.env \
switchback:local
--env OPENAI_API_KEY with no =value passes the variable through
from your shell without the value ever appearing in the command line, the image, or your
shell history file. Whatever holds your secrets (an orchestrator's secret object, a
secrets manager sidecar, an --env-file written by your deploy tooling) puts
them in the environment, and the config file only names them. A credential never belongs
in the config file, and the loader will refuse it if you try.
Warning
Note the two different 0.0.0.0s. The router binds 0.0.0.0
inside the container because container loopback is unreachable from outside.
That is the default command and it is correct. What you must not do is publish that port
to the world: -p 127.0.0.1:4011:4011 keeps it on the host's loopback, and
anything wider needs an authenticating ingress in front, because the router still
authenticates nobody. The non-loopback exposure warning in the log is expected here and
is not something to suppress.
The compose topology
The repository also has a docker-compose.yml, and it wires the one topology
worth having locally: the router plus a Redis budget backend. Two services, one named
volume, nothing published that does not need to be.
| Service | What it is | Notable |
|---|---|---|
| router | Built from the Dockerfile in the repository root. |
Waits for Redis with condition: service_healthy, publishes
${SWITCHBACK_PUBLISH_PORT:-4011}:4011, and repeats the image's
default arguments explicitly so you can see them.
|
| redis |
redis:8-alpine, pinned by digest, started with
--appendonly yes.
|
expose: 6379 only, never published, because it holds the spend ledger
and only the router needs it. Its data lives on a named
redis-data volume.
|
Append-only persistence on Redis is not a default worth overriding. The budget ledger is money, and a Redis restart that forgets committed spend re-opens the cap it existed to enforce.
The router service is locked down beyond what the image already does: a
read_only root filesystem (it writes nothing but stdout),
cap_drop: ALL, and no-new-privileges, on top of the non-root
node user the image runs as. restart: on-failure works correctly
here precisely because the entrypoint is exec form, so a config that fails to load exits
non-zero and the container exits non-zero with it.
# your config file (default: ./switchback.config.json), mounted read-only at
# /app/switchback.config.json. Its budget backend must point at the redis service:
# "backend": { "kind": "redis", "url": "redis://redis:6379" }
export SWITCHBACK_CONFIG=/etc/switchback/switchback.config.json
# the host side of the published port; the short form publishes on every host
# interface, so scope it if the host is not already private
export SWITCHBACK_PUBLISH_PORT=127.0.0.1:4011
$ docker compose up --build
$ curl -s http://127.0.0.1:4011/ready
{"status":"ready","budgetBackend":"redis"}
Keys reach the router the same way they do outside compose, by reference only. The
environment: entries are value-less names (OPENAI_API_KEY,
ANTHROPIC_API_KEY, OPENROUTER_API_KEY) which pass the values
through from your shell, and env_file points at an optional
.env with required: false so the stack still starts without one.
No secret is written into the compose file, and none is baked into a layer.
Danger
If you use that optional .env file, add it to .gitignore
before you put a key in it. It is not ignored by default in this repository.
The build context already excludes .env files, so the risk is not the
image; the risk is a commit.
Budget state in a container
If your backend is redis, the container is stateless and there is nothing to
mount. If you use memory with a persistPath, or
file, then the ledger is on a volume and two facts collide: the router runs
as uid 1000, and it refuses at startup any state file that is group- or
other-accessible. So the mounted path has to be owned by uid 1000 and mode
0600. A chmod 644 volume, or a file owned by root, is a startup
failure with the octal mode in the message, which is the good outcome, but only if you
know to expect it.
What this project does not ship
Stated plainly, because the alternative is you discovering it during a rollout.
-
No published image, no registry pull, no Helm chart, no Kubernetes manifests.
The repository has a
Dockerfile; you build it. The publish workflow is gated off on an unset repository variable and skips. - No inbound authentication. Not a token, not mTLS, not an allowlist. That is your ingress's job.
- No TLS termination, no PID file, no daemonization, no log rotation.
-
No OpenTelemetry, no tracing, no log shipping. The sinks are
stdout/stderr,
/metrics,/route/reports, the in-memory journal, and optionally recorded fixtures. - No live catalog refresh in this build. The network refresh path refuses to arm while the pinned verifying key is the committed development key, and logs why. The bundled snapshot still verifies, so plan on shipping price-catalog updates by redeploying.
-
No
--versionflag and no version in the startup log. Track what you deployed by image tag or commit, because the process will not tell you.
An operator's short list
If you read one section, read this one. Everything here has a way of being learned the expensive way.
-
Ship the catalog snapshot. A deploy that copies only
dist/*.jsfails at the first priced lookup. The container build asserts this for you; a hand-rolled copy step does not. - Set the termination grace period and every idle timeout above your longest expected generation.
-
Liveness on
/health, readiness on/ready. Never liveness on/ready: a Redis blip would then restart the fleet instead of draining it. -
Scrape
/metricson every replica; there is no aggregate. Same for/route/reports: dump it periodically and feed the dumps toswitchback reportif you want fleet-wide history. -
More than one replica on
memorymeans N replicas times your cap. Use Redis the moment you scale past one process. -
fail-openmeans your caps are not enforced. If you choose it, alert on thedegradedreadiness field and on the fail-open log line. -
Run
config checkin the rollout, in the target environment. A missing variable is a startup failure, and you would rather find it in the pipeline.