switchback — part of the aimock family private · pre-launch

Deploy

One stateless Node process, its config read once at startup, its keys held by reference, and exactly one piece of state that matters. This page is the operator's side of that: what to bind, what to probe, what a bad config does, and how to put the thing in a container you build yourself.

The shape of the process

The router is a single Node process on its own http.createServer. It is not embedded in a framework, it does not fork children, and it does not daemonize. It reads the config file once at startup, materializes the provider keys into memory, and dispatches. Node >= 20.17.0 is the floor (see Install for why).

There is no PID file, no log rotation, and no built-in TLS termination. Logs go to stdout and stderr and nowhere else. Whatever supervises your other services (systemd, an orchestrator, a process manager) supervises this the same way, and whatever terminates TLS for the rest of your estate terminates it here.

Only one thing in the process is correctness-bearing state: the budget ledger. The routing-report ring buffer (capacity 1 000, in memory) and the journal are observability: losing them loses history, never money and never a routing decision. Everything else is derived from the config file.

Danger

The router authenticates no inbound request. Anyone who can reach the port can spend your provider keys and your entire budget, and can read /route/reports for other callers' routing history. A private network or an authenticating ingress in front is part of the deployment contract, not a hardening nicety. See Security.

Bind host and port

route.listen.port is required in the config file. There is no port default and no environment variable for it. route.listen.host defaults to 127.0.0.1, so a fresh config is loopback-only until you say otherwise.

Two equivalent ways to accept off-host traffic: set listen.host in the config, or pass --host 0.0.0.0 on the command line. Either path logs the same exposure warning, because the check is against the bound host (127.0.0.1, localhost, ::1) and not against how it got set. --host and --port simply override the config values.

the warning you will see on any non-loopback bind log
[switchback] WARNING: frontend binding non-loopback host "0.0.0.0" — the router holds all
configured provider keys; exposing it on the network is a deliberate choice (§17.2).
Ensure upstream auth and network access are locked down.

listen.port: 0 is legal and binds an OS-assigned port. The port it actually got is in the startup line, which is the only place it appears: [switchback] router listening on http://<host>:<port>. That is fine for a sidecar registered by discovery, and a trap for anything that needs a fixed port in a probe.

Request bodies are capped at 8 MiB, which is the only condition that produces a 413. An unknown path is a structured 404 and is never proxied upstream, so a misrouted health check cannot become a request against a provider.

Config and secrets at runtime

One JSON file describes the whole edge, and it never contains a credential. Each upstream's auth is a reference: { "env": "VAR_NAME" } or { "file": "path" }. An inline secret is refused at load, and the refusal never echoes the value. A key file must be mode 0400 or 0600; any looser mode is a hard error that names the actual octal mode.

Env references are read once, at load. A variable that is unset or empty is a startup failure naming the variable. That is deliberate: the alternative is an upstream that is silently keyless and 401s during an incident. So the deployment shape is the ordinary one: config as a file (a config map, a mounted file, a path on disk), secrets as environment variables injected by whatever already holds them.

the whole runtime contract shell
# secrets come from your secret store into the environment;
# the config file only ever names the variable.
export OPENAI_API_KEY="$(your-secret-tool read openai/router-key)"

node dist/switchback-cli.js --config /etc/switchback/switchback.config.json --redact-env

Validate before you roll

config check loads the config and resolves every referenced environment variable, then exits without serving and without dispatching anything. It prints variable names only, never values, which makes it safe to run in a pipeline log. Run it as the pre-flight step of your rollout, in the same environment the router will get.

pre-flight shell
$ node dist/switchback-cli.js config check --config /etc/switchback/switchback.config.json
Config check passed: /etc/switchback/switchback.config.json is valid and all referenced env vars are set.

Exit 0 means the config parses, every schema rule holds, and every referenced variable is present. Exit 1 means it told you which one is not. A config with no route section is also exit 1: has no "route" section — nothing to serve.

--redact-env, and what it costs

With --redact-env, the referenced key variables are deleted from process.env after the keys are materialized into the isolated auth store. Dispatch keeps working, because the router reads from the store and never from the environment, so what you gain is that a diagnostic dump, or anything that later reads the environment, finds nothing to read.

Warning

The cost is key rotation. With the variables gone there is nothing left for SIGHUP to re-read, so rotating a provider key requires a restart. On a reload, key material carries forward only for auth references that did not change; a re-pointed reference, a changed file path, or a changed auth kind each warn loudly and leave that upstream keyless until restart. Pick per deployment: rotate-by-signal, or scrub-the-environment. You do not get both.

The startup line tells you which mode you are in, every time:

auth mode, at startup log
# default
[switchback] auth: default mode — keys held in memory; SIGHUP re-reads keys from env

# with --redact-env
[switchback] auth: redact-env mode — provider key vars scrubbed from process.env after load;
SIGHUP reloads non-secret config only (key rotation requires restart)
[switchback] redact-env: scrubbed 1 provider key var(s) from the environment after load

Fail fast: what a bad config does

It exits 1 before it serves anything. That is the whole design, and it is worth knowing exactly how far startup gets before each refusal, because the message you find in the logs tells you which gate you hit.

Gate Message shape Exit
Config unreadable or not JSON Failed to load config from <path>: <msg> 1
Schema violation, inline secret, unset auth.env Invalid route config in <path>: <msg> 1
No route section has no "route" section — nothing to serve. 1
backend.kind: "file" with no persistPath budget.backend.kind is "file" but budget.backend.persistPath is not set 1
Bad catalog.refresh URL (checked before any timer arms) Invalid catalog.refresh URL in <path>: <msg> 1
Port already in use, or any bind failure Failed to start switchback frontend: <msg> 1

Two more refusals come from the budget state file specifically, and both will surprise an operator who has not met them. A state file that is group- or other-accessible is refused with its octal mode in the message. A corrupt or wrong-shaped snapshot is also refused, rather than booting from a zeroed ledger that would read every cap as fully available, which is the failure mode that quietly spends real money.

Nothing here degrades into a running-but-wrong process. The one exception is SIGHUP: a reload that throws is swallowed, the last-good config stays in force, and the process survives. See below.

Probes: /health and /ready

Wire liveness to /health and readiness to /ready. They are not interchangeable, and the difference is load-bearing.

Probe Endpoint Behavior
Liveness GET /health Static 200 {"status":"ok"}. It answers if the event loop is turning. It says nothing about dependencies, which is exactly what you want from a restart signal.
Readiness GET /ready 200 {"status":"ready","budgetBackend":"memory"} normally. 503 when the backend is redis, unreachable, and onStoreError is fail-closed.

That 503 is the point of having a separate readiness probe. On a Redis outage with the default fail-closed posture, the router will refuse requests anyway, so it reports itself not-ready and your load balancer drains it, instead of leaving a healthy-looking replica in rotation to serve nothing but refusals. Under fail-open, /ready answers 200 {"status":"ready","budgetBackend":"redis","budgetStore":"degraded"}; alert on that degraded field, because it means caps are not being enforced.

/metrics is Prometheus text exposition and it is per-process: the registry is minted per server instance and there is no shared singleton. Scrape every replica. The same is true of /route/reports: each process serves only its own, in memory, newest first, and a restart loses the window. Details and the exact series names are in Observability.

Signals and lifecycle

Signal What happens
SIGTERM Graceful: deregister handlers, stop the catalog refresher, log [switchback] shutting down…, close the server, exit 0 (exit 1 only if the close itself errors).
SIGINT
SIGHUP Re-read the config file and hot-swap the reloadable derived state, then log [switchback] SIGHUP: re-read config and reloaded derived runtime state. A reload that throws logs retaining last-good config and the process survives.

Warning

Streaming responses are held open by the router for as long as the generation runs. Set your orchestrator's termination grace period above your longest expected stream, or a rolling deploy will cut live generations mid-token. The same number applies to your ingress and load-balancer idle timeouts. The router's own upstream deadlines are separate and per-upstream: timeouts.connectMs, timeouts.ttftMs, timeouts.totalMs.

What SIGHUP actually reloads

The whole file is re-read, but most of it is not reloadable. Anything declined is declined loudly, with a restart-required warning, so a reload never silently diverges from the file on disk.

Section Reloaded on SIGHUP Notes
Budget caps (global, per-upstream, per-route) Yes Rebuilt in place, with accrued spend preserved.
budget.priceMap overlay Yes The price source is re-pointed at the reloaded map.
Budget-cap metric gauges Yes Re-set from the new caps.
Auth registry Default mode only Keys re-read from the environment. Under --redact-env there is nothing left to re-read.
Budget backend (kind, url, persistPath, onStoreError) No Restart to change the backend or its store-error posture.
catalog (refresh, url, interval) No An armed refresher keeps its startup URL.
Observer hooks and the shadow recorder (record) No Restart required.
listen host and port No The socket is bound at startup. The original bind is retained and the served config is restored so it cannot misreport.

--host, --port and --record are re-applied after every reload, so the flags in force never drift from the served config. Full flag and exit-code detail is in the CLI reference.

State: pick a budget backend

This is the one decision that drives your whole topology, so make it first. The question is not "which is best", it is "how many processes have to agree about how much has been spent".

Processes backend.kind Why
One memory Atomicity is free: a reservation is a synchronous read-modify-write with nothing awaited in the critical section. This is the default.
Several, one host file An advisory lockfile plus a whole-ledger rewrite through a temp file and rename. Correct, and deliberately low-throughput.
A fleet redis The only fleet-shared component in the system. Reserve, commit and release are each a single atomic Lua script across the scope keys.

Horizontally, the router scales as identical independent processes behind any TCP load balancer. There is no leader, no gossip, and no shared cache. Point them all at one Redis and the caps become fleet-wide; leave them on memory and each replica enforces its own copy of the cap, which means N replicas can spend N times the ceiling.

single process, bounded loss window json
{
  "backend": {
    "kind": "memory",
    "persistPath": "/var/lib/switchback/budget.json"
  }
}
a handful of workers on one box json
{
  "backend": {
    "kind": "file",
    "persistPath": "/var/lib/switchback/budget.json"
  }
}
fleet-wide caps json
{
  "backend": {
    "kind": "redis",
    "url": "rediss://budget.internal:6379"
  },
  "onStoreError": "fail-closed"
}

memory, with a snapshot

persistPath is optional on memory and it bounds your loss window: the ledger is loaded on boot, written every 30 000 ms, and written again on a clean close. Snapshots are written 0600 with an explicit chmod. Without it, a restart starts from zero spend, which for a monthly cap means the month effectively restarts too.

file, for a handful of workers on one box

persistPath is required here; omitting it is a startup failure. Every operation takes an advisory lock and rewrites the whole JSON ledger, so this is for a few workers sharing a local filesystem, not for a fleet. The lock timeout is 5 000 ms and a lockfile older than 10 000 ms is treated as stale and stolen. Make sure the ledger and its .lock and .tmp siblings live on a local filesystem all the workers share, at mode 0600.

redis, for a fleet

The client is a hand-rolled minimal RESP2 implementation, part of the project's one-runtime-dependency posture, and worth knowing before you plan around a client library feature. Keys are prefixed {switchback-budget}, and the braces are a Cluster hash tag, so every scope key lands in a single slot. Reservation expiry is enforced inside the Lua scripts rather than by key TTL, so a multi-scope reservation expires atomically and a process that dies mid-leg cannot leak reserved budget forever.

Warning

MOVED and ASK redirects are not followed. The supported shapes are standalone, Sentinel behind a proxy, or Cluster behind a single-endpoint proxy. Pointing this at a multi-endpoint Cluster will not work, and the way you will find out is a budget store that fails during traffic rather than at startup.

onStoreError decides what an unreachable store means, and it is the most consequential single word in your config:

budget.onStoreError "fail-closed" | "fail-open" optional default: fail-closed

fail-closed refuses requests and returns 503 from /ready, so the replica drains. fail-open keeps routing uncapped and says so on every occurrence ([switchback][budget] fail-open: cap NOT enforced), while /ready answers 200 with budgetStore: "degraded". It trades your bill for availability. Choose deliberately, and alert on both signals if you choose it.

A fleet config, end to end

Redis-backed caps, a public bind behind your own ingress, fail-closed. This is the shape most multi-replica deployments want.

switchback.config.json json
{
  "route": {
    "listen": { "host": "0.0.0.0", "port": 4011 },
    "upstreams": {
      "openai-direct": {
        "baseURL": "https://api.openai.com/v1",
        "dialect": "openai-compat",
        "profile": "openai",
        "auth": { "env": "OPENAI_API_KEY" },
        "timeouts": { "connectMs": 2000, "ttftMs": 15000, "totalMs": 120000 }
      }
    },
    "models": { "gpt-4o": { "openai-direct": "gpt-4o" } },
    "routes": {
      "chat-default": {
        "match": {},
        "chain": ["gpt-4o"],
        "policy": {
          "advanceOn": ["rate_limited", "server_error", "timeout_ttft"],
          "maxTotalAttempts": 4
        }
      }
    },
    "budget": {
      "backend": { "kind": "redis", "url": "rediss://budget.internal:6379" },
      "global": { "dollars": "5000.00", "window": "monthly" },
      "onStoreError": "fail-closed"
    }
  }
}

Field-by-field meaning lives in Configuration; the failover semantics of chain and advanceOn are in Routing; cap scopes and windows are in Budgets.

Containers: build your own image

Warning

There is no published container image. Nothing has been pushed to any registry, so there is no image to pull. The publish workflow is gated off behind a repo variable that is unset, and it skips on every build. There is also no Helm chart and no Kubernetes manifest anywhere in the repository. What the repository does have is a Dockerfile and a .dockerignore, which means you can build your own image from a clone. That is the only container path today.

What the image actually is

The Dockerfile is a four-stage build: a toolchain stage that activates the pinned pnpm through Corepack, a build stage that runs pnpm build, a separate production-only dependency install, and a runtime stage that copies just those two results in. No source, no dev toolchain, no tests, no docs.

Property Value Why it is that
Base image node:22-alpine, pinned by digest The tag is documentation; the digest is the pin. Node 22 is the active LTS line and clears the >= 20.17.0 floor. The pinned digest currently resolves to Node 22.23.1 on Alpine 3.24.1 and covers linux/amd64 and linux/arm64.
Alpine is safe here no native addons The one production dependency is dependency-free pure JS, and the Redis backend speaks RESP2 over a hand-rolled socket client, so there is no native module to mis-link against musl.
User node (uid/gid 1000) Non-root, reusing the user the official image already has rather than inventing one. The payload stays root-owned and world-readable; the router never writes into /app.
Exposed port 4011 The image's documented port, not a code default. route.listen.port is still required in your config. EXPOSE and the healthcheck are kept in sync with it through one build arg.
Healthcheck GET /health Every 30s, 5s timeout, 10s start period, 3 retries, run with plain node so the image needs no curl or wget.
Entrypoint exec form, node is PID 1 No shell wrapper, so the router receives SIGTERM directly and its exit status is the container's exit status.
Default command --config /app/switchback.config.json --host 0.0.0.0 Replaced wholesale by any arguments you pass to docker run. The 0.0.0.0 bind is required inside a container; loopback would be unreachable from outside it.
No init process no tini, no --init baked in The router spawns no child processes, so there are no orphans to reap and nothing should sit between the runtime and the router's exit code.

Two assertions in the build are worth calling out because they exist to turn a 3am production mystery into a red build. The build stage fails unless dist/switchback-cli.js is executable and the signed price-catalog snapshot (dist/switchback/catalog/snapshot/catalog.json and its .sig) actually landed in dist/. The catalog loader reads those two files from disk and verifies the signature before parsing, so an image without them is a router whose every priced lookup fails. This is the classic runtime-image-missing-its-data-files failure, and it is asserted at build time instead.

Warning

The entrypoint must be the .js entry. The CLI only self-runs when the basename of its script path is switchback, switchback-cli.js or switchback-cli.ts, so the CJS twin, dist/switchback-cli.cjs, starts nothing, exits 0, and looks exactly like a healthy no-op. If you rewrite the entrypoint, keep the .js.

Build it

Ordinary docker build from a clone of the repository. Nothing is fetched from a package registry for this project itself, because the router is built from the source you cloned.

from a clone shell
$ docker build -t switchback:local .

# the two build args, if you need to move off the defaults
$ docker build \
    --build-arg SWITCHBACK_PORT=8787 \
    --build-arg NODE_IMAGE=node:22-alpine@sha256:<digest> \
    -t switchback:local .
NODE_IMAGE build arg optional default: node:22-alpine, digest-pinned

Base image for every stage. Override it with a digest, not a floating tag, or you lose the reproducibility the default is there to provide.

SWITCHBACK_PORT build arg optional default: 4011

Drives EXPOSE and the healthcheck's target port. It does not configure the router. That is still route.listen.port in your config file.

SWITCHBACK_HEALTHCHECK_PORT runtime env var optional default: the SWITCHBACK_PORT build arg

Read only by the container healthcheck. If you change the router's port at run time (a different config, or --port), set this to match and the probe follows, with no rebuild.

Run it

The config is a read-only mount and the keys are environment pass-through. Neither is ever baked into a layer, and the .dockerignore enforces the config half of that: switchback.config.json and switchback.config.*.json are excluded from the build context outright, along with .env files, .git, and the host's dist/, so a stale local build cannot silently become your release.

config mounted, keys by reference shell
$ docker run --rm \
    -p 127.0.0.1:4011:4011 \
    -v /etc/switchback/switchback.config.json:/app/switchback.config.json:ro \
    --env OPENAI_API_KEY \
    switchback:local

# or hand the whole set over from your secret tooling
$ docker run --rm \
    -p 127.0.0.1:4011:4011 \
    -v /etc/switchback/switchback.config.json:/app/switchback.config.json:ro \
    --env-file /run/secrets/switchback.env \
    switchback:local

--env OPENAI_API_KEY with no =value passes the variable through from your shell without the value ever appearing in the command line, the image, or your shell history file. Whatever holds your secrets (an orchestrator's secret object, a secrets manager sidecar, an --env-file written by your deploy tooling) puts them in the environment, and the config file only names them. A credential never belongs in the config file, and the loader will refuse it if you try.

Warning

Note the two different 0.0.0.0s. The router binds 0.0.0.0 inside the container because container loopback is unreachable from outside. That is the default command and it is correct. What you must not do is publish that port to the world: -p 127.0.0.1:4011:4011 keeps it on the host's loopback, and anything wider needs an authenticating ingress in front, because the router still authenticates nobody. The non-loopback exposure warning in the log is expected here and is not something to suppress.

The compose topology

The repository also has a docker-compose.yml, and it wires the one topology worth having locally: the router plus a Redis budget backend. Two services, one named volume, nothing published that does not need to be.

Service What it is Notable
router Built from the Dockerfile in the repository root. Waits for Redis with condition: service_healthy, publishes ${SWITCHBACK_PUBLISH_PORT:-4011}:4011, and repeats the image's default arguments explicitly so you can see them.
redis redis:8-alpine, pinned by digest, started with --appendonly yes. expose: 6379 only, never published, because it holds the spend ledger and only the router needs it. Its data lives on a named redis-data volume.

Append-only persistence on Redis is not a default worth overriding. The budget ledger is money, and a Redis restart that forgets committed spend re-opens the cap it existed to enforce.

The router service is locked down beyond what the image already does: a read_only root filesystem (it writes nothing but stdout), cap_drop: ALL, and no-new-privileges, on top of the non-root node user the image runs as. restart: on-failure works correctly here precisely because the entrypoint is exec form, so a config that fails to load exits non-zero and the container exits non-zero with it.

the two variables it reads shell
# your config file (default: ./switchback.config.json), mounted read-only at
# /app/switchback.config.json. Its budget backend must point at the redis service:
#   "backend": { "kind": "redis", "url": "redis://redis:6379" }
export SWITCHBACK_CONFIG=/etc/switchback/switchback.config.json

# the host side of the published port; the short form publishes on every host
# interface, so scope it if the host is not already private
export SWITCHBACK_PUBLISH_PORT=127.0.0.1:4011

$ docker compose up --build
$ curl -s http://127.0.0.1:4011/ready
{"status":"ready","budgetBackend":"redis"}

Keys reach the router the same way they do outside compose, by reference only. The environment: entries are value-less names (OPENAI_API_KEY, ANTHROPIC_API_KEY, OPENROUTER_API_KEY) which pass the values through from your shell, and env_file points at an optional .env with required: false so the stack still starts without one. No secret is written into the compose file, and none is baked into a layer.

Danger

If you use that optional .env file, add it to .gitignore before you put a key in it. It is not ignored by default in this repository. The build context already excludes .env files, so the risk is not the image; the risk is a commit.

Budget state in a container

If your backend is redis, the container is stateless and there is nothing to mount. If you use memory with a persistPath, or file, then the ledger is on a volume and two facts collide: the router runs as uid 1000, and it refuses at startup any state file that is group- or other-accessible. So the mounted path has to be owned by uid 1000 and mode 0600. A chmod 644 volume, or a file owned by root, is a startup failure with the octal mode in the message, which is the good outcome, but only if you know to expect it.

What this project does not ship

Stated plainly, because the alternative is you discovering it during a rollout.

An operator's short list

If you read one section, read this one. Everything here has a way of being learned the expensive way.