Skip to main content
Run a server for real — transport selection, environment, and the config-provider swap point. A production server is tai serve with a transport, a bound address, a manifest, and the environment its tools need. This page covers those choices. Provider-specific settings (a config provider’s own environment variables) live in that provider’s repository README.
Running inside your own ASGI process instead of tai serve — a plain uvicorn run or a mount in an existing FastAPI/Starlette app? See Embed in your own ASGI process.

​
Choose a transport

tai serve runs the MCP server over one transport. Select it with -t/--transport:
For a fast same-machine path, bind a Unix domain socket with --uds /path/to/socket instead of a host and port.

​
Scale workers

Run multiple worker processes with -w/--workers. The transport constrains the worker count:
  • stdio and sse run a single worker.
  • http / streamable-http run a single stateful worker by default; pass --stateless-http to lift the single-worker restriction and run several.
More than one worker also requires the worker bus and shared sub-MCP and hooks stores: set TAI_BUS_REDIS_URL, SUB_MCP_REDIS_URL and HOOKS_REDIS_URL (TAI_DEFAULT_REDIS_URL stands in for all three). Without the bus, sibling workers would serve stale config after a reload with no channel to converge on. Without a shared Redis, each store is per-process: a sub-MCP app registered on one worker is not served by its siblings, and a hook registered on one worker never runs for an event another worker accepts.
The full worker and transport option surface:
examples/deploy/serve_workers_help.sh
--workers > 1 is refused for stdio and sse, and for http / streamable-http unless --stateless-http is set. A stateful HTTP server keeps session state a second worker would not see.tai serve hands its worker count to each worker, and each worker refuses to boot without the bus or with either store in-memory, naming the setting to set. The worker checks after it applies the stored env (the config-dir .env or the config provider’s store), so a URL kept only in the stored env counts when no worker’s deployment sets it, and a URL the deployment’s environment sets keeps its value whatever the stored env holds, on every worker; a worker started without the URL while another worker’s environment sets it refuses to boot, naming it. A worker that refuses fails its startup, and tai serve exits non-zero. A config reload, and any config write (an env change, a manifest change, a profile replace), that would leave a multi-worker server without the bus or with either store in-memory is refused the same way.
A single-worker deployment may run on the default in-memory stores, unless it registers an execution backend: a backend_module needs the shared hooks store at any worker count, because the backend process matches the events it raises against its own hooks registry, and an in-memory one holds none of the hooks registered through the server.

​
When a worker dies

The serve process checks each worker on a health ping and replaces any worker that dies. Each replacement logs one WARNING on the uvicorn.error logger. It gives the worker’s pid, why it died, and its exit code or the signal that ended it: Every worker and the serve process also enable Python’s fault handler. A worker that crashes in native code writes Fatal Python error: ... and the stack of every thread to stderr before it dies, unless PYTHONFAULTHANDLER already configures it.

​
Run under a supervisor

Run tai serve and every tai backend runtime under a supervisor that starts a process again when it exits: a Compose restart policy, Kubernetes, or a systemd unit with Restart=. A change to a recycle-class setting makes the affected processes exit gracefully, and the supervisor starts them again under the new env. Without a supervisor, an exited process stays down, and the roll reports the target where it stopped. A supervisor restarts a process that exits, not one that stays up and stops working. On Kubernetes, give each tai backend process the liveness probe its kind declares, so a stalled process is restarted too: see Backend processes. The graceful-shutdown timeout each worker gets is set on the tai serve process’s environment (APP_ARGS_TIMEOUT_GRACEFUL_SHUTDOWN, default 10 seconds) or passed as the uvicorn option --timeout-graceful-shutdown SECONDS after the tai serve options, and takes effect on a tai serve restart. On SIGTERM or SIGINT a worker stops taking new connections, writes the replies of the calls still in flight, MCP replies included, and then closes its open event streams; the timeout caps that stop phase of the server. The app’s shutdown follows it, within the bounds the app’s modules declare: twice HOOKS_RUN_SHUTDOWN_GRACE_SECONDS for the hook-run attempts, 30 seconds for the credential refreshes to finish a write-back, STATES_OUTBOX_SHUTDOWN_GRACE_SECONDS for the states outbox, and TAI_TOOL_RUNS_SHUTDOWN_DRAIN_SECONDS for the tool-run supervisors (100 seconds at the defaults). A leaving worker announces the sum, the timeout plus those bounds (110 seconds at the defaults), on its census row, which reads recycling from the start of its stop until the stop ends, and a fleet restart waits that long for it to go; a tai backend worker announces its backend’s drain bound plus the same bounds (430 seconds at the defaults for the shipped backends). Give the deployment’s termination grace room for the sum. A Celery worker run with -B also waits up to 10 seconds, after its drain, for its embedded beat to end, a wait outside the sum it announces: give it that much more grace.

​
Point at a manifest and environment

The server loads a manifest and reads its environment at startup. Point at the manifest with --manifest-path, or set TAI_MANIFEST_PATH as the flag’s default. Provide the environment your tools need — see the repository’s .env.example.
The default config directory the file provider anchors .env, manifest.yml, and templates under is /app; override it with TAI_CONFIG_DIR_PATH. That .env is the stored env: the .env that tai loads into the process environment at startup is never this file — when python-dotenv’s search lands on it, startup logs a WARNING and skips it. A .env that names its own folder as the config directory while the process environment sets none is refused at startup, naming the file: the config directory is a deployment value, set in the environment, never in the stored env. A key you set to a non-empty value in the environment of any tai serve or tai backend process belongs to the whole deployment: its value wins over a stored value, Studio shows it locked, a save that sets it is refused, and no process on the same worker bus applies a stored copy of it (a process on another bus, or on none, judges by its own environment alone). A process you did not give the value runs its TAI_DEFAULT_* fallback or the field’s default. Change it in the environment and restart. Leave a setting out of the environment when an admin should edit it from the app; the app then stores it. See Deployment values and stored values.

​
HTTP routers and headless mode

Out of the box the server serves the full Studio surface — every built-in page and its API — with no router list in the manifest. The default_routers field controls this:
  • all (the default) — serve the whole Studio UI plus its API. A bare manifest gives you the full surface; you never enumerate the core routers, and no page is left dark for want of a listing.
  • api — a headless deployment: mount the core JSON API (/api/*) but do not serve the browser SPA. Use it for an API-only backend with no UI.
  • none — mount nothing by default and list every router in routers_modules yourself. For a fully manual or MCP-only surface.
Feature lifecycles do not follow the router selection: the conversations completion tools, re-drive and delivery sweep run under every value, and the marketplace advisory poll starts only where GET /api/marketplace/advisories is served (under none, list tai42_skeleton.routers.marketplace to serve it). routers_modules adds extras — your own routers, a plugin’s — on top of the selected set; the loader keeps the Studio SPA catch-all last. See the manifest for the field reference.

​
Run mode and logging

TAI_RUN_MODE selects how tai serve runs. Leave it unset (or empty) for the normal multi-worker server; set it to debug for a single in-process server, which ignores --workers. Any other value is rejected loudly at startup rather than silently falling through — a typo never selects a mode.
Set the log level with TAI_LOG_LEVEL (default INFO). A config reload that changes it re-applies logging under tai serve. The tai serve parent process and tai metrics apply no stored env; they read TAI_LOG_LEVEL from their own environment only. A stored TAI_LOG_LEVEL applies in the serve workers and tai backend, at boot and on reload.
See Config and secrets for the full environment-variable reference, including the timeout and limit knobs.

​
Swap the config provider

Where the server reads its env and manifest is a swappable provider. The default file provider reads from disk with zero dependencies. Select a different provider — for example a Kubernetes-backed one — with TAI_CONFIG_MODE:
A config provider is a plugin; its own settings live in its repository README. See Config and secrets.
TAI_CONFIG_MODE=kubernetes requires the worker bus. The kubernetes config mode exists for multi-pod shared config, and a pod cannot see its own replica count, so a busless kubernetes boot is refused — set TAI_BUS_REDIS_URL. The refusal names the missing bus, not a later kubeconfig error.

​
Access control at deploy

Enabling access control (ACCESS_CONTROL_ENABLE, on by default) adds three deploy requirements. Postgres for the policy store. The access-control policy and route tables live in the named database the core skeleton component binds to — the default database unless TAI_DB_BINDING_SKELETON points it elsewhere. Configure that database, including its password:
Any plain Redis. The auth gate reads its live context from plain Redis hashes. There is no baked-in default: point the connection with ACCESS_CONTROL_REDIS_URL, or let it resolve from the shared TAI_DEFAULT_REDIS_URL — with the gate on, one of the two must be set, or every request that carries a key is denied. The Redis API-key provider keeps its records in its own Redis, TAI_IDENTITY_REDIS_URL, which also falls back to TAI_DEFAULT_REDIS_URL. Any standard, module-less redis-server — redis:7, redis:8, or valkey — runs access control, the worker bus, the task backends, and the feature stores. The one exception is the default checkpoint store and LLM store (LLM_PROVIDER_CHECKPOINT and LLM_PROVIDER_STORE set to redis): they need a Redis with the JSON and search modules — Redis 8, or a Redis 7 with RedisJSON and RediSearch loaded. See Checkpoints and Infrastructure by feature. An identity provider in the manifest. The skeleton ships no concrete identity provider, so a deployment with access control enabled must name one as a lifecycle_modules entry — its module registers the provider at import. The usual choice is the Redis-backed api-key provider, which registers itself as the "redis" provider. ACCESS_CONTROL_AUTH_PROVIDERS — an ordered JSON list — names the chain the verifier tries; unset, the chain is every registered identity provider, sorted by name. Swap the module to change the authn backend.
examples/deploy/access_control_manifest.yaml
Serving the Studio needs no SPA public prefixes. With access control on, you do not pin /, /assets, /vendor, or any SPA path public — the dataless Studio SPA shell is served on any unmatched non-/api GET by default (spa_shell_public), and it leaks nothing because all data sits behind authenticated /api/* calls. The login routes under /api/login are public by their own route declaration, and everything else under /api/* stays authed. A headless (no-Studio) deployment can set spa_shell_public to false to turn the fallback off. No route-to-scope step is needed to make the deployment usable. The seeded roles (admin/editor/viewer) carry the universal scope, so on a fresh deployment every role-holder reaches every registered /api route — including the MCP transports — out of the box, with the route table empty. tai scopes add is only for scoped keys: map a route to a named scope to let a key minted with that scope (rather than *) reach it. See the access-control guide for the end-to-end walkthrough.

​
Liveness and readiness

Orchestrators check a tai serve worker over its HTTP probe routes, and a tai backend process by the signal its kind declares.

​
The server

The server exposes two operational routes for orchestrators and load balancers. Both are GET, unauthenticated, and served outside the OpenAPI surface — there is no CLI command for them. /health stays green as long as the process is up. /ready lets an orchestrator hold traffic off a worker whose Redis or Postgres is unreachable — such a worker would 500 every real request but stay live. Each subsystem declares the stores it uses, and checks carries one row per name its targets declare. A row appears only when its store is configured. The platform’s own subsystems name their rows after themselves: Each enabled identity provider adds the stores it declares for itself, under the row names it chooses: the accounts provider reports its Postgres and its Redis as accounts, and the Redis API-key provider reports its record store as access_control. Key a readiness gate or alert on the row names your deployment’s /ready returns. Rows that share one connection are pinged once and report that one result. A dead conversation-bridge or worker-bus Redis fails readiness like any other wired store. A stalled background task fails readiness. Beyond the backing-store pings, /ready also reports the platform’s own long-lived background tasks — the fleet-bus subscription, the failed-MCP re-probe, and the expiry reapers. If one of these stops unexpectedly, /ready returns 503 not_ready and names the stopped task under checks (as perpetual_task:<name>), and the process exits so a supervisor that restarts it (restart: unless-stopped in the bundled Compose, or an equivalent orchestrator policy) brings it back from a clean boot. /health keeps returning 200 until the process actually exits, so a liveness probe alone will not notice the stall — wire the restart policy, and probe /ready where your orchestrator supports a readiness gate. The probe routes are public by default — no manual step. With access control enabled, unknown routes are denied, but /health and /ready are served public by the app’s own route-level declaration: they ship in ACCESS_CONTROL_ACKNOWLEDGED_PUBLIC_ROUTES, so a probe carrying no key reaches them with nothing to configure. Do not also list them in ACCESS_CONTROL_ALWAYS_PUBLIC_PATH_PREFIXES — an acknowledged route that overlaps an always-public prefix is rejected at boot.
/ready passes the access-control gate without reading the gate’s own Redis, because the route is declared public. A dead gate Redis therefore reaches the handler and fails the access_control row, so the probe returns 503 while every request to a protected route is refused.

​
Backend processes

A tai backend process serves no /health or /ready. Each process an execution backend runs declares how an orchestrator checks it: a process that runs a loop reports each pass of that loop into a liveness file, and a process that serves HTTP is probed over HTTP. The liveness file. Each report writes the process’s pid and a horizon into the file at TAI_BACKEND_LIVENESS_PATH: the seconds within which the loop promised its next pass, plus TAI_BACKEND_LIVENESS_INTERVAL. The process removes the file when it exits. The process rewrites the file only when the promise already in it would expire before the next pass. A steady loop therefore writes about once per interval, and a beat whose sleep lengthens writes at once. The probe. tai42-backend-probe ships with tai42-kit, so every image that runs tai backend has it. Run in the process’s container, it reads the file and prints one line. It exits 0 while the file’s age is at most its horizon plus the grace (--grace SECONDS, default TAI_BACKEND_LIVENESS_GRACE), and 1 when the file is stale or missing:
A file that does not hold a pid and a horizon, or cannot be read, fails the probe with the error. An invalid --grace exits 2. The probe reads its settings from the process environment only, never from an env file, so set them in the container’s environment, where the process and the probe both read them: These are excluded-band deployment values: a stored env or a profile that carries one is refused. One file per process. A report into a file that holds the pid of another live process is refused with an error naming TAI_BACKEND_LIVENESS_PATH, because two writers on one file would hide either one’s stall. A file left by a process that no longer exists is taken over. Each container has its own /tmp, so the default path suits one process per container; processes that share a filesystem, such as several on one host, each need their own path. What a report proves. A report comes from the engine’s own loop, so a loop that stops passing stops reporting. A job that blocks the loop stops the reports. Where the pool cannot always end a job at its time budget, a job still running past that budget stops the reports too, so the probe ends the worker. On RQ’s solo pool and Celery’s solo, threads and eventlet pools, that happens once one such job runs. The gevent pools of RQ and Celery cancel a job’s work at the budget, so there it happens once a job’s work still runs a tick after that cancel. A job or a fork that arrives while a config reload re-imports the manifest modules waits for the re-import to finish. On RQ and Celery the worker reports each second while it waits; an arq job waits without holding up the worker’s loop, so the reports go on. A Celery worker reports nothing while its broker connection is down and resumes once it reconnects. Each engine’s report points and per-pool behaviour are on its plugin page: arq, Celery, RQ. A Celery worker with an embedded beat. tai backend worker -B is one process with one probe. Its beat reports into its own file, <TAI_BACKEND_LIVENESS_PATH>-beat, and the worker reports nothing while its beat’s promise is overdue, so the probe on the worker’s file covers both. The worker removes both files when it exits. One Celery beat. A Celery beat that cannot hold RedBeat’s lock waits for it before its first tick, trying once a second and reporting each try, so it reads alive while it waits and starts ticking within a second of the lock lapsing (RedBeat holds it for 5 × --max-interval, 300 s at the default, and a cleanly stopped beat releases it at once). Run one beat, standalone or embedded with -B; a second one waits and takes over only if the first stops. tai backend worker and tai backend beat refuse at launch any scheduler other than the reporting RedBeat scheduler they run. The refusal covers the --scheduler option (the beat’s -S), the CELERY_WORKER_SCHEDULER / CELERY_BEAT_SCHEDULER environment variables, and beat_scheduler= as command-line configuration. On Kubernetes. Give a loop process (a worker or a beat) an exec startupProbe and livenessProbe on the probe. timeoutSeconds: 5 leaves room for the probe’s Python start, which can exceed the 1-second default under a CPU limit. The startup probe’s 300 seconds cover the backend’s 120-second wait for its app to become ready, plus the engine’s start:
Give an HTTP process an httpGet probe on its path and the port it serves: Flower’s --port (default 5555) and the RQ dashboard’s -p/--port (default 9181). Under Flower’s --url_prefix the path is /<prefix>/healthcheck, and under the dashboard’s --url-prefix it is /<prefix>/favicon.ico. GET /healthcheck answers without authentication even when Flower’s own auth is on. The dashboard’s /favicon.ico sits behind its basic auth once --username is set, so the probe then sends the Authorization header:
Timing. A steady loop rewrites its file once the last write is an interval old, so after the loop stops the file turns stale within its promised next pass plus the grace, plus up to one interval. At the defaults that is about 25 to 35 seconds for a worker (arq 25.5 to 30.5, Celery 26 to 31, RQ 30 to 35), and the liveness probe above restarts the container within about a minute. A beat at its default 60-second interval promises 65 seconds, so a stalled beat is restarted within about two minutes; a beat with a schedule due sooner promises less and is restarted sooner; a beat waiting for RedBeat’s lock promises 6 seconds and reads alive. Raise TAI_BACKEND_LIVENESS_GRACE to ride out longer pauses, such as a Celery broker outage, without a restart. The boot signal. Every tai backend process writes a readiness sentinel file once its first boot resync has built its tool registry, and removes it when shutdown begins: TAI_READY_SENTINEL_PATH, default /tmp/tai-ready. Keep the path container-local and ephemeral, never on a persistent mount, so a restarted container starts unready. A backend pod has no Service endpoints for readiness to gate, but an exec readinessProbe on the file (["test", "-f", "/tmp/tai-ready"]) holds a rolling update until the new pod’s registry is built.

​
Preflight check

Run read-only health diagnostics before serving — Python version, config mode, the skeleton database (reached with the migrator identity), whether its schema is applied, and the connector-store Redis (CONNECTOR_STORE_REDIS_URL or TAI_DEFAULT_REDIS_URL). An unconfigured database or Redis counts as a failed check. It exits non-zero when any check fails, and redacts credentials. It does not check the other Redis features; Infrastructure by feature lists them.
Requires Python 3.13+. tai ships with the runtime — pip install tai42-skeleton. To run it against an unreleased change instead, install from the repository: pip install git+https://github.com/tai42ai/tai42#subdirectory=core/skeleton.

​
Self-host the whole stack

Running the platform as containers — the minimal-core image, Postgres and Redis, and the Compose bundle — is covered in Operate. Datastores are never host-published there; only serve’s port is, and access control gates it exactly as above.

​
See also