> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tai42.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# backend-celery

> Celery execution backend for the TAI ecosystem — worker/beat/flower runtime, prefork-pool turnover on worker-bus ops, and RedBeat scheduling.

<Info>`Backend` plugin · listing `tai42/backend-celery`</Info>

## Install

```bash theme={null}
tai plugins install tai42-backend-celery
```

## Permissions

| Capability | Declared |
| ---------- | -------- |
| Network    | yes      |
| Subprocess | yes      |
| Filesystem | no       |

## Provides

<h3 id="celery">
  celery
</h3>

`Backend` — Celery execution backend — worker/beat/flower runtime, prefork-pool turnover on worker-bus ops, and RedBeat scheduling.

Choose Celery when you already run RabbitMQ, or when you want beat and flower.
The broker, result backend, and RedBeat schedule store are configured
independently.

## Enable it

```yaml manifest.yml theme={null}
backend_module: tai42_backend_celery
```

## Configuration

The `CELERY_` env group. `CELERY_BROKER_URL`, `CELERY_RESULT_BACKEND`, and
`CELERY_REDBEAT_REDIS_URL` each fall back to the shared `TAI_DEFAULT_REDIS_URL`
when unset — with the default set and `CELERY_BROKER_URL` unset, the broker
resolves to Redis (a first-class Celery broker) rather than the AMQP default. An
operator on RabbitMQ sets `CELERY_BROKER_URL` explicitly.

| Variable                        | Default                    | Effect                                                                  |
| ------------------------------- | -------------------------- | ----------------------------------------------------------------------- |
| `CELERY_BROKER_URL`             | `amqp://localhost:5672//`  | The broker.                                                             |
| `CELERY_RESULT_BACKEND`         | `redis://localhost:6379/0` | Result backend.                                                         |
| `CELERY_REDBEAT_REDIS_URL`      | `redis://localhost:6379/0` | Redis holding the RedBeat schedule store.                               |
| `CELERY_REDBEAT_KEY_PREFIX`     | `redbeat:`                 | Key prefix for the schedule store.                                      |
| `CELERY_BEAT_MAX_LOOP_INTERVAL` | `60`                       | Maximum seconds between beat ticks.                                     |
| `CELERY_WORKER_CONCURRENCY`     | `1`                        | Prefork pool size.                                                      |
| `CELERY_TASK_TIMEOUT`           | `300`                      | Seconds a synchronous dispatch waits for a result.                      |
| `CELERY_MANIFEST_KEY`           | `MANIFEST_KEY`             | Env var the live manifest JSON is written into for prefork inheritance. |
| `CELERY_TOOL_NAME_ARG`          | `backend_tool_name`        | Dispatch kwarg naming the target tool.                                  |

## Run the processes

```bash theme={null}
tai backend worker
tai backend beat
tai backend flower
```

## Worker lifecycle

Everything around the Celery worker is the shared backend lifecycle every backend
gets, not this plugin's own: the launch waits for the app's boot self-resync
before the worker consumes (a worker running against a half-built tool registry
would fail its jobs permanently), the worker's blocking run loop runs on a
dedicated thread so the process's event loop stays free for worker-bus ops, the
first SIGTERM or SIGINT asks Celery for a warm drain, a repeated one escalates to
a cold stop, and the drain is awaited to completion before the process tears
down.

Celery is the one shipped backend whose workers PERSIST ACROSS JOBS, so it is the
one that declares pool turnover: after a registry-mutating fleet op applies, the
live manifest is re-exported into the env and the prefork pool is replaced and
confirmed before the op reports `applied`. The turnover's budget is derived from
the bus apply window (`TAI_BUS_APPLY_TIMEOUT`) less a margin, so a pool that will
not come back reports a truthful `failed` rather than a guessed timeout.

`beat` and `flower` pull no queued work, so they run on the event loop and keep
whatever signal handling their own CLI installs. Nothing else runs in those
processes, so that is deliberate.

## Quirks

* `CELERY_MANIFEST_KEY` and `CELERY_TOOL_NAME_ARG` pin wiring built at boot — a
  forked child reads the env key it was given and a queued job carries the kwarg
  name its producer used — so a change to either converges through a process
  recycle rather than in place. A profile change touching one is refused on an
  unsupervised (`bare`) deployment shape, which has no way to recycle.
* The concurrency cap of one is load-bearing. Celery's pool restart is advisory,
  so a wider prefork pool can miss the turnover confirmation that follows a live
  config change and turn a bus operation into a failure. Raise
  `CELERY_WORKER_CONCURRENCY` only on a deployment that never live-reloads, or
  move to a threads or gevent pool.
* On macOS, prefer the `solo` or `threads` pool — forking around the system
  resolver deadlocks.
* `backend_list_failed_tasks` is not implemented: Celery keeps no queryable
  failed-task index.

## See also

* [Backends](/concepts/backends) — the contract all three shipped backends implement.
* [Schedule a tool](/guides/schedule-a-tool) — putting work on a schedule.