TAI42 — the OS for AI tools

A state is a declared JSON document with one instance per subject. The platform owns the store: a conversation turn, a hook fire, a scheduled job, a builtin tool, and the HTTP API all read and write the same subject’s document through one seam, and every write lands in one audit trail. Domain behaviour — which consumer reads which state, how a record is shaped for a use case — lives in the consumer that binds the state, never in the store. The store is a first-class platform component. It ships with the platform but serves nothing until its database is bound (see Enabling the store); until then every read and write refuses loudly with a 501 rather than returning an empty document.

The subject

A subject is four fields:

{
  "target_kind": "agent",
  "target_name": "A-42",
  "kind": "person",
  "key": "alice"
}

target_kind and target_name are the conversation-target scope — the only tenancy the platform has. A subject never crosses this pair.
kind names the subject family the state serves. Kind person is the one platform-validated family: its key is a person id resolved against the conversations identity store, and the resolved person’s target must equal the subject’s target. Every other kind is declared by the state and its key is opaque to the platform.
key addresses one subject within its kind. It is trimmed and must be 1 to 512 characters.

The store refuses a subject loudly (422) when its rule fires: an undeclared kind, an empty key, an unknown person, or a person whose target does not match the subject’s. A subject kind is a lowercase identifier of at most 63 characters (^[a-z][a-z0-9_-]{0,62}$).

Declarations and kinds

A state is declared before it is written. A declaration names the state, its base JSON Schema, the subject kinds it serves, and the default kind a door’s ambient subject resolves to:

{
  "name": "status",
  "description": "Per-person status document.",
  "schema": { "type": "object", "properties": { "note": { "type": "string" } } },
  "subject_kinds": ["person"],
  "default_subject_kind": "person",
  "retention_days": 90
}

subject_kinds is a non-empty, unique list; each entry matches the subject-kind charset above.
default_subject_kind must be one of subject_kinds. It is the kind a door’s ambient subject resolves to when no explicit kind is named (see Doors).
retention_days is optional: a positive integer, or unset to keep records forever.
A state name matches ^[a-z0-9][a-z0-9_-]*$ — no colon, because the write op-id namespace splits on :.

Two schema fields are computed by the platform and served on every read: effective_schema (the base schema composed with every attached template’s fragment) and regimes (the absolute write-regime rules composed over the attachments). A client that supplies a non-null value for either on a write is refused — neither can be forged across the wire.

Re-declaring

Once records exist, a re-declare accepts only additive schema changes: a re-declare that would remove or change an existing field while records exist is refused (409), and removing a subject kind still present in records is refused the same way. retention_days is metadata, not schema, so changing it alone is never gated.

Templates and attachments

A state template is a reusable schema fragment with fillable parameters, per-path writer regimes, attach-time declarations, a set of template jq programs, an optional reconcile program, and a trace switch. It is a document the platform owns:

{
  "kind": "state-template",
  "name": "utilities",
  "description": "A reusable fragment.",
  "parameters": {},
  "schema": { "type": "object", "properties": { "items": { "type": "array" } } },
  "regimes": [{ "path": ["items"], "regime": "composing" }],
  "template_jq": {},
  "reconcile": null,
  "trace": { "enabled": true }
}

A state template is data and the store is generic: nothing in a template is a use-case feature — it is a mechanism any state can reuse. A name is a slug — ^[a-z](?:[a-z0-9]|-(?=[a-z0-9])){0,62}$, so no leading, trailing, or consecutive hyphens — because a template name is a URL path segment (/api/state-templates/{name}). A document that carries keys outside its own schema is refused, so an operator who pastes a whole consumer document is told exactly where those belong. Every section is typed:

Section	Shape
`parameters`	`{name: {schema, default?}}` — `default` is absent when the parameter has none (an explicit `null` is a default)
`regimes`	`[{path, regime}]` — `path` is a list of non-empty keys or `"*"` (`[]` is the root); `regime` is `single`, `composing`, or `free`
`trace`	`{enabled}` — a boolean, `false` when the template declares no switch

The studio page of the state template display on its Template tab: the Fields schema, a Write policies table with the path density under the single regime, a Parameters table with default_density, its enum schema of compact and comfortable and the default comfortable, and the line Trace writes: On. — A state template in the studio, each section shown typed.

A shipped default template (a seed) is validated exactly like an upload; a seed that does not validate stops the boot, naming it.

Rendered templates

The platform also serves a template rendered: every program body (inline or a stored id) as jq text, a stored fragment as its schema, and the input programs in dependency order (input_order). The states facet serves it three ways — get_rendered_template(name), rendered_attachments(state) (every attachment on a state with its rendered template), and render_template(document) for a candidate not yet stored. Each carries an opaque version that changes whenever anything the rendering read changes; compare it for equality only.

Template jq

A template’s template_jq is a map of named jq programs a door or another consumer runs against a subject’s record, each declaring a purpose:

input — {description, purpose: "input", params, jq}. jq runs over the record subtree at the attach path (. is the attached data) with the effective $parameters, the $declarations, and its declared params delivered as the single object $params, and returns a value. Read-only.
update — {description, purpose: "update", params, reads, writes, jq}. jq runs over the record’s attached subtree (. is the current data at the attach path) with the caller’s argument bound as $input and $parameters/$declarations bound, and returns an ordered, template-relative op batch (the keyed-op shape the store applies). params are the key names the $input object must carry — uniform with an input program’s params — so apply refuses an $input missing a declared key or carrying an undeclared one (422); a named update that declares params needs an adapter on the binding’s update to fill them: a jq over the run’s output, with the run input bound as $input, that builds the program’s $input object. reads/writes are template-relative paths declared for readability; at put each reads/writes path is validated only for structure — that it resolves against the fragment — while the write regime (that a writes path lies under a declared-writable regime) is enforced at apply by the store guard.

A program is named by reference, resolved by one rule wherever a reference is used — an evaluate or apply call over HTTP, MCP or the CLI, a binding validated or saved on a door or as a plugin’s per-node binding (a Babelfish node, for example), and a binding’s run:

<template>.<name> names that template’s program. The template must be attached on the state and declare the program, else 404.
An unqualified <name> resolves to the one template attached on the state that declares it. None declaring it is 404; two or more is 422, naming them and asking to qualify it as <template>.<name>.
The program’s purpose must be the caller’s: an evaluation or an input injection needs an input program, an apply or an update an update program; a mismatch is 422 (template_jq 'x' on template 't' has purpose 'update', needs 'input').

A door binding validated without attaching (a dry run) resolves as though each of its own templates not yet attached were attached at [<template>], so the reference it accepts is the one its runs resolve. A plugin’s per-node binding resolves through the same rule (Babelfish). The states facet serves the rule as resolve_template_jq(state, name, *, purpose, declared=()), returning a ResolvedTemplateJq (template, program, purpose, params, path); declared names templates a caller is about to attach. resolve_template_jq_batch(refs) resolves a list of TemplateJqReferences (state, name, purpose, declared) in one call: one statement reads every named state’s declaration version, and each reference resolves against its state’s catalog at that version. A write that lands on a state during the call can make any reference on that state — the first one too — read the catalog’s current rows and resolve against the newer version. Its answer holds one item per reference, in order: the ResolvedTemplateJq, or the StateNotFoundError / ValueValidationError that resolve_template_jq raises for that reference, with the same message, so one refused reference never hides the others’ verdicts. Sibling input programs are ordered dependency-first, so one may call another by name. A binding on any door (or a plugin’s per-node binding, such as a Babelfish node) injects an input program’s value into a run’s input before the run and applies an update program’s batch after it; the record-level HTTP and CLI surfaces evaluate an input program and apply an update program directly (see Template jq). A binding applies its whole update set in one transaction — every engaged state’s updates land together, and if any update fails the run’s writes roll back as a unit. The atomicity is automatic; nothing configures it.

Attaching

Attaching a template places its fragment at a path in a state’s document. The template is named in the URL segment (PUT …/attachments/{template}), never in the body; the body carries only path/parameters/declarations/options (the last a per-operation, never-stored bag):

{ "path": ["extras"], "parameters": {}, "declarations": {}, "options": {} }

The effective schema is the base schema with every attached fragment composed in. Composing two fragments at overlapping paths, attaching a second template at an already-occupied path, or attaching where the base schema already carries the property is a conflict (409) naming the colliding path. A fragment’s local $refs keep the fragment’s meaning once it is placed: a pointer reference (# or #/…) is rebased to the attach path, so #/$defs/t in a fragment attached at ["box"] reads #/properties/box/$defs/t in the effective schema, while an anchor reference (#name) is left as it is. What a fragment may carry:

With its parameters filled in, the fragment must be a valid draft 2020-12 schema.
At a path, a pointer reference whose target sits inside a const or enum value is refused: that value is data a record must equal, and rebasing the reference would change it.
At the root (path: []), the fragment’s root type must be object, and its root may carry only properties, required and $defs, which merge into the state’s root (a key the root already holds is a conflict, as a property is). Annotation keywords there (title, description, $comment, examples, default, deprecated, readOnly, writeOnly, and a $schema naming draft 2020-12) are dropped; any other root keyword is refused. A pointer reference must name a target under #/properties/<key> or #/$defs/<key>; # or any other root pointer would name the state’s root once merged, so it is refused. Attach such a fragment at a path instead.

The effective schema then meets the same rules as a declared schema. It is held to the same JSON Schema rules as a form’s schema, and it is one draft 2020-12 document, so:

a value a validator cannot use is refused: a pattern that does not compile, patternProperties keys that cannot be combined, a bound that is not a finite number;
every $ref resolves to a valid schema: #, a #/… pointer, a #anchor or an absolute reference under the root $id, or into a JSON Schema meta-schema by its URI (no schema is ever fetched); a #/… pointer indexes an array only with 0 or a whole number without a leading zero;
$dynamicRef, a $schema naming a dialect other than draft 2020-12, and a $id below the root are refused, and so is a root $id that is a meta-schema’s URI;
an anchor name is unique across every $anchor and $dynamicAnchor in the schema, and a $dynamicAnchor below the root that a meta-schema a reference leads to would jump back to is refused, as for a form;
a cycle of references that never descends into the document is refused.

These rules are checked whenever an effective schema is composed, so a declaration, an attach, a declarations write, a detach or a template replace whose result breaks one is itself refused. A template’s parameter schema and its declarations.schema are held to the same rules: each must be a valid draft 2020-12 schema whose $refs resolve by the rules above. A template breaking this is refused when it is saved, previewed or seeded. A consumer may register an attach validator on the states facet with register_attach_validator: given the state name, the template document, the attachment’s declaration values, and the state’s effective schema, it raises loudly to refuse an attach or a declarations write — it is consulted before every attach, every declarations write, and every template replace (once per live attachment).

Attach reconciliation

Replacing a live attachment’s declarations can retire part of a template while the state still holds OPEN records shaped by the old declarations. The platform runs a built-in reconciler that reads the template’s reconcile program, and a consumer may register its own with register_attach_reconciler(reconciler) — a pre-write hook run inside every attach and update_attachment_declarations write, after the validators and before the write. It receives an AttachReconcileContext: the state name, the template document, the operation (attach | update_declarations), the previous_declarations (None on a first attach) and new_declarations, the attach options, and a record door bound to the state. The reconciler either raises to refuse the attach (naming the records the new declarations would orphan) or writes resolutions through the context’s record door and returns, letting the attach commit with those writes. The options bag on the PUT/PATCH attach body (AttachBody.options, --options on the CLI) is a free-form, per-operation directive bag the reconciler reads to decide how to settle open records against the new declarations — for example, whether to close records the new declarations no longer cover. It is passed to the reconcilers for THIS attach only, never stored or served back; any key outside path/parameters/declarations/options is refused.

Detaching

Detaching a template is guarded so a live reference is never stranded. The detach door consults every registered detach referee before it removes the attachment and refuses (409, listing the human-readable references each referee returns) while any binding still calls the template’s template jq. The platform’s own binding holders — preset versions, conversation-route configs, hooks, and schedules — are checked, and any consumer that registers a referee is too: the Babelfish flows engine, for example, registers one for its per-node bindings, just as it does against a state delete. Nothing is auto-detached — the author removes or rewrites the referencing bindings first, then detaches. A consumer registers a referee with register_detach_referee, beside register_delete_referee on the tools facet. A detach referee is a coroutine called with the state’s name and the template’s name; it answers a list of ReferenceHolder entries, each carrying the feature tag its holder lives under (see Plugin referees). The detach preview door (GET /api/states/{name}/attachments/{template}/referees, tai states attachment-referees on the CLI) runs the same gather and changes nothing. It answers {"action": "detach", "holders": [...]}, where each holder entry carries its kind, its tag, its name, a locator and a one-line summary, so a UI can link each one to the place that fixes it. The preview and the refusal show the caller’s view: the holders under a tag the caller’s role cannot read come back as one count per kind and tag (see Holders a role cannot read). Beside the referees, a consumer that keeps rows keyed on a tool name registers a delete cascade with register_delete_cascade. A cascade is called once for each tool a preset delete removes — the preset and each extension branch tool bound off it — after every veto has passed and before the record is removed.

Regimes

A regime is a per-path writer rule composed over a state’s attachments. Each rule is a {path, regime} where the path may use the "*" wildcard (one list index or key) and the regime is one of:

free — no restriction. An undeclared path is free.
composing — the path is built up by many writers, so a whole-path set/remove over it is refused (422); it admits only a keyed op or an append set. The refusal names the path and template, for example: composing path ['items'] of template 'utilities' admits only keyed ops or an append set.
single — a single-writer path. This is enforced as an authoring check when a consumer binds a state, not at the store: a hook, schedule, or builtin-tool write on a single path is recorded in the audit trail with its door and actor, never refused at write time.

When a template’s trace switch is enabled, the effective schema admits a _trace object on every write under the attachment, and the platform stamps it (see The audit trail).

Doors

Every door that can write a subject’s document deposits one ambient StateContext — the entering door, the resolvable subject candidates, and the accountable actor — so a downstream write resolves its subject and provenance without a per-door argument. The context is one generic object, so a run parked at an async ask and resumed later rejoins the same attribution.

Door	What it supplies	Explicit subject
Conversation turn	door `conversation`; the route’s target scope; candidates `{"thread": <thread id>}` always, plus `{"person": <person id>}` on a multichannel target; actor = the turn’s attribution user id; the turn id and inbound id	a binding’s `subject_expr` reads the `turn.subject` block; a tool takes a `subject` argument
Park → continuation	the original turn’s `StateContext`, stored whole beside the continuation identity and re-deposited when the run resumes	—
Hook fire	door `hook`; the declared target and `{kind: key}`; actor = the hook’s execution key	a hook’s `tool_kwargs` may carry `subject`
Schedule fire	door `schedule`; the job’s `subject` under its target; actor `null` (the fire is anonymous)	the job’s `subject`
Builtin tools	resolved from the ambient context, or from an explicit `subject` argument	`subject` argument
HTTP API	door `api`/`operator`; actor = the request principal	the subject is in the path

A tool run with no ambient context — a sync run-tool call, a background run, the MCP edge — is the api door and a state tool it calls must carry an explicit subject; called with none it raises no subject in scope.

Subject references

A binding’s subject_expr result and a builtin tool’s subject argument are both a subject reference, resolved by one platform rule — the states facet’s resolve_subject(state, ref):

Reference	Resolves to
`{target_kind, target_name, kind, key}`	that subject, validated
`{kind, key}`	the ambient context’s target, with that kind and key
a key string	the ambient target, the state’s `default_subject_kind`, that key
omitted	the ambient target and the context’s candidate for the state’s `default_subject_kind`

A reference that cannot be resolved is refused (422, SubjectRefusedError) naming the rule that fired: a mapping giving only one of target_kind/target_name, no ambient context where the form needs one, a blank key, no ambient candidate for the default kind, a mapping that does not validate (naming each failing field), or any other shape. An undeclared state is 404: a key string or an omitted subject reads the declaration’s default_subject_kind and is refused there, while an object form resolves without reading the declaration and the record read or write it addresses refuses it. A builtin tool’s subject argument takes the two object forms or is omitted.

A binding’s subject and scope

A binding — on any door definition (a preset, hook, schedule, or channel route) and on a plugin’s per-node binding (a Babelfish node, for example) — resolves its subject with a jq subject_expr and gates itself with an optional jq scope_expr. Both mean the same on every door and every node:

subject_expr is required and non-empty, and yields a subject reference: a full subject object {target_kind, target_name, kind, key}, a {kind, key} object, or a bare key string — for a bare key the kind is the state’s declared subject kind; for a key or a {kind, key} object the target is the run’s ambient target. A null result, like a missing or empty subject_expr, is refused (422), and so is a reference that does not resolve.
scope_expr is an optional boolean predicate over the run’s inputs, evaluated first: false skips the state for that run — no input injections and no updates; a non-boolean is refused loudly; an absent expression means engaged.

When a door dispatches a preset that carries its own binding, the door’s scope_expr wins when present, else the preset’s. A preset’s own binding fires only when that preset is the outermost dispatch; called as a sub-tool of another dispatch, its binding is a no-op. A binding fires on every door it reaches through, the MCP edge included: an MCP tools/call that omits arguments still receives the binding’s input injections — absent arguments are treated as an empty argument object, never a skipped injection.

Saving a binding

A binding is checked when the definition carrying it is written — a hook registered, a schedule created, a preset created, saved as a version, or rolled back, a per-target conversation config set. Each of these has one write service that the HTTP door and the backup restore both run: it compiles the binding’s jq slots, resolves each state’s named template jqs in one resolve_template_jq_batch call, and attaches every named template (attach-on-use, idempotent) before the definition is stored. A refusal stores nothing. At a door, the refusal maps by its exact error type:

Refusal	Status
The binding names an undeclared state, a template that does not exist, or an unknown template jq	`404`
A named template’s attach path is already occupied	`409`
A jq slot that does not compile, or binding content the store refuses	`400`
The states store is unbound or fails	`500` — a store failure is never reported as a bad binding

The preset validate dry run runs the same checks without attaching, and reports a refused binding as an invalid verdict; a store failure there is a 500 too, never an invalid verdict. In a backup restore the states section replays after storage (with access_control and sub_mcp between them) and before every section that restores a binding, so the states and templates a restored binding names exist by the time its definition is written.

The turn subject block

Every tool turn’s payload carries a generic turn block so a binding’s subject_expr can read the subject the platform resolved:

{
  "id": "<turn id>",
  "inbound": { "id": "<inbound id>", "kind": "message", "source": "<source>" },
  "subject": {
    "target_kind": "agent",
    "target_name": "A-42",
    "person": "alice",
    "thread": "<thread id>"
  }
}

person is null off a non-multichannel target (a person is resolved only for multichannel targets); thread is always present.

Hook and schedule subjects

A hook may declare an optional subject: the target scope, a subject kind, and a key_expr — a jq expression evaluated over the event payload at fire, which must yield a non-empty string, else the fire fails loudly like any hook error. A schedule carries its subject in the job’s tool kwargs; a malformed subject is refused when the job is created (invalid schedule subject: …), so a job that could never resolve its subject is never persisted. Both surface a Subject group on their Studio forms; see Hooks and the scheduling reference.

The builtin tools

Four LLM-facing tools read and write the calling subject’s document. Each resolves its subject from the ambient context, or from an explicit subject argument ({target_kind, target_name, kind, key}, or {kind, key} with the target taken from the context), by the same rule as a binding (see Subject references); with neither resolvable it refuses, naming the state, kind, and door.

Tool	Effect
`state_read(state, subject=None)`	read the subject’s document (a miss returns null)
`state_replace(state, data, subject=None)`	replace the whole document
`state_merge(state, patch, subject=None)`	shallow top-level merge of a patch
`state_apply(state, ops, subject=None, op_id=None)`	apply a batch of path operations

The tools supply only the consumer half of a write’s provenance (their tool name, run, and — for state_apply — the idempotency op_id); the door, actor, and turn id are stamped by the platform, never derived by the tool.

The audit trail

Every write is recorded. A state’s writes trail is a keyset page {items, next_cursor} — the same paging idiom as subjects and record search — with each items row carrying the write’s seq and timestamp at, the completed origin that produced it, and the absolute paths it touched. next_cursor feeds the next call (?cursor=) and is null on the last page. The origin carries what the consumer supplied — consumer, meta (an opaque object the writing consumer supplies — its own identifiers, for instance — stored and shown as is), run_id, op_id — and, stamped by the platform write chokepoint from the ambient context, door, actor, turn_id, and inbound_id. Because door, actor, and turn are stamped by the platform, the trail cannot be forged by a consumer. Under a traced attachment the platform stamps a _trace object on every write: at is always present; run, turn, and inbound are the run, turn, and inbound ids, or null when the writer has none (a hook, a schedule, the API, or a builtin state_* tool has none); and meta is an opaque object the writing consumer supplies — its own identifiers, for instance — stored and shown as is. A plugin consumer’s binding write and a hook’s keyed write get the same stamp.

Unit of work

A consumer that stages several writes and lands them together opens a unit of work over the states facet, bound to the scope it owns. Every write set staged against the unit runs through the same applier the persisted write path runs — the guards, the path ops, the composing-shape rule — and is projected, not stored. The whole-document schema is checked once per touched subject per staged batch, on the document the batch leaves: a batch is accepted or refused exactly as one multi-op apply would be (an intermediate document between two of its writes is never checked on its own). A validation, composing-shape, or schema error raises at stage time and leaves nothing of the refused batch staged. While the unit is open, a read of a subject the unit has staged is served from the projection — the committed document overlaid with the scope’s staged writes in order, so a later step reads its own earlier writes — while every other scope still sees only the committed document. commit writes the unit’s validated intent — every staged batch, plus any calls deferred with defer_call — as one pending save in one small transaction. On the serving loop commit returns before the records are applied, so the run’s reply never waits on the record writes; the save is applied after the reply (see Pending saves): every staged subject lands in one transaction, all-or-nothing, and the save is removed in that same transaction. A subject whose committed record and declaration are unchanged since it was staged gets its already validated projected document written as it stands; a subject whose record, declaration, or op_id ledger moved in the meantime has its writes replayed onto the latest committed document instead, under the declaration as it stands at apply: a subject kind it no longer declares is refused, and the schema is checked once per staged batch. The guards and op_id idempotency are authoritative at apply; a replay whose answer differs from the staged projection is logged and counted. commit returns the staged answers, the pending save’s id, and the number of deferred calls. A unit that staged nothing and deferred nothing writes no save. discard drops the staging. A unit left open when its scope ends is discarded at teardown and the discard is logged. A committed or discarded unit refuses any further stage, savepoint, commit, discard, or deferred call with StateUnitClosedError (a RuntimeError). defer_call(tool, arguments, *, run_id=None) queues a tool call to run after the save’s records apply. The arguments must be plain JSON and the tool must exist, or the call is refused at once with DeferredCallRefusedError. The call runs later under the identity, run attribution, state context, and trace the caller had when it deferred it, as an outermost tool dispatch (so authorization, retry policy, run recording, and tracing all apply), with the idempotency key <save id>:<index> in its extras under tai42/idempotency_key. A discarded unit, or a savepoint rolled back, drops the calls it deferred. A savepoint nests inside a unit: writes staged in the child scope are kept on a clean exit and dropped on an exception, rolling back only the child’s writes and never the parent’s. Savepoints nest to any depth, so a nested run rolls back its own writes within the enclosing unit. The unit of work is a programmatic facet API for in-process consumers; it has no HTTP route.

Caches and consistency

Each process keeps derived artifacts of the catalog — a declaration’s effective schema and its document validator, its composed regime paths, a template’s validated form, and its rendered programs. Every declaration and template row carries a version drawn from one database sequence on each write to it (an attachment write and an attached template’s replace included), so a version never repeats for a name, even when the row is deleted and re-created. Each artifact is served only for the version read inside the statement that serves it, so a change made through any process is seen by every other on its next read. A rendered program also keys on the resource manager that rendered it, so a stored-resource upload, a cache eviction, or a reload re-renders; a rendered template’s version also changes whenever a re-render (after the template cache’s TTL, or on every read while that cache is off) produces different text.

Pending saves

A pending save is the durable intent a unit commit writes before the reply: the staged writes of every subject the run touched, and the calls it deferred. It is applied after the reply, and every reader of its subjects waits for it, so a participant’s next message reads its own previous turn. Who writes one. A unit of work’s commit, a door binding’s updates on a tool call, and a parked run’s deferred binding updates each write one pending save. Applying. On the serving loop the save is applied in its own task right after the commit returns. Off it — a backend whose jobs run on their own loop, such as Celery — the records are applied inline and the deferred calls are left to the next sweep pass; a backend whose jobs run on the process’s serving loop, such as ARQ, runs them in the worker right after the job. A sweep runs in every serve process and backend worker every STATES_OUTBOX_SWEEP_SECONDS, and applies any save left behind (a crash, a lost task, a retry that is due). Per-subject order. Saves that share a subject apply their records in the order they were written. A save applies its records and removes itself in one transaction; when it carries deferred calls it moves on to them after the records land, and its calls start only once every older save on its subjects is gone, so deferred calls on a subject run in order too. The records part is applied exactly once. A deferred call runs at least once: a call interrupted by a process exit is re-run only when its tool declares tai42/crash_resume, and otherwise the save fails. Readers wait. A read, write, fold, erase, template-jq call, or write-history read of a subject first applies the subject’s pending save (or waits for the one being applied), bounded by STATES_OUTBOX_DRAIN_TIMEOUT_SECONDS. When nothing is pending this costs no extra round trip: the check rides on the subject lookup the read already makes. A subject’s write history includes its pending save. A tool call, a hook or schedule fire, a parked run’s continuation, and a turn also apply their subjects’ pending saves when they start, and wait for those saves’ deferred calls to finish, so a run never starts behind its subject’s earlier save. Failures. A transient failure (a lost connection, a timeout, an unavailable dependency) is retried with an exponential backoff from STATES_OUTBOX_RETRY_BASE_SECONDS, capped at STATES_OUTBOX_RETRY_CAP_SECONDS, up to STATES_OUTBOX_MAX_ATTEMPTS attempts. Any other failure, or the last attempt, marks the save failed, and the failure is loud: an ERROR log, an entry in the operator notifications inbox, the states_outbox_save_failed platform event (see Hooks), and the tai42_state_outbox_* metrics (see Observe).

Held subjects

A failed save holds its subjects until an operator retries or discards it. A newer save that shares a subject with it is held behind it. A read or write of a held subject raises StatePendingSaveFailedError (409, naming the save) rather than serve a record that silently lacks the save; a wait that runs out raises StatePendingSaveTimeoutError (503). The run-tool door answers the same 409 / 503, and a turn that meets a held subject fails through the route’s ordinary error reply. On the agents stream the stream.error frame names the held subject. The scans over a whole state treat a held save in one of two ways:

Scan	Held save
backup export and restore of the state, `fold`, `erase`, deleting the declaration	refused, naming the save
`list_subjects`, `search`, `stats`, retention prune, attach, updating an attachment’s declarations	skipped: the scan serves the committed records, logs a WARNING, and names the held saves in its `held` field
re-declare (`PUT /api/states/{name}`)	counted: the held saves’ records count as present, so a re-declare that would invalidate one, or remove a subject kind one writes, is refused (`409`, naming the saves); any other re-declare is accepted and names the held saves in `held`

A widening re-declare followed by a retry therefore repairs a save that failed on the declaration. A tool rename is refused while any save is outstanding under the tool — pending, still running its deferred calls, or failed — and the refusal names each one. The sweep applies a pending save; a failed one needs a retry or a discard. The rename preview lists the same saves and applies nothing.

Retry and discard

Method & path	Purpose
`GET /api/state-pending-saves`	list outstanding saves, newest first (`?status=` `outstanding` or `failed`, `?limit=`, `?cursor=`), with the totals of outstanding and failed saves
`POST /api/state-pending-saves/{id}/retry`	retry a failed save: its attempts reset and its records apply at once; answers its status after the records phase
`DELETE /api/state-pending-saves/{id}`	discard a failed save: its writes and calls are never applied and its subjects are released

Only a failed save can be retried or discarded (409 otherwise). A listed save carries its status (pending, calls, running, or failed), run, states, subjects, the calls it runs (kind and target), attempts, last error, failed phase, and times — never its record data or call arguments. A discard is logged at WARNING with the principal who made it and writes an operator notification. The same three operations are MCP tools, and tai states pending-saves list|retry|discard on the command line (discard requires --yes). The Studio’s States screen shows them in its Pending saves card.

Checking records

A record is validated against its state’s effective schema — the base schema composed with every attachment — on every write. Two read-only doors check that same conformance without writing, so a schema can be checked before a document is stored and after a declaration or attachment changes what the schema admits.

A candidate, before it is stored. POST /api/states/{name}/records/validate takes a candidate document as its whole body and checks it against the state’s effective schema exactly as a write would, but stores nothing. The answer is a RecordValidation — {fits, reasons}. A document that fits is {fits: true, reasons: []}; one that does not is a 200 with fits: false and every reason a {path, message} ordered by JSON path, so a caller sees all of a document’s misfits at once rather than only the first. Only a non-object body or an effective schema with a broken $ref is a 422, and an undeclared state a 404.
The stored records, against the schema as it stands now. GET /api/states/{name}/records/check pages the state’s stored records and reports the ones that no longer fit the current effective schema — the records a re-declare or an attachment change left behind. The answer is a RecordCheckPage — {checked, items, next_cursor, held}. checked counts the records the page read, and items are the misfits only, each a RecordCheckItem (subject, updated_at, and the same path-ordered reasons). A page that read records but found none wanting therefore carries an empty items beside a non-zero checked. It pages like the subjects listing and record search — a keyset ?cursor=, an optional ?kind=, and a ?limit= — and held names the subjects a failed pending save made the scan skip.
Without a store. The skeleton’s RecordValidator checks a document against a state’s effective schema the same way the candidate door does, with no server: build it from the effective_schema a state’s read serves, or compose one with compose_effective_schema from the base schema and the attachments, then call validate. The answer is the same RecordValidation, with the same path-ordered reasons. Compiling the schema is the costly step, so build the validator once and reuse it for every document.
from tai42_skeleton.states import RecordValidator from tai42_skeleton.states.templates import compose_effective_schema, validate_template # The `effective_schema` from GET /api/states/alerts, or composed from its parts: template = validate_template(template_document) effective_schema = compose_effective_schema(base_schema, [(template, ["box"], {})]) validator = RecordValidator(effective_schema) result = validator.validate({"n": 1, "box": {"tag": 5}}) # With a fragment that declares `tag` a string: result.fits is False and # result.reasons[0].path is "$.box.tag".
Building the validator checks the schema by the declaration rules (above) and raises SchemaValidationError for a schema they refuse. validate raises ValueValidationError for a document that is not a JSON object, and for a $ref it cannot resolve while validating.

The two doors are also on the command line — tai states validate <name> (--data / --file) and tai states check <name> (--kind) — and the Studio’s States screen runs the stored-records check from a card on a state’s Records tab.

The HTTP surface

All routes are authed. Templates are a top-level sibling collection and prune is its own route, so no literal segment sits on the /api/states/{name} position — a state literally named templates stays reachable at GET /api/states/templates while GET /api/state-templates lists templates.

States

Method & path	Purpose
`GET /api/states`	list declarations
`GET /api/states/{name}`	declaration, effective schema, attachments, regimes
`PUT /api/states/{name}`	declare (`?replace=`)
`DELETE /api/states/{name}`	delete a declaration
`GET /api/states/{name}/stats`	record and subject counts, and the held saves
`GET /api/states/{name}/attachments`	list attachments
`GET /api/states/{name}/attachments/{template}`	read an attachment (404 if not attached)
`PUT /api/states/{name}/attachments/{template}`	attach a template
`PATCH /api/states/{name}/attachments/{template}`	update an attachment’s declarations
`DELETE /api/states/{name}/attachments/{template}`	detach
`GET /api/states/{name}/subjects`	list subjects (`?kind=&limit=&cursor=`)
`POST /api/states/{name}/records/search`	search records by document content
`POST /api/states/{name}/records/validate`	check a candidate document against the effective schema, storing nothing
`GET /api/states/{name}/records/check`	page the stored records that no longer fit the current effective schema
`GET /api/states/{name}/consumers`	what binds this state

Records

The subject is four path segments: /api/states/{name}/records/{target_kind}/{target_name}/{kind}/{key}.

Method & path	Purpose
`GET …`	read the record
`PUT …`	replace the document
`PATCH …`	shallow-merge a patch
`POST …/deltas`	apply a batch of path operations
`DELETE …`	erase the record
`POST …/fold`	fold this subject into another
`GET …/writes`	the audit trail

Template jq

The record-level jq programs a state’s attached templates declare share one path, the method selecting the purpose:

Method & path	Purpose
`GET …/template-jq/{program}`	evaluate an `input`-purpose program (query params supply its `params`)
`POST …/template-jq/{program}`	apply an `update`-purpose program (body `{input?, op_id?}`)

{program} is a program name, or {template}.{name} when a name is shared across the state’s attachments. A GET on an update program, or a POST on an input program, refuses loudly.

Templates and retention

Method & path	Purpose
`GET /api/state-templates`	list templates
`GET /api/state-templates/{name}`	read a template
`PUT /api/state-templates/{name}`	upload a template (`?replace=`)
`DELETE /api/state-templates/{name}`	delete a template
`POST /api/state-retention/prune`	prune expired records

Errors map to status: 501 when the store’s database is unbound (a stable code states-not-configured), 404 not found, 409 exists / in use / conflict / a subject held by a failed pending save, 422 validation (schema, subject, regime, path, value), and 503 when a subject’s pending save is still not applied after STATES_OUTBOX_DRAIN_TIMEOUT_SECONDS.

The command line

Two command groups mirror the HTTP surface. tai states manages declarations, attachments, subjects, records, and the audit trail — list, get, put, delete, stats, attachments, attach, get-attachment, update-attachment (its --options carrying the per-operation reconcile directives), detach, subjects, search, validate, check, read, replace, merge, apply, template-jq (eval an input program, apply an update program), erase, fold, writes, consumers, prune, and pending-saves (list, retry, discard). tai state-templates manages template documents — list, get, put, delete. Record commands take --target-kind, --target-name, --kind, and --key; apply, replace, and merge read their JSON from --data or a --file. See the CLI reference.

Enabling the store

The store is the states DB registry component. Like the skeleton component it defaults to the default database when its binding is unset, so an ordinary deployment serves the store out of the box. A deployment that wants records in a separate database points the binding elsewhere with TAI_DB_BINDING_STATES, then applies the component’s migration chain with tai db migrate. Until the bound database is configured, every read and write returns 501 states-not-configured. Record retention is opt-in: nothing is deleted until a retention is configured, either per state (retention_days on the declaration) or as a deployment default (STATES_DEFAULT_RETENTION_DAYS). tai states prune runs the sweep.

​The subject

​Declarations and kinds

​Re-declaring

​Templates and attachments

​Rendered templates

​Template jq

​Attaching

​Attach reconciliation

​Detaching

​Regimes

​Doors

​Subject references

​A binding’s subject and scope

​Saving a binding

​The turn subject block

​Hook and schedule subjects

​The builtin tools

​The audit trail

​Unit of work

​Caches and consistency

​Pending saves

​Held subjects

​Retry and discard

​Checking records

​The HTTP surface

​States

​Records

​Template jq

​Templates and retention

​The command line

​Enabling the store

The subject

Declarations and kinds

Re-declaring

Templates and attachments

Rendered templates

Template jq

Attaching

Attach reconciliation

Detaching

Regimes

Doors

Subject references

A binding’s subject and scope

Saving a binding

The turn subject block

Hook and schedule subjects

The builtin tools

The audit trail

Unit of work

Caches and consistency

Pending saves

Held subjects

Retry and discard

Checking records

The HTTP surface

States

Records

Template jq

Templates and retention

The command line

Enabling the store