ADR-0002: Stack state: three roles, one authority per role (the manifest model)
- Status: Accepted (2026-06-22). Implemented (B1, 2026-07-04). The
single-file collapse + RuntimeState retirement landed earlier (T17):
state.json→manifest.jsonis the single on-disk state file (writers produce manifest.json only; the read-fallback and the prune-on-write that carried the old name are REMOVED as of v0.0.16, measured absent on all six nodes of both live estates, so a dir carrying onlystate.jsonnow reportsX404014MANNOF), and the process-widestack.GetRuntimeState/SetRuntimeStatebridge is retired in favour of the composition root (Services.StackState()/AppStackRepo()). The state-model collapse itself is now done:state_mirror.gois deleted, the in-memoryapp.Stateis demoted to the offline/test store (repo==nil fallback), the appServiceroutes every stack read/write through one activedeploymentStore(the DBdeploymenttable on the daemon, memory offline), node canonicalisation moved to that DB write path, and the 5s disk-reconcile ticker is replaced by a one-shot readiness-gatedImportOrphanManifestsboot import (RegisterRecoveredStackis kept, rewired to a direct store write). Still owed: a fresh 3-node proxmox clone/recover/move +shc ls/statusre-prove on real hardware before declaring the rewrite field-proven. Tracked in DRIFT-REGISTER.md.
Context
Section titled “Context”Stack identity/status was mirrored across four surfaces with no single
authority: the database deployment table, an in-memory app.State map, a
per-stack state.json sidecar, and live docker enumeration. A bridge file
(the since-deleted state_mirror.go in the app module) reconciled them, and its own history is
a catalogue of consistency bugs (empty shc ls over running containers; zombie
running rows resurrected from disk; a 10-minute in-memory tombstone that is “a
mutex re-implemented in wall-clock”). Five fix-commits in four days, each bolting
the mirror onto one more write-path that didn’t know about the others.
A multi-agent steelman-vs-critique judge found that three of the four surfaces are genuinely justified (they play distinct roles), and only the fourth is debt.
Decision
Section titled “Decision”Each surface owns exactly one role; none is a second authority for another’s role:
| Surface | Role | Authority for |
|---|---|---|
database deployment table | cluster-replicated record | identity + status |
state.json → manifest.json | immutable desired-state SPEC (the bytes docker needs), the k8s static-pod-manifest pattern | the offline/daemonless shc install path |
| live docker | observed STATUS, reconciled one-directionally into the DB | nothing it writes; it is observed, not authored |
in-memory app.State map | read-through cache of the DB (NOT a co-equal writer) | nothing |
Renaming state.json → manifest.json is deliberate: the name state invited
treating it as a mutable status store, which grew the dual-authority mess.
manifest names its real role (immutable SPEC) and stops the next contributor
re-treating it as status.
Delete the scar tissue: state_mirror.go’s mirror/MirrorAllToRepo/
ReconcileDiskStacks-as-perpetual-ticker/tombstone/nodeIDResolver; demote the
map to a read-through cache; replace the 5s disk-reconcile ticker with a one-shot
boot import of any manifest.json lacking a DB row.
The one preserved subtlety (the steelman’s real catch): clone reads disk-first because the in-memory map is genuinely wrong after a Rename; the collapse must keep disk-first identity reads for the Rename window. That is a seam fix, not a reason to keep four authorities.
Regression guard (carried from HANDOFF.md, fix 25617e486). The deletion of
state_mirror.go must preserve the cross-node clone/recover node+runtime
recording: RegisterRecoveredStack (app/state_mirror.go) is the authoritative
post-bring-up write that records the target node + real runtime (beating the
disk-reconcile tick), called from services_clone_remote.go (clone) and
services_recovery.go (recover). ReconcileDiskStacks reads Runtime/Node from
the manifest, not a hardcoded compose. When this ADR is implemented, that
behavior moves into the DB-write path; re-verify the cross-node-clone e2e (test 24)
records the right node, and do NOT collapse the swarm-only node-pin gate in
services_seams_clone.go (recording a node for a local fallback would be a lie).
Consequences
Section titled “Consequences”- Offline/daemonless install survives (the CLI still writes + reads the manifest with a nil Service and no DB).
deletebecomes atomic; the tombstone window (where “is this deleted?” has no truthful answer) disappears.- A new write-path (clone/move/recover) writes the DB row in the same operation instead of needing its own mirror patch.
Alternatives rejected
Section titled “Alternatives rejected”- Full database-only collapse (route every CLI write through the daemon, drop offline install): removes a real product capability; rejected.
- Keep the 4-surface model: the in-memory co-equal writer + reconcile/tombstone tax is structural, not a one-time migration cost; rejected.