Failover delivery
Layer 11. Lifecycle. Layer page.
Failover delivery is the leader finishing a deployment’s stored render for the node a bind named. The node does not render.
sequenceDiagram participant Leader participant Store participant Node Leader->>Store: take the deployment relocation lease Leader->>Leader: FinishStoredRender for the bound node Leader->>Store: store the spec and bind replica rows Node->>Store: materialise clusterfs Node->>Node: start replicas from the stored spec Leader->>Store: anchor the deployment on the nodeThe daemon wires that delivery in ce daemon/scheduler_deliver.go.
deliverStoredRender holds the relocation lease, calls
FinishStoredRender, and then anchors the deployment on that node
(deployment_node) while the replica is still bound to it at the
generation the leader read. FinishStoredRender is refused off the leader
(ce modules/app/stored_render.go).
The scheduler tick, while this node leads, converges deployment bindings and
replica rows from the stored spec, then runs registered passes
(ce subsystems/impl/scheduler.go). On this pin nothing calls
scheduler.Deliver or Store.WriteHolds. The delivery function is
wired. The words a failover pass would write are defined
(ce modules/scheduler/holds.go) and read onto listing rows
(ce modules/app/failover_hold.go).
Stacks.Undelivered is a deployment whose replica is bound away
from the node that holds its bytes, with no move under way
(ce modules/scheduler/stack.go). StaleCopies is a node whose lease
is not lost and which still holds containers after the deployment is
bound and recorded elsewhere and runs there: every replica row its
operator wants running reads ready or completed. Until then the copy is
kept, so a replacement that never starts leaves the deployment failed on
its new node with the old copy still up. A lost node’s copy is not read.
Ruled, not on this pin: an automatic volume-failover engine. ce
modules/storage/volume_failover_edition.go says this build ships
none. shc volume rebind is the operator move that remains.
Sources: ce daemon/scheduler_deliver.go, modules/app/stored_render.go,
subsystems/impl/scheduler.go, modules/scheduler/stack.go,
modules/scheduler/holds.go, modules/storage/volume_failover_edition.go.