Skip to content
SHC Docs

A drain and a disruption budget

This page sets how much of a deployment a drain may take down, its disruption budget. A drain empties a node for maintenance by moving what runs there to other nodes. A service with no volume moves bring-up-first: its copy on the new node is ready before the old one stops. A member with a volume cannot: its data has to stop changing before it is copied, so the member is down for the move.

kv-3 runs on far01 with its volume. For a replica set that can be fine, or it can be the copy you cannot lose for a minute. The budget is yours, not the app’s: the deploy.disruption config key on the deployment. rolling, the default, lets a drain move one replica at a time, each ready on its new node before the next is touched. never says no drain may move the deployment at all, and a drain of a node it runs on is refused instead of carried out.

home01
$ shc update scores -c deploy.disruption=never
...

Drain far01.

home01
$ shc node drain far01 --yes
far01 is a docker-runtime host.
- the drain is blocked: deploy.disruption is never for scores, so no drain moves it without --force
- 1 deployment must relocate; the other docker-runtime nodes are home01, home02, home03
- traefik runs on every node, so its replica here stops
shc: [X400423CLSDRM] drain of far01 is blocked because deploy.disruption is never for scores, a drain may not move any of their replicas, set it to rolling with shc update <stack> -c deploy.disruption=rolling, drain a different node, or re-run with --force to accept the disruption deliberately, nothing was changed, the node is still schedulable
$ shc node ls
...

The drain is refused before it moves anything, and far01 is still active: scores is set to never, and the move would take kv-3 down. To drain far01 anyway, set scores back to rolling with shc update scores -c deploy.disruption=rolling, or pass --force to accept the outage you were warned about.

Next: A privilege.