Placement: labels, locations, node selectors and service selectors
Status: decided 2026-11-09 (FZ-10323). Clay closed the design on the phone. Each grammar line and each case below says whether it ships today (EXISTS) or which card builds it. A line marked with a card is a target, not shipped behaviour. Behaviour marked EXISTS was read from ce at 1c128d886, 74f8714e75 and d8c4dbc758, not run on a cluster.
The model
Section titled “The model”A label is a flat fact on a node, like pool=ci, or on a replica, like role=primary. It does nothing by itself. You point at it.
Location is where a node is, a path such as hetzner/hel1. You set it with shc node update <node> --location hetzner/hel1, and SHC publishes it as a read-only label holding every prefix, so location=hetzner finds a node at hetzner/hel1. Only location has hierarchy. Labels stay flat, because & already narrows: pool=ci & arch=arm64 says what pool=ci/arm64 would.
Node selectors and service selectors pick things by label, in one grammar. They’re separate kinds. Always write the full term.
- A node selector picks nodes. It’s used inline all the time, in
--where,--preferand an expose’s@(...). Naming one withshc node selector setis optional. - A service selector picks replicas of its own deployment. It’s declared in the app only, and it implicitly creates a VIP and a DNS name.
Every service is one name and one VIP over its replicas. A compose service gets that automatically. A service selector makes another one from a slice of the replicas. That’s the simplification over Kubernetes: no Service objects, and the deployment controls everything.
Decisions
Section titled “Decisions”Clay, 2026-11-09 unless dated.
- Primitives, not features. “a bunch of primitive that support all of these more flexible use cases”, “I don’t just add features”. A use case is a combination of primitives. No flag, key or command exists for one use case.
- Only location has hierarchy. Labels stay flat.
- Reserved is the taint, and the
--wherenaming the value is the toleration.shc label set pool=ci --reserved. No new word. EXISTS. - Names. “How about service selectors and node selectors instead”, “The cli would need to be updated to differentiate the two as well”, “And I fucking told you it’s service selectors which is separate from node selectors”. Always the full term, in keys too, never bare “selector”. This replaces his 10-09 word “alias”. The app key is
x-shc-service-selectors. The CLI isshc node selector set|ls|show|rm, besideshc node label. FZ-10334. - A service selector implicitly creates a VIP. “And service selectors that we create end up implicitly creating a vip”. The name answers from any service of the deployment, and as
<name>.<deployment>.<environment>.<tenant>.shc.internalfrom the tenant’s containers. With no match, or two on a:1, the VIP stays and connections are refused. EXISTS under today’s keyx-shc-aliases. - Service selectors are never inline. “we wouldn’t really create our rarely create service selectors from the cli so inline doesn’t make sense for them especially because it creates a vip but node selectors are common inline for scheduling”.
- Service selectors live in the app only. “deploy files and cli should have full compatibility”, “If no cli then no deployfile support”, “Is that fine where you just use them in the app?”. No CLI sets one and no deployfile key holds one.
- Deployfile and CLI have full parity, as a rule. A deployfile renders into an
shc installorshc updateargv. - Blue/green is one deployment. “Blue green would be in the same deployment the whole part of the simplicity of SHC is that you control everything with your deployment so you don’t have all of these like conventions floating around”. Service selectors don’t reach across deployments.
- Location never goes in an app file. “Why would location be a var?” Location names belong to the estate. The app says which sides exist, the deployment says where each runs with
--where, and a var says which side is live. - Service selectors read service facts only:
service,group,member, compose labels and live labels.locationandnodeare refused there and stay node selector terms. FZ-10334. - Node selectors can follow other services. “shouldn’t location support a way of selecting nodes based on where other services containers are or are not”.
location=@(stack=db),location!=@(stack=db),node=@(stack=db),node!=@(stack=db). Inside the parens picks services, outside is the node selector. It copies the replica’s full location path, solocation!=@(stack=db)with db athetzner/hel1still allowshetzner/fsn1. A named node selector can’t hold@( )(X400266SELGRP). EXISTS. - Any expose can target a name, a service or a service selector, for hostnames and L4 ports. A target that is neither is refused with
X400498EXPTGT. EXISTS, FZ-10335. <n>/<kind>for any kind. “each location might have multiple replicas”. Today the count must be 1. FZ-10325.- Distance is hierarchy. Inside the cluster
nearestis own node, then the deepest shared location level, then every ready replica (nearestince/modules/vip/compile/compile.go). No latency is measured. Latency would at most be a tie-break later. Multicast never. Anycast is an estate network choice that looks like one edge to SHC. EXISTS. - Prefix admission: admit. A placement inside a reserved location counts.
--where location=hetzner/hel1admits a reservedlocation=hetzner, since hel1 is inside hetzner. The reverse stays refused. Today the match is exact (namedByWhereince/modules/fence/engine_rules.go). FZ-10336.
A carried default
Section titled “A carried default”Clay closed the design without answering this one directly. It stands unless he says otherwise.
Geo DNS for outside traffic: not now. It gets built when a real app needs it. It would add one new fact per location, where it physically is. Today public DNS answers one location per deployment, the one holding most of its nodes with ties by name, and passes over a dead edge (entryFor, RankLocations in ce/modules/app/dns_edge_location.go). No card is filed, on purpose.
Grammar
Section titled “Grammar”key=value key!=value & | ( ) @name EXISTSnode=@(sel) node!=@(sel) location=@(sel) location!=@(sel) EXISTS--where must, and the only thing that admits to a reserved value EXISTS--prefer ranks, never blocks, never admits EXISTS<service>:<sel> <service>! ! EXISTS--scale <name>=3 | <name>=2@east,1@west | <name>=1/location EXISTS--scale <name>=<n>/<kind> FZ-10325--expose <outside>=<service or service selector>[:port] EXISTS--where location=<path> admits a reserved prefix of the path FZ-10336shc node label <node> k=v EXISTSshc node update <node> --location cloud/site EXISTSshc label set k=v --reserved EXISTSshc node selector set|ls|show|rm FZ-10334, today shc selectorx-shc-service-selectors: name: "sel[:1]" | {select, route, scope} FZ-10334, today x-shc-aliasesx-shc-live-labels: {role: {test, interval}} EXISTS{{ replica.location }} {{ m.location }} EXISTS<n>/<kind> gives / a second meaning, per, after a bare count (Clay picked it over +per@, rv_602f8db912d8d2dd); a path keeps its / after @. location means each leaf location that holds a node, and pool each value of that label. Today’s spelling for N per location is 2@a,2@b,2@c, which needs an edit whenever a location is added.
The four cases
Section titled “The four cases”One writer, replicas in every location. Needs FZ-10334 for the key and FZ-10325 for the count.
x-shc-service-selectors: postgres: role=primary:1 postgres-ro: role=replicashc install postgres --where pool=db --scale postgres=2/location --expose 5432=postgresEach member declares a role live label. The app’s manager elects the writer and SHC carries the label. With zero or two primaries the name refuses connections, which is the right failure for a writer.
A proxy in the same app dials postgres. A proxy in another app takes the socket’s host or postgres.<deployment>.<environment>.<tenant>.shc.internal. From outside, a connection enters at whichever edge DNS answers and crosses the mesh to the writer. To keep writers in one location, pin the candidates with a node selector, --where 'postgres-1:location=hetzner/hel1', or have the manager only promote there. Never in the service selector.
Nodes only runners and tests may use. EXISTS.
shc node label ci-01 pool=cishc label set pool=ci --reservedshc install gitlab-runner --where pool=ciEverything else stays off ci-01, even when every other node is full.
Spread everywhere, traffic to the closest location. EXISTS inside the cluster.
shc install web --scale web=1/locationNames route nearest by default. Outside, one location answers, see geo DNS above.
Full copies in other locations, flip between them. One deployment. Needs FZ-10334 for the key. The var inside a service selector has never run, and FZ-10334 proves it on a cluster.
x-shc-service-selectors: app: side={{ active }}services: web-blue: { labels: { side: blue } } web-green: { labels: { side: green } }shc install app --where 'web-blue:location=home' --where 'web-green:location=hetzner/hel1'shc update app --var active=greenAn app that replicates its own data can skip the var: app: role=active, a live label that moves when the app promotes the other side. Replicating the data is the app’s job. When a whole location dies, public DNS passes over its dead edge for the next location.
Simple deploy. shc install nextcloud stays one command. No where, no prefer, scale 1, any ready node holding no reserved value. You meet reserved the day you reserve something.
Kubernetes mapping
Section titled “Kubernetes mapping”| Kubernetes | SHC | |
|---|---|---|
nodeSelector, required affinity | --where 'disktype=ssd' | EXISTS |
| preferred affinity | --prefer 'tier=gold' | EXISTS |
| pod affinity, anti-affinity | node=@(stack=db), node!=@(stack=db) | EXISTS |
| anti-affinity between replicas | automatic | EXISTS |
taint NoSchedule | shc label set k=v --reserved | EXISTS |
| toleration | the --where that names k=v | EXISTS |
taint NoExecute | shc node drain | EXISTS |
PreferNoSchedule | none, by ruling | |
| zone pin | --where 'location=hetzner/hel1' | EXISTS |
topologySpreadConstraints | --scale web=1/location | EXISTS |
| N per zone | --scale web=2/location | FZ-10325 |
DaemonSet | deploy.mode: global | EXISTS |
| cordon | shc node cordon | EXISTS |
| Service | every compose service is one name and one VIP | EXISTS |
| Service with a label selector, leader endpoint | a service selector, with :1 | FZ-10334, today x-shc-aliases |
| Ingress, LoadBalancer to any of those | --expose <outside>=<name> | EXISTS |
Considered and not doing
Section titled “Considered and not doing”No use case needs these. Prefer weights or list order. A per-node admission rule, or one requiring every | branch to name the value: the second breaks tier=edge|tier=gpu. SHC_LOCATION in every container, since {{ replica.location }} prints it. shc replica label. A CLI flag or deployfile key for service selectors. Service selectors across deployments. Retiring cluster.keep_workloads_off_voters.
Not verified
Section titled “Not verified”A template var inside a service selector (FZ-10334). Publishing a service selector on a port (FZ-10335). How fast a dead node’s rows leave ObservedNodes, and DNS TTL during a failover. Whether changing one compose service in a deployment leaves the other side running.