Skip to content
SHC Docs

The address block

Every address SHC hands out comes from the genesis block, chosen with shc init --address-block <cidr> and carved for the life of the cluster, or from a block the cluster adds later. Unset is 10.64.0.0/16. A block is a /8 at the largest and a /16 at the smallest, written as its own network address. It must stay clear of the docker address pool 172.16.0.0/12 (X400349ADRBLK).

SHC carves the genesis block by fixed fractions, whatever its size. The first eighth is left to the operator’s own networks: SHC never addresses it and never checks a network in it. Service VIPs take the third sixteenth and mesh addresses the fourth. Deployment networks take the second quarter and the fabric the second half. On the default block that gives:

RangeUsed for
10.64.0.0/19nothing: the operator’s own networks
10.64.32.0/20service VIPs
10.64.48.0/20mesh addresses (mesh0)
10.64.64.0/18deployment networks (a /25 each)
10.64.128.0/17the fabric (shc0, /26 chunks per node)

The deployment-network quarter is one budget per bridge owner. A docker, podman or apple node’s bridges are its own, so each such node carves the whole quarter and the default /16 holds 128 deployment networks on every one of them. An Incus bridge is every member’s, so the members of an Incus cluster carve the quarter together: the default /16 holds 128 deployment networks for the whole Incus cluster. A network takes one /25 however many members its replicas run on, and keeps it when a replica moves to another member.

A node takes /26 chunks of the fabric as it needs them, and each of its containers is a routed /32 from them. A chunk is 64 addresses, 62 of them pins. The default /16’s fabric holds 512 chunks, the first never handed out, and every node holds at least one, so it has room for 511 nodes. A /14 holds 2,048.

A node whose own LAN or underlay sits inside one of the last four would lose that network once SHC’s rules converge. The VIP table rejects every packet into the VIP range that it did not rewrite, so a LAN there stops answering. So shc init and shc node join refuse such a node before anything is written: X400450ADROVL at init, X409450ADROVL from the leader. The error names the interface, the network and the range, and suggests a block that fits. shc doctor reports an existing node that overlaps under fabric.addressblock.

A network the node only routes to counts the same way: a site behind a WireGuard or VPN gateway has no address on any interface, and it would be cut off just the same. Every routing table is read, so a VPN client’s own table counts, and the error names the route: route 10.64.35.0/24 via 192.168.1.254 dev eth0. The default route never counts, and neither do the routes SHC installs. A node that rejoins with shc node join --force while its containers run still holds their host ends, each with the node’s fabric address and the route to one container. The leader finds those in the store, so they do not count either; any other network on the fabric range still does.

Pick a block clear of every network any node is on, overlays included, and pass it at genesis. A site with a LAN on 10.64.40.x sits in the default VIP range, so that site runs:

Terminal window
shc init --address-block 10.80.0.0/16

That gives service VIPs 10.80.32.0/20, mesh addresses 10.80.48.0/20, deployment networks 10.80.64.0/18 and the fabric 10.80.128.0/17, with 10.80.0.0/19 left alone. Any /8 to /16 the site doesn’t use works, 192.168.0.0/16 included. Don’t use 100.64.0.0/10: carrier-grade NAT and the overlays nodes already run live there.

The block is recorded at genesis and every joiner takes it from the leader. Nothing renumbers a live cluster.

A cluster that runs out of room adds a block:

Terminal window
shc cluster block add 10.81.0.0/16

An added block is one IPv4 block, a /8 to a /26, and it is not carved. It is one pool of /26 chunks that node fabric chunks, service VIPs and mesh addresses are drawn from once the genesis ranges are full. An Incus cluster’s deployment networks draw on it too once the genesis deployment-network quarter is full, a /25 (two chunks) at a time, so an Incus cluster whose quarter is full installs and moves again after an add. A docker or podman node’s fabric network holds every added block, and an engine refuses a deployment network inside a network it already has, so those nodes keep to the genesis quarter’s 128 per node. The add is refused when the block shares an address with the genesis block, a block already added, the docker pool, or a network or route on any node, and the refusal names the overlap. Every node is asked for its networks, so the add is refused while a node cannot be reached. A block is never removed or shrunk, so pick one no site LAN, VPN or cloud network will want. shc cluster block list prints the genesis block, then every added block with how many chunks it holds, how many are held and how many are left.

shc doctor’s fabric.addressblock and shc cluster fabric status read a pool amber at 80% held and red when full. A genesis pool that draws on the added blocks, the fabric, service VIPs, mesh node addresses and an Incus cluster’s deployment networks, is neither while an added block still has room: the next address comes from there, and the added block’s own line says how much is left. Mesh client addresses and a docker or podman node’s deployment networks never leave the genesis block. On an Incus cluster the deployment line counts the members’ networks together.

Terraform sets it with address_block on shc_cluster, the bootstrap shc_node, the shc_cloudinit data source (role cluster) and the core module’s bootstrap.address_block.

0.1.5 changed the carve. A cluster formed by an earlier release holds mesh addresses outside the mesh range its block gives now, so it cannot be upgraded in place. Form a new cluster with shc init and move the apps over.