Node module
Layer 2. Core. Core The module is the node record and voter slots, not the container runtime.
SHC supports highly available multi-node deployments using distributed consensus for fault tolerance.
Architecture Overview
Section titled “Architecture Overview”Every node runs the full shc server process. There is no separate agent.
A Linux node can carry mesh, fabric and voter roles. A macOS node carries
mesh only, using userspace WireGuard on a utun. The voter target is the
largest odd number no greater than the eligible control-plane members,
capped at five; other members are non-voters. Sources: CE
core/hostplat and modules/cluster/voters.go.
Database Strategy
Section titled “Database Strategy”| Mode | Database | Overhead | Use Case |
|---|---|---|---|
| Single-node | SQLite | Zero | Default installation |
| Multi-node | rqlite (raft-replicated SQLite) | ~50MB RAM | HA clusters |
Conversion from SQLite to rqlite happens automatically on first shc node join.
Inter-Node Communication
Section titled “Inter-Node Communication”The node-plane services bind and dial mesh0 addresses. Their mTLS certificates are separate from the WireGuard keys that authenticate mesh0.
| Port | Protocol | Purpose |
|---|---|---|
| 4001 | HTTP | rqlite API |
| 4002 | Raft | rqlite consensus |
| 4003 | HTTP | shc internal API |
Internal API Endpoints (Port 4003)
Section titled “Internal API Endpoints (Port 4003)”The listener is
subsystems/impl/internal_api.go.
A node-to-node call is a named internal RPC function (node.health,
vault.unlock, certs.rotate, storage.nfs_enable_client, …) that
its owning module registers with modules/internal_rpc, all served by
the one RPC route (SHC-1207). The generated
internal RPC reference lists every
function with its arguments and result. Live streams keep their own
WebSocket routes. Enrolment is POST /api/join/node.enroll on the public
API through Traefik on HTTPS 443, authenticated by a join token.
tests/unit/internalroutes pins the internal route set:
a new route under /internal/ fails it.
| Endpoint | Method | Purpose |
|---|---|---|
/internal/rpc/{function_name} | POST | A node-to-node call, by function name (see communication patterns) |
/internal/transfer/{id} | GET | Raw bytes of a staged RPC transfer handle; only internal_rpc pulls it |
/internal/containers/{id}/logs/ws, /internal/stacks/{id}/logs/ws, /internal/ws/logs | WebSocket | Log follow |
/internal/containers/{id}/exec, /internal/ws/exec | WebSocket | Interactive exec |
/internal/ws/forward | WebSocket | Port-forward. Also the one /internal/ route the unix socket serves, for the local CLI |
Node Types
Section titled “Node Types”Leader-aware subsystems perform their leader work only on the rqlite leader.
Node-local work, including lease renewal and the node agent, runs on every
node. A non-voter receives replicated state without voting. Sources: CE
subsystems/register.go and modules/cluster/voters.go.
Cluster Operations
Section titled “Cluster Operations”Creating a Cluster
Section titled “Creating a Cluster”# On first node - generate join tokenshc node token
# On second node - join the clustershc node join <token>
# Check cluster statusshc node statusNode Management
Section titled “Node Management”# List cluster nodesshc node ls
# Drain workloads before maintenanceshc node drain node-2
# Gracefully leave clustershc node leave
# Remove unreachable nodeshc node forget node-2
# Force leader election (emergency recovery)shc cluster repair --forceContainer Runtimes
Section titled “Container Runtimes”A node runs docker, podman, incus or apple. The daemon owns
deployment operations. The node-agent subsystem runs assigned work
through the runtime executor. A replica runs on the node it is bound to, and
that node’s runner starts the project the leader finished for it (SHC-1426);
see Runtime support.
shcd watchdog runs outside the daemon and stops containers labelled
shc.single_writer after the node lease expires. It covers all four
engines. Sources: CE modules/watchdog and subsystems/impl/node_agent.go.
Storage Providers
Section titled “Storage Providers”| Provider | Shared | Quotas | Snapshots | Use Case |
|---|---|---|---|---|
| Local FS | No | ZFS/XFS/Btrfs | ZFS/Btrfs | Single-node |
| NFS | Yes | No | No | Simple shared |
| CephFS | Yes | xattr | Native | Production HA |
Health Monitoring
Section titled “Health Monitoring”Node leases record liveness. The suffrage reconciler adjusts voter roles
to the control-plane target, one member at a time. A down member keeps its
vote until explicit removal; quorum loss requires operator recovery.
Sources: CE subsystems/impl/node_lease.go, modules/cluster/voters.go
and suffrage.go.
Error Handling
Section titled “Error Handling”| Scenario | Response |
|---|---|
| Quorum Lost | Writes rejected, reads continue, 503 error |
| Leader Failure | Automatic re-election (~seconds) |
| Split-brain | Minority partition becomes read-only |
See Also
Section titled “See Also”- Node Commands
- Storage: provider details and managed-provider lifecycle