Skip to content
SHC Docs

Registry connections

Status: open design (SHC-800). The docker-registry connection type is in the tree, and shcd still composes its credential before the vault starts.

Image-pull credentials are a shc_connection, not an env-var pair on the terraform provider.

A registry provider capability slot and a generic docker-registry connection type.

Terminal window
shc connection connect --type docker-registry --name hub \
--cred username=shc-puller --cred password=- < docker-hub.token
shc connection connect --type docker-registry --name gitlab \
--cred registry_server=registry.gitlab.com \
--cred username=shc-deploy --cred password=- < gitlab.token

Omit registry_server for Docker Hub. That is the same default the terraform provider already applies, so moving an existing credential across changes nothing about where it authenticates.

The daemon enumerates every registry-capable connection visible to a node, composes them into one docker config under its state home, and points the container engine at it with DOCKER_CONFIG. Several registries merge, because docker’s auths is a map and always was.

shc connection ls grows a registry column. shc doctor grows docker.registry-auth.

The terraform registry_auth block has three faults, and each is structural rather than a bug to patch.

It expresses one registry. shc init’s own genesis pull spans docker.io (percona, postgres, busybox, dexidp/dex), public.ecr.aws (openobserve) and registry.gitlab.com (keycloak, traefik, forward-auth). A single block cannot cover the bootstrap it exists to protect.

It truncates. writeDockerConfigOverSSH stages the file with sh -c 'umask 077 && mkdir -p ~/.docker && cat > ~/.docker/config.json'. docker login merges into that map; cat > replaces it. Any estate whose CI runs docker login registry.gitlab.com on nodes loses that entry the next time terraform touches the node.

It never reached shc node join. Nothing in ce has ever written ~/.docker/config.json. A node joined by hand got no registry auth at all, and pulled anonymously until the per-IP budget ran out.

A connection fixes all three by construction: vault-backed, probed at connect, visible, grantable, and re-composed by the daemon on every node whenever the set changes.

What did NOT ship, and what it needs first

Section titled “What did NOT ship, and what it needs first”

No cloud connection lights the registry slot. Not aws → ECR, not azure → ACR, not gcp → Artifact Registry. This is deliberate and the guard is a test (TestRegistrySlotIsLitByExactlyOneType).

All three issue SHORT-LIVED credentials:

backendcredentiallifetime
ECRGetAuthorizationToken~12h
ACRAAD access token~1h (refresh token ~3h)
GCP Artifact RegistryOAuth access token~1h

ce has no credential-refresh machinery anywhere. Grepping modules/connections for refresh/renew/expires/ttl/rotate turns up exactly one thing: PurgeExpiredRetired, the 24h tombstone sweep. CapabilitySpec carries no expiry field and the mint path’s idempotency gate (readMinted) never re-runs to notice one.

So an aws entry declaring Providers{"registry": "ecr"} would authenticate cleanly at shc connection connect, write a valid docker config, and rot within twelve hours with nothing on the box able to tell. That is a trap, not a feature.

Three pieces, none of which exist:

  1. An expiry on the resolved capability: CapabilitySpec gains a TTL, or the resolved credential carries a not_after the consumer can read.
  2. A leader-only refresh cron. The infrastructure is already there: registry.Crons, used by modules/app (LeaseReapCronJobs), retention, health and storage, with the leader-only discipline documented in modules/app/crons_lease.go.
  3. A re-probe on read, so a consumer resolving a credential past its expiry triggers a mint rather than serving a dead token.

Once those land, the cloud types are a small addition: an ecr / acr / gar impl behind the registry slot that exchanges the account credential for a fresh registry token, and RegistryConnCredSource re-materializing on the cron tick. Nothing about the slot, the composition or the doctor check changes.

Until then, an operator wanting ECR uses a docker-registry connection holding a long-lived credential, or keeps the pull anonymous and accepts the budget.

Two live estates do. Nothing breaks and nothing is required.

SHC_REGISTRY_SERVER / _USERNAME / _PASSWORD still feed the terraform provider’s registry_auth block, which still stages ~/.docker/config.json at cluster init and node join. That path is untouched; genesis pulls happen before an operator has run shc connection connect, so something has to seed the first one.

To move the credential onto the connection substrate:

Terminal window
printf '%s' "$SHC_REGISTRY_PASSWORD" | shc connection connect --type docker-registry --name hub \
--cred username="$SHC_REGISTRY_USERNAME" \
--cred password=-
# add --cred registry_server="$SHC_REGISTRY_SERVER" only if you set a non-Docker-Hub server

Then restart the daemon on each node (the composition runs at wire time), and confirm with shc doctor --category docker or shc connection ls.

Keep the terraform block. It is the genesis seed, and the daemon’s composition SEEDS FROM IT rather than replacing it: the bootstrap-staged ~/.docker/config.json is read and merged, so a credential you staged once at bootstrap survives the handoff even though DOCKER_CONFIG shadows the ~/.docker lookup.

Everything behaves exactly as it does today. With no docker-registry connection the daemon writes no file and publishes no environment, so every engine child inherits the daemon’s environment byte-for-byte and resolves ~/.docker/config.json as before. The doctor check reports info: that the node pulls anonymously, a statement, not a fault.

  • The composition runs at daemon wire time. A connection added while the daemon is running reaches the node on the next restart. docker.registry-auth reports the gap explicitly (warning: N registry connection(s) are not represented in the credential this node is using) rather than leaving it silent. A connect-time re-materialize hook is the obvious follow-up.
  • The cat > truncation in the terraform provider is still there. It is out of this lane’s scope (the lane’s mandate was not to weaken the bootstrap path), but it is a live defect: the writer should read-modify-write the auths map. Fixing it would also let terraform stage more than one registry at genesis, which is what the four-registry bootstrap actually needs.
  • SHC_REGISTRY_* still names three unrelated things. The terraform pair (image pulls), the cicd template’s SHC_REGISTRY_HOST/_USER/_TOKEN (a docker login on nodes), and SHC_REGISTRY_API/_AUTH_HEADER (the SHC apps-package HTTP registry, not images at all). Renaming is its own change: two of the three live in CI templates consumed by downstream customer pipelines.
  • --with-registry-auth is still not passed by the swarm driver. pullthrottle’s own header comment names the flag; grepping ce for it finds only that comment. A swarm-mode estate’s credentials do not reach its workers.