The sso socket hands out the cross-realm admin credential, and the boundary that stops it is a property of the edge rather than of the contract
Status: design settled; nothing about the socket contract changed in this lane. Landed here: the verified chain, the per-app classification, the target design with its minimal role set, the staged migration, plus two guards (one in this repo, one in ce) that keep the surface from growing while the fix is built. A v0.0.19 roll is in progress on both estates and an SSO regression there is estate-wide, so stages 2-5 below are deliberately not in this lane.
Baselines. shc origin/main 07ace7b62, ce origin/main 1053a03.
Every figure below was measured against those two trees. Estate facts were
read from the promanager leader (shc ls --all-tenants) and nothing else;
no estate, cloud, DNS or pipeline was mutated.
Verdict up front. The reported finding is half right, and the half that
is wrong matters. The publication is real, the plumbing to a consumer job’s
shell is real, and netbird does mint a master-realm token with it. But ce
already blanks the credential for consumer-side renders on a
boundary-crossing edge, and both estates are deployed on exactly that
geometry, so an ordinary tenant app declaring an sso plug receives
adminPassword="" today. What is true, and is the finding’s durable
contribution, is that this protection is a conditional on the edge, not a
property of the contract: three reachable configurations resolve as
same-tenant and hand over the real credential, the socket schema still
offers the field to all 32 consumers, and the username half of the
credential is not covered at all. The identity plane has no per-app
blast-radius boundary; it has one tenant-boundary check that happens to be
load-bearing.
Note added at landing (the sso->oidc rename). Since this was authored, the
keycloak socket and every SSO consumer plug were renamed on both mains:
builtin/keycloak/oidc.socket.yaml (was sso.socket.yaml) in ce, and each
apps/<app>/oidc.plug.yaml (was sso.plug.yaml) in shc. All file-path
references in this document have been updated to the new names; the resolver
behaviour they describe is unchanged. The CLI spellings in the S1.5 table
(-i sso=KEYCLOAK, …) are quoted as originally measured against the
pre-rename plug name; the post-rename canonical spelling is -i oidc=...,
which resolves identically (the left side is the consumer’s plug stem).
1. The chain, verified end to end
Section titled “1. The chain, verified end to end”1.1 What the socket publishes
Section titled “1.1 What the socket publishes”ce builtin/keycloak/oidc.socket.yaml declares three admin fields in its
consumer-facing schema.socket block and emits all three from data::
adminUrl@url!: Keycloak admin URL adminUser@string!: Admin username adminPassword@secret!: Admin password kcadmPath@string!: Path to kcadm.sh CLI adminUrl: "http://localhost:8080{{ vars.basePath | default('') }}" adminUser: "{{ vars.adminUser }}" adminPassword: "{{ vars.adminPassword }}" kcadmPath: /opt/keycloak/bin/kcadm.shThe socket’s own jobs authenticate with them against
/realms/master/protocol/openid-connect/token with client_id=admin-cli.
check-keycloak-admin-api, create-oidc-client, delete-oidc-client all use
the identical preamble. Those are the master-realm admin credentials of
the single shared system Keycloak.
1.2 That credential is a cross-realm admin: ce says so
Section titled “1.2 That credential is a cross-realm admin: ce says so”ce modules/auth/keycloak_admin.go, on ListRealms:
“The preamble’s token is a master-realm admin, a cross-realm admin, so that is the WHOLE realm inventory, master included”
and again at the kcCurlAdminPreamble constant:
“a master-realm admin token is a cross-realm admin, so it lists any tenant realm’s users.”
So a holder can read and write every tenant realm: rewrite user profiles, mint clients, reset passwords, read client secrets.
1.3 How socket data reaches a consumer job’s environment
Section titled “1.3 How socket data reaches a consumer job’s environment”ce modules/app/plug_jobs.go: runConsumerPlugPostDeploymentJobs, which the
install pipeline calls at install_runner.go:1799 after the app’s containers
are up:
tmplCtx := jinja.Context{ "integration": jinja.Context{ "plug": jinja.Context{"data": ic.PlugData}, "socket": jinja.Context{"data": socket}, }, ...runOnePlugJob renders that context into the job’s environment: via
renderHookEnv and dispatches it. context: service execs into the
consumer’s compose project, context: container spins a one-shot image of
the consumer’s choosing. Either way the credential lands in a shell the
consuming app’s package author wrote. The teardown mirror
(unintegration_runner.go) has the same shape.
1.4 What netbird actually does with it
Section titled “1.4 What netbird actually does with it”shc apps/netbird/oidc.plug.yaml, job netbird-configure-keycloak-client
(context: container, image: {{ vars.images.curl }}):
KC_ADMIN_USER: "{{ integration.socket.data.adminUser | default('') }}" KC_ADMIN_PASS: "{{ integration.socket.data.adminPassword | default('') }}" TOKEN=$(curl -sf -X POST "$${KC_URL}/realms/master/protocol/openid-connect/token" \ -d grant_type=password -d client_id=admin-cli \ -d "username=$${KC_ADMIN_USER}" \ --data-urlencode "password=$${KC_ADMIN_PASS}" ...It then, against $REALM: reads the client by clientId, creates an
oidc-audience-mapper protocol mapper, PUTs serviceAccountsEnabled:true
on the client, reads the service-account user, reads the realm-management
client and three of its roles, and POSTs a client-role mapping granting that
service account view-users / query-users / manage-users. Every one of
those calls is scoped to $REALM, but the token is not. Nothing in the
job constrains it to $REALM; the same token would answer for any other
tenant’s realm if the command were changed by one line.
shc apps/mailcow/oidc.plug.yaml does the same in job
mailcow-template-claim-mapper to mint one oidc-usermodel-attribute-mapper.
1.5 The mitigation the finding missed, and its exact reach
Section titled “1.5 The mitigation the finding missed, and its exact reach”ce modules/app/integration_render.go blanks every @secret socket field in
a consumer-facing copy (ConsumerSocketData) when the edge crosses a
tenant boundary:
if sp.ProviderTenant != "" && sp.ProviderTenant != tenant { ... if secret := SecretSocketFields(socketDoc); len(secret) > 0 && ic.SocketData != nil { redacted := copyAnyMap(ic.SocketData) for name := range secret { if _, ok := redacted[name]; ok { redacted[name] = "" } } ic.ConsumerSocketData = redacted }}Both consumer-side readers prefer that copy: the compose render
(integration_render.go:865) and the post_deployment runner
(plug_jobs.go:129). Provider-side jobs read the full SocketData, which
they must: create-oidc-client runs inside the master Keycloak and cannot
work without the credential.
SecretSocketFields correctly handles the @secret! spelling
(strings.TrimSuffix(marker, "!") == "secret"), so adminPassword is
classified. adminUser is typed @string! and is not.
I measured all five reachable geometries by driving the real
ResolveIntegrations and the real runConsumerPlugPostDeploymentJobs over a
keycloak-shaped provider and a netbird-shaped consumer, and reading the
rendered job environment, not the intermediate map:
| edge spelling | resolved provider tenant | consumer tenant | redaction fires | job’s KC_ADMIN_PASS |
|---|---|---|---|---|
-i sso=KEYCLOAK | master | corp | yes | "" |
-i sso=keycloak | master | corp | yes | "" |
-i sso=kc-main (own-tenant keycloak, custom stack name) | corp | corp | no | the real credential |
-i sso=keycloak@default | corp | corp | no | the real credential |
-i sso=KEYCLOAK | master | master | no (tenants equal) | the real credential |
ApplyMagicToken stamps ProviderTenant=master for the token and for the
lowercase stack spelling (strings.EqualFold), which is why rows 1 and 2
agree. Rows 3-5 are the surviving surface.
1.6 What the estates actually look like
Section titled “1.6 What the estates actually look like”promanager leader, shc ls --all-tenants:
keycloak master default keycloak runningopenobserve master default openobserve runningpostgres master default postgres runningtraefik master default traefik runningaffine, huly, keygen, license-broker, minio, … corp prod runningKeycloak is at (master, keycloak, default); every consumer is in corp.
Every sso edge on this estate is therefore row 1 or 2: redacted. The
two ce builtins that ship an oidc.plug.yaml and do live in master
(builtin/traefik, builtin/openobserve) reference the admin fields zero
times, so row 5 has no occupant either. There is no live exposure at HEAD on
either estate.
1.7 The consequence nobody has looked at: the two admin jobs are inert in production
Section titled “1.7 The consequence nobody has looked at: the two admin jobs are inert in production”Both admin-reading jobs guard on the credential and soft-skip when it is
empty, because the sso socket stem is shared with apps/external-oidc,
which publishes no admin API:
if [ -z "$${KC_ADMIN_USER}" ] || [ -z "$${KC_ADMIN_PASS}" ]; then echo "[netbird sso] provider exposes no admin API (external issuer)."adminUser survives redaction but adminPassword does not, so on every
row-1/row-2 deployment (i.e. every real one) that branch is taken. Which
means:
- netbird’s audience mapper and its IdP-manager service account are never created. The plug’s own comment says that without either one “every authenticated call to the management API returns 401, i.e. nobody can log into the dashboard”. The job was proven live before it soft-skipped.
- mailcow’s
mailcow_templateclaim mapper is never created, so Attribute Mapping silently reaches nothing.
Neither is a security defect; both are functional ones, and both are consequences of the security boundary working. This is the strongest argument for the target design: the two apps that legitimately need realm admin are broken precisely because the only credential on offer is one they must not be given. Scoped credentials fix the security posture and the functional bug with the same change. Neither is in this lane. Both need a live SSO proof.
2. Per-app classification of all 32 sso-plugged apps
Section titled “2. Per-app classification of all 32 sso-plugged apps”Classes: (a) needs only its own client’s credentials (clientId /
clientSecret / issuer coordinates); (b) needs realm-scoped admin on its
own tenant realm; (c) genuinely needs cross-realm admin.
(c) is empty. No consumer performs a single operation outside its own
realm. The one genuinely cross-realm operation in the whole system is
POST /admin/realms (realm creation) and it is already provider-side, in the
socket’s own create-oidc-client job.
keys below are the socket fields each plug reads; jobs names the
post_deployment / unintegration jobs and what they target.
| app | class | socket keys read | jobs, and what they touch |
|---|---|---|---|
| affine | a | none | none |
| blockscout | a | none | none |
| defectdojo-classic | a | none | register-keycloak-provider / disable-keycloak-provider → its own defectdojo-uwsgi |
| dependency-track | a | none | none |
| frappe | a | hostname, port | frappe-set-social-login-key → its own frappe-web |
| gitea | a | basePath, hostname, port, publicUrl | configure-oidc / remove-oidc → its own gitea |
| gitlab | a | none | gitlab-reconfigure-oidc → its own gitlab |
| highlight | a | none | none |
| huly | a | none | none |
| immich | a | hostname, port | curl container → its own /api, plus Keycloak’s unauthenticated /realms/<realm> |
| ivatar | a | none | none |
| journiv | a | none | none |
| kutt | a | none | none |
| librechat | a | none | none |
| mailcow | b | adminPassword, adminUser, basePath, hostname, port, publicUrl | configure-keycloak-oidc → its own nginx-mailcow (class a); mailcow-template-claim-mapper → Keycloak admin API |
| matomo | a | none | none |
| matrix | a | none | none |
| mautic | a | hostname, port | configure-saml / remove-saml → its own mautic |
| minio | a | none | none |
| minio-classic | a | none | none |
| monica | a | none | none |
| netbird | b | adminPassword, adminUser, basePath, hostname, port | netbird-configure-keycloak-client → Keycloak admin API; netbird-rewrite-management-json → its own netbird-management (class a) |
| nextcloud | a | basePath, hostname, port, publicUrl | configure-oidc / remove-oidc → its own nextcloud |
| paperless | a | none | none |
| peertube | a | none | none |
| penpot | a | none | none |
| postiz | a | none | none |
| supabase | a | none | none |
| vaultwarden | a | none | none |
| wazuh | a | none | none |
| windmill | a | basePath, hostname, port, publicUrl | configure-sso / remove-sso → its own windmill-server |
| wordpress | a | hostname, port | cli container → its own wp-cli |
Tally: 30 × (a), 2 × (b), 0 × (c). The prior was right. And the 30 are (a)
by demonstration, not by inspection: they run today on both estates with
adminPassword="" and their SSO works.
The fields the 30 actually consume are pure coordinates: hostname, port,
basePath, publicUrl, plus their own clientId/clientSecret from plug
data. None of that is privileged.
3. The target design: per-consumer scoped credentials
Section titled “3. The target design: per-consumer scoped credentials”3.1 The minimal Keycloak role set
Section titled “3.1 The minimal Keycloak role set”Roles named below are client roles on the realm-management client of the
consumer’s own tenant realm. A realm-management role has no meaning
outside the realm that hosts it, which is the whole point: a token minted from
it is structurally incapable of touching another tenant.
Operations the two class-(b) consumers perform, and the least role that admits each:
| operation | endpoint | least role |
|---|---|---|
| find own client by clientId | GET /admin/realms/{r}/clients?clientId= | view-clients |
| list protocol mappers | GET .../clients/{id}/protocol-mappers/models | view-clients |
| create protocol mapper (audience; user-attribute) | POST .../clients/{id}/protocol-mappers/models | manage-clients |
| enable the client’s service account | PUT .../clients/{id} | manage-clients |
| read the service-account user | GET .../clients/{id}/service-account-user | view-clients + view-users |
find realm-management and read its roles | GET .../clients?clientId=realm-management, GET .../clients/{rm}/roles/{role} | view-clients |
| grant a client role to that user | POST .../users/{uid}/role-mappings/clients/{rm} | manage-users |
Keycloak’s manage-* permissions imply the corresponding view-*, so the
minimal sets are:
- netbird:
manage-clients,manage-users. (view-usersmay be granted explicitly for legibility; it is implied.) - mailcow:
manage-clients, nothing else. It only mints one mapper. - the other 30: nothing. No admin credential of any kind.
Explicitly never granted, and the bind must refuse them by name:
realm-admin (the composite that reintroduces everything), manage-realm,
create-realm (a master-realm role, and the sole genuinely cross-realm
operation; stays provider-side forever), impersonation,
manage-identity-providers, manage-authorization, view-events.
Honest limit of this design. A bind credential with manage-clients on a
tenant realm can still rewrite other clients in that same realm, and a
tenant realm holds every app in that tenant. So the fix reduces blast radius
from cross-realm (the whole estate) to intra-realm (one tenant). It
does not reach per-app. Getting to per-app needs Keycloak fine-grained admin
permissions (authorization-services scopes on the realm-management client,
whose API changed shape in KC 26), which is a much larger lift and belongs
after stage 5, not inside it. Say the reduction plainly rather than claiming
per-app isolation this design does not deliver.
3.2 How the socket hands a consumer its credential
Section titled “3.2 How the socket hands a consumer its credential”Four additive pieces; the mechanism for all of them already exists.
-
The consumer declares its need, in the plug. A new plug-schema field
adminScopes@string: a comma-separated subset of a fixed allowlist{view-clients, view-users, query-users, manage-clients, manage-users}. Absent or empty (the 30 apps) means no admin credential is minted at all. A value outside the allowlist fails the bind loudly. netbird declaresmanage-clients,manage-users; mailcow declaresmanage-clients. -
A new provider-side socket job mints it.
mint-bind-service-account,context: service,service: keycloak, running aftercreate-oidc-clientwith the fullSocketDataexactly as today. It:- creates/reconciles a confidential client
shc-bind-<consumerTenant>-<consumerStack>in the consumer’s realm, withserviceAccountsEnabled: true,standardFlowEnabled: false,directAccessGrantsEnabled: false,publicClient: false; - grants its service-account user exactly the declared roles in that realm;
- revokes any
realm-managementrole on that service account not in the declared set, so narrowing a declaration actually narrows: reconcile, never accumulate. (create-oidc-client’s existing else-branch already establishes reconcile-on-every-run as this socket’s idiom.)
- creates/reconciles a confidential client
-
The secret reaches the consumer by the existing harvest.
create-oidc-clientalready returns a minted secret withecho "::shc-connection-value::clientSecret=$${CLIENT_SECRET}::", parsed byparseConnectionValues. The bind job emitsbindClientIdandbindClientSecretthe same way. The socket schema gainsbindClientId@stringandbindClientSecret@secret, the latter marked@secretso the existing redaction covers it for free on cross-tenant edges, which is correct: a consumer gets its bind secret from the harvest, not from the socket payload. -
Consumers mint against their own realm. The two jobs change one URL and one grant:
Terminal window TOKEN=$(curl -sf -X POST "$${KC_URL}/realms/$${REALM}/protocol/openid-connect/token" \-d grant_type=client_credentials \-d "client_id=$${BIND_CLIENT_ID}" \--data-urlencode "client_secret=$${BIND_CLIENT_SECRET}" | …)/realms/$REALM, not/realms/master. The resulting token cannot address another realm at all. -
Teardown. The socket’s
unintegrationphase deletes the bind client alongside the OIDC client, mirroringdelete-oidc-client. Without this an uninstall leaves a live service account behind.
4. Staged migration
Section titled “4. Staged migration”Each stage is independently shippable and independently revertible. The stage boundaries are chosen so that no stage can break the 32 apps.
Stage 1: this lane. No behaviour change. This document, plus two guards:
shc tests/unit/catalog/sso_admin_credential_scope_test.go: a ratchet over the committed catalogue. Onlynetbirdandmailcowmay referenceintegration.socket.data.admin*; each must keep its soft-skip branch so the credential stays optional; no plug may readkcadmPath(shellingkcadm.shis the same privilege by another door).ce modules/app/sso_socket_secret_boundary_test.go: pins the@secretmarker on the realbuiltin/keycloak/oidc.socket.yaml(the sole input to the redaction, and nothing else in Go parses it), and pins the install-side redaction end to end. ce already tested the teardown side (TestCrossTenantUnintegrationRedactsProviderSecretsFromTheConsumerJob); the install path, the wider surface, had no equivalent.
Stage 2: mint the bind service account. Purely additive. Add
mint-bind-service-account and the two new socket fields. Publish them
alongside the existing admin fields; convert no consumer. Every existing
plug keeps working byte-for-byte. Gate: one estate installs an sso-plugged
app and the bind client appears in the tenant realm with exactly the declared
roles and no others.
Stage 3: convert the two consumers. netbird and mailcow switch to
bindClientId/bindClientSecret and /realms/$REALM. Keep the soft-skip,
now keyed on the bind secret, for apps/external-oidc. This is the stage
that needs a live proof, and it is also the stage that fixes §1.7: netbird
dashboard login and mailcow attribute mapping should start working for the
first time on a cross-tenant deployment. Gate: a real login on one estate,
both apps, before the other estate is touched.
Stage 4: make the boundary unconditional. Blank adminUser,
adminPassword and kcadmPath in ConsumerSocketData on every edge, not
only cross-tenant ones, closing rows 3, 4 and 5 of §1.5 and the adminUser
gap. Safe only after stage 3, and the stage-1 soft-skip guard is what makes
it non-breaking: any consumer still reading the fields degrades rather than
failing a deployment. Fold in the separate defect in §5 here, in the same
change, since both are about the same boundary.
Stage 5: withdraw the fields. Remove adminUser / adminPassword /
kcadmPath from schema.socket’s consumer contract entirely. They remain in
SocketData for the provider-side jobs, which is where they always belonged.
At that point the ratchet in stage 1 has nothing left to guard and its
allowlist should be emptied, not deleted; an empty allowlist with the
reasoning intact is the durable artefact.
After stage 5, separately: evaluate Keycloak fine-grained admin permissions to get from intra-realm to per-client, per §3.1’s honest limit.
5. A second, separate defect found while verifying: keycloak@<env> escapes the magic rewrite
Section titled “5. A second, separate defect found while verifying: keycloak@<env> escapes the magic rewrite”Not fixed in this lane. Recommended for stage 4.
ApplyMagicToken (ce modules/app/integration_magic.go) is deliberately
case-insensitive on the canonical stack spelling, because “D2/terraform
and shc_app lower the magic name before it reaches the CLI”:
for _, mp := range magicProviders { if strings.EqualFold(spec.ProviderStackName, mp.Stack) { … }}The guard that refuses an @env-qualified magic token
(ValidateIntegrationProviders, ce modules/app/register.go:459) is
case-sensitive, matching only the all-caps Token:
if base, env, found := strings.Cut(sp.ProviderStackName, "@"); found { if _, isMagic := MagicProviderByToken(base); isMagic { return X400903INVAPP("magic token "+base+" cannot be qualified with @"+env, nil) }And parseProviderRef returns from its hasEnv branch before
ApplyMagicToken runs at all. So:
-i sso=KEYCLOAK@staging→ correctly refused,X400903INVAPP.-i sso=keycloak@staging→ silently treated as an ordinary same-tenant, cross-environment provider. It loses both cross-tenant rules: theOverrideFieldcompartmentalisation that forces the consumer into its own tenant realm, and the@secretredaction (row 4 of §1.5, measured).
This is aggravated by the documentation: shc docs/architecture/access-control.md, in “Cross-environment access”, names
keycloak among the “infrastructure-shaped apps (mail, s3, keycloak,
etc.)” an operator should reach with @env, inviting precisely the spelling
that escapes the rewrite. A magic provider is env-less by construction
(MagicProviderEnvironment), so keycloak does not belong in that list.
Why it is not landed here. Reachability is limited: the escaped edge
resolves GetStackFromDisk(consumerTenant, "keycloak", env), and neither
estate has a keycloak stack outside master, so today it fails
X404906STKNOF before any credential is rendered. It is a latent asymmetry,
not a live exposure. The fix is one line: make the @env refusal
case-insensitive over the magic stack spellings, mirroring ApplyMagicToken.
But ValidateIntegrationProviders runs on every install, and a
v0.0.19 roll is in flight on both estates. A one-line change to the install
door is not worth an estate-wide risk for a latent asymmetry. It lands with
stage 4, where the same change also closes rows 3 and 5, and where it can be
proven on one estate first.
Fix the access-control.md sentence at any time; it is prose and carries no
risk.
6. What is honestly true about severity
Section titled “6. What is honestly true about severity”- Pre-existing. Nothing in this lane or in v0.0.19 widened it. The publication, the plumbing, the redaction and both admin-reading plugs all predate this work.
- Not third-party code execution today. The app catalogue is SHC-authored, reviewed, and committed in this repo. This is not “any app the operator installs can escalate”; it is “any app we package could, and nothing structural stops us”.
- No live exposure at HEAD on either estate, measured in §1.6.
- But the assumption it inverts is real. The owner’s model is that an app gets its own credential. What the contract actually offers all 32 consumers is the credential, and what prevents them taking it is a tenant-boundary conditional that three configurations bypass and that covers only the password, not the username. A boundary that holds because of how the estate happens to be laid out is not a boundary; it is a coincidence with good hygiene.
- And the boundary working is currently costing two features (§1.7), which is the part that makes this worth fixing properly rather than documenting and moving on.