Skip to content
SHC Docs

Brokers

Layer 6. Services. Services

A plug/socket edge is deploy-clocked. Its jobs run when an operator connects, updates, backs up or disconnects, and each run provisions standing state: one database, one OIDC client, one grant, per edge.

A broker is the other clock. It is a durable convergence loop over both sides’ observable state, it converges only when both ends exist, and it keeps a ledger so that deletion is expressible. Anything keyed on the user set or on runtime object churn is a broker, not a socket job.

flowchart LR
subgraph bind["at bind, deploy-clocked"]
socket["socket jobs<br/>grant ONE capability<br/>per consumer"]
end
subgraph run["at runtime, level-triggered"]
broker["broker loop<br/>spend that capability<br/>per user, forever"]
end
socket -->|"capability + label"| broker
broker -->|"observe"| provider[("provider state<br/>mailboxes, calendars, users")]
broker -->|"observe"| consumer[("consumer state<br/>accounts, subscriptions")]
broker -->|"ledger"| ledger[("what it created,<br/>for whom, when")]

The one-line rule: plugs and sockets grant standing capabilities once at bind; brokers spend them continuously at runtime. Sockets mint per consumer; brokers act per user.

Nothing on this page is engine machinery. There is no broker: key, no broker runner, no broker registry. A broker is code you write and ship in an app, and this page is the doctrine that decides whether you need one, where it runs, what it is allowed to hold, and how it is embodied.


The temptation is always to add one more job. It fails for four separate reasons, and each is measurable in this tree today.

1. There is no runtime trigger keyed on people

Section titled “1. There is no runtime trigger keyed on people”

Lifecycle hooks are the app’s only event surface, and all ten fire on a deploy, a backup or a restore: pre_install … post_restore. There is no on_user_created, no on_user_deleted, no on_object_changed. The integration phases are the same clock seen from the edge: discovery and integration on connect, post_deployment after the consumer is up, unintegration on teardown.

An app does get one cadence (crons: in app.yaml, mapping a job name to a cron expression), and it is not nothing. But it is the wrong shape twice over, and the catalogue’s one user says so at the site:

  • A cron job could not see the other side of the edge (fixed by SHC-1181). A job dispatched through the job service, which is what the cron path does (schedule row → CreateJobRun → execute_job_task), used to get a template scope of {"vars": <deployed vars.yaml>} and nothing else, so {{ integrations.s3.socket.endpoint }} rendered as the empty string. The catalogue’s one cron user still carries the warning at apps/flagship/matrix/s3-media-offload.job.yaml:109 (“DO NOT REACH FOR integrations HERE”). A job run now renders against the deployment’s render context (app.DeployedRenderContext), the one its update renders hooks with, integrations included, and a field that fails to render fails the run.
  • It existed only under a daemon (closed by SHC-1328). The chain is app.yaml crons: → LoadCronsMap → schedule.SyncApp (rows with source="app") → the leader-gated catch-up sweep → execute_job_task. A CLI-thin install had no schedule syncer, so SyncDeploymentCrons returned silently and wrote no row. shc install and shc update now go through shcd and refuse with X503738DMNREQ when it is unreachable, and shcd installs the syncer (daemon/services_extra.go SetScheduleSyncer), so every install reconciles its crons (modules/app/schedule_sync_seam.go). shc schedule ls shows the rows.

So the one clock an app can declare did not see the integration it would have to converge, and existed only where a daemon ran the install.

2. The values a broker needs exist in exactly one render

Section titled “2. The values a broker needs exist in exactly one render”

Per-consumer credentials do not travel through socket data. A socket’s data: block renders once per provider, against vars/stack/environment/tenant and no integration context, which is why every per-consumer field in apps/collaboration/nextcloud/nextcloud.socket.yaml and apps/collaboration/mailcow/mailcow.socket.yaml is published as the empty string. The real value rides the mint-and-return channel: an integration job prints ::shc-connection-value::<key>=<value>::, and ce folds it into the integrations namespace with MergeJobMintedValues (modules/app/install_runner.go:1473).

That fold lands in one place. apps/devops/gitlab/gitlab-seal-sentry-dsn.job.yaml states the constraint with its ce line references:

the post_install/post_update HOOK render reads it (renderCtx["integrations"]) and nothing else does. A plug post_deployment job renders from the STATICALLY rendered socket data instead (plug_jobs.go), where dsn is the empty string sentry.socket.yaml publishes on purpose.

And a hook fired outside an install builds no integrations namespace at all. A cron job never had one. So a would-be “reconcile job” cannot read the credential the socket minted for it unless it runs inside a deploy, which is the clock it was trying to escape. The established fix is to seal: the post-install hook re-emits the value as ::shc-hook:secret:<key>=<value>::, PersistHookMintedVars routes it through the vault, and a ref+vault://…?autogen ref lands in vars.yaml, which the next render reads as an ordinary var. A broker holds its capability the same way: sealed at bind, read from its own configuration forever after.

3. There is no ledger, so deletion is inexpressible

Section titled “3. There is no ledger, so deletion is inexpressible”

The only durable records an edge has are the integration row (unique on (tenant_name, consumer_deployment_id, consumer_plug_name), so one binding per plug) and the bind-time plug context frozen into edge_states in the deployment’s state.json. Both are per edge. Neither is per user.

Without a per-user ledger you cannot tell these three states apart:

ObservedCould mean
provider has no mailbox for aliceshe is new; or the broker created one and someone deleted it; or she was deleted upstream
provider has a mailbox for bob, identity provider does nothe was deprovisioned; or he was always a local account the broker must leave alone
ledger says the broker made carol’s, provider disagreesdrift the broker owns, and may safely re-converge

Only the third row is actionable, and only a ledger produces it. This is why “a broker holds a ledger” is part of the definition rather than an implementation note: a converger without one can create but can never safely delete, and a create-only loop leaves an estate that quietly accumulates access nobody revokes.

A per-user fact is a join: the identity provider’s user set × the resource provider’s object set. The plug/socket grammar has no vocabulary for it. A socket job runs once per connecting plug and sees integration.socket.data and integration.plug.data: its own edge, and nothing else. A plug has at most one binding. Neither side can enumerate the other’s users, and no third party is invited.

The grammar refuses to grow one on purpose. apps/collaboration/mailcow/imap.socket.yaml argues it where the temptation is strongest:

Why a capability and not a mailbox list: the bridge IS the broker, per-user cardinality never enters the plug/socket grammar. The grammar this adds is two scalars. A socket that took a list of mailboxes would need re-integrating on every hire, would duplicate the identity map the consumer already owns, and would put N secrets into a render context sized for one.

None of this is new reasoning. docs/proposals/2026-08-06-cert-delivery-broker.md made the identical case for certificate renewal, choosing an app.yaml declaration over a tls socket:

Integration jobs run on connect and on backup; disconnect runs none. Certificates rotate every ~60–90 days and the delivery must CONVERGE on rotation with no operator step. That is a sweep-reconcile problem … not a connect-time job. A tls socket would bolt a renewal loop onto machinery that has none.

That one landed as platform code: ce modules/certbroker, reconciling at the ingress sweep’s 60-second cadence, with a per-delivery posture.json record and atomic serial rotation. Renewal and provisioning-per-user are the same shape (a fact that changes on its own clock, on the far side of an edge), and they get the same answer.

Read the status header before citing that proposal. Four of its design decisions were reversed or dropped on the way in (staging material is delivered and labelled, never withheld; certdeliver.allow_staging and the host: declaration key were never implemented; there is no ent ledger), and it has zero adopters: shc cert delivery printing no certificate deliveries is the un-adopted state, not a pass.


Four properties, all required:

  1. Level-triggered, not edge-triggered. It reads current state and converges. It does not consume an event stream as its only input: an event may wake it, but a missed event must not be able to leave the estate wrong forever.
  2. It observes BOTH sides. A loop that only reads the provider is a provider feature. A loop that only reads the consumer is a consumer feature. A broker converges the relation.
  3. It converges only when both ends exist. A user with no account on the consumer is not an error and not a retry; it is a state the loop waits in. This is what makes a broker safe to run before an estate is complete.
  4. It holds a ledger. Per user, per object: what it created, where, when, and under whose authority. See gap 3 above.

Two consequences worth stating because they get argued:

  • A broker is not a queue consumer. Point 1 forbids treating the event bus as the source of truth. Ride the bus for latency; converge from state for correctness. huly/bridges/huly-nextcloud is written to exactly that split: Redpanda for immediacy, Postgres for the state that decides.
  • A broker is not a cron job that happens to poll. Points 2 and 4 are what a cron job cannot have (gap 1 and gap 3). Polling is a scheduling choice inside a broker, not a substitute for one.

Other things in this tree called “broker”

Section titled “Other things in this tree called “broker””

The noun is overloaded and the surfaces are unrelated. Do not cite one for another:

NameWhat it isStatus
edge-broker / identity-broker (this page)app-level convergence loop over an integrationdoctrine settled; the two named components are unbuilt
cert-delivery broker (ce modules/certbroker)platform loop delivering edge certificates to self-terminating deploymentsBUILT + wired, zero adopters
in-deployment exec broker (docs/proposals/2026-08-08-in-stack-exec-broker.md)a request-time RPC letting an app run a declared verb in a sibling servicenot built as proposed: a deployment’s own docker socket answers its sibling services instead (SHC-3333, The Docker socket)
license broker (license-broker/)the vendor-side EE activation servicevendor infrastructure, not an estate component

Only the first two are convergence loops. The exec broker is a request/response front door; the license broker is a signing service.


There are two placements and the question that picks between them is does this fact have a consumer edge?

An edge-broker converges a fact that only makes sense for one specific pair (“this ERPNext user’s calendar, mirrored into that Nextcloud”), so it runs on the consumer side. The bridge IS the broker. It is pair-fluent by construction: it already speaks both dialects because that is its entire job.

The reason is not aesthetic. If the broker lived provider-side, the provider would have to switch on consumer type (a if consumer == "frappe" branch inside Nextcloud’s app), and every new consumer would be a change to the provider. The provider stays generic: it grants one capability and publishes facts. apps/collaboration/nextcloud/nextcloud.socket.yaml states the placement as doctrine at the point where it would be most convenient to violate it, and then declines to violate it:

dav:* is occ, i.e. CLI inside the provider’s container; the edge-broker runs consumer-side by doctrine. … Until one is proven, this socket grants OBSERVATION and says so.

The provider-side surface stays generic in both directions. mailcow.socket.yaml grants an API key and documents a five-clause per-user contract, but performs no per-user action itself; the consumer mints inside its own process and keeps the plaintexts in its own store.

An identity-broker converges facts keyed on the realm, with no consumer edge at all: “every user in group X has a mailbox”, “every subject has a local id on every platform”. There is no pair, so there is no bridge to be, and the loop belongs beside the authority that owns the user set.

huly/bridges/identity-broker is the shape: a standalone service holding a Keycloak sub → per-platform local-id map plus per-user credential seeding at login, answering GET /v1/principals/:sub, POST /v1/credentials/:sub/:platform, DELETE /v1/credentials/:sub. Its own README is careful about what that buys: most /v1/credentials answers are {"kind":"none"}, because for most platforms a single service credential already reaches all the data and per-user identity is wanted for attribution, not access.

Status: design + runnable scaffold. It builds, typechecks, lints and unit-tests clean, and has never been run against a live Keycloak, Postgres or platform API. It is not in the shc catalogue.

Provider-adjacent is not provider-internal. An identity-broker is still its own component with its own ledger; it is placed next to the identity provider because that is where its input set lives, not merged into it.


Plugs and sockets and brokers are not alternatives. They compose, and the seam between them is sharp:

plug / socketbroker
clockdeploy (connect, update, backup, disconnect)runtime, continuous
cardinalityone grant per consumerone action per user
what it movesa standing capabilitythe exercise of that capability
durable recordintegration row + edge_statesits own per-user ledger
teardownunintegration jobs revoke the granta sweep keyed on the label

The socket hands over a capability and a label; the broker spends the capability and every artifact it creates carries the label. That label is the whole revocation story, and both grant jobs in the catalogue refuse to run without a well-formed, unique one: mailcow.socket.yaml and nextcloud.socket.yaml both reject anything outside [A-Za-z0-9][A-Za-z0-9_-]*, with the same reason:

two consumers sharing a label revoke each other on every deploy.

The broker discipline: confine, revoke, verify

Section titled “The broker discipline: confine, revoke, verify”

apps/flagship/matrix/matrixAppservice.socket.yaml maps the contract onto a socket that brokers per-user identity rather than per-user credentials, and its three clause names are the house vocabulary:

  • CONFINE: every artifact the broker creates carries a prefix the provider can enumerate. For mailcow it is the app-password name <label>-<id>; for Nextcloud it is shc-<label>-<utc-stamp>; for the matrix appservice it is usersRegex, which is stronger because Synapse itself refuses a ?user_id= outside the namespace, published so the consumer can confine itself honestly, authored provider-side so binding can never widen it.
  • REVOKE, stated in increasing blast radius, and with the first line read twice: detaching a consumer revokes nothing by itself. The mailcow-revoke-app-passwords sweep is deliberately a standalone job, not an unintegration hook, because a label spans every edge a consumer holds and a sweep riding one edge’s detach would revoke credentials a surviving edge still uses. The operator names the label explicitly.
  • VERIFY: revocation is real only when the credential stops authenticating. matrix-verify-appservice-revoked exists because success is the 401; the read-back decides, never the action’s own exit status. mailcow.socket.yaml’s preflight follows the same rule: it mints a probe app password and re-lists, because “reading only proves the key authenticates”, and what a consumer depends on is add/app-passwd accepting a username for a mailbox the caller does not own.

Given that you need a broker, what does it run inside? Prefer the highest rung that works. Each rung down adds a process, an image, a version, a backup question and a failure mode.

The best broker is one upstream already ships. Check first, because the answer is sometimes yes and the cost of missing it is an entire component.

Nextcloud Mail is the reference. lib/Http/Middleware/ProvisioningMiddleware.php registers a middleware whose beforeController runs on every Mail request and calls provisionSingleUser($configs, $user) against the admin-configured provisioning rows. It is level-triggered per request, per user, and needs no state of its own because the user’s own presence is the trigger.

That single fact is why the NC↔mailcow mail integration needed no broker at all; see the worked example below.

Its limits are the general limits of the rung, and both matter:

  • It fires only when the user shows up. A middleware cannot create an artifact a user needs before their first visit.
  • It is upstream’s semantics, not yours. Here that turned out well: the method ordering means passwordless users are provisioned correctly (the CredentialsUnavailableException|PasswordUnavailableException catch skips only updatePassword, which master-password mode never needs), but that was established by reading the source and then probing live, not assumed.

2. A plugin vApp or image-owned parent extension

Section titled “2. A plugin vApp or image-owned parent extension”

If the parent app has an extension model, the broker becomes an extension of the parent: it inherits the parent’s process, scheduler, database and backup.

huly/bridges/frappe-medusa is the worked case (huly/ here is the house Huly fork, the out-of-tree repo the bridge images are built from). The ERPNext↔Medusa product/inventory bridge is a Frappe app, not a bridge container, and huly/HANDOFF.md §3.4 gives the reasoning:

Frappe gives hooks for outbound (real-time), a scheduler for inbound polling, and doctypes for visible, queryable sync state.

Hooks are the wake-up, the scheduler is the level-trigger, and the doctypes are the ledger: all three broker requirements, satisfied by the parent.

Which mechanism, on which parent, is already ruled. On a bake-capable parent (frappe: no upstream marketplace, and a live gunicorn cannot import a runtime-added Python app) a plug’s mechanism extension is IMAGE-OWNED: baked into the image and activated through vars.activateApps, never declared as a vapp: dependency. On a store-app parent (nextcloud, wordpress: runtime volume content, no preload window) it is vapp: … mode: create. The full rule is in plugs and sockets; apps/flagship/nextcloud/imap.plug.yaml is the store-app reference and apps/flagship/frappe/ carries no vapp: keys at all, pinned by apps/flagship/frappe/tests/tests.bats. The direction rule that scopes all of this is integration doctrine.

When the mechanism has to live inside an app’s own build and upstream will not take it, it rides the house fork’s patch series.

  • huly/patches/0010-uploader-nextcloud-link-registration.patch registers a “Link from Nextcloud” upload handler into Huly’s existing UploadHandlerDefinition registry, rendering only when NEXTCLOUD_PICKER_URL is non-empty. No upload path is patched and no upload primitive is touched; the patch’s own header says it is not upstreamable as-is because the handler is useless without the picker sidecar.

  • Medusa’s ERPNext subscribers are fork-carried by necessity, and apps/business/medusa/Dockerfile explains why in a sentence that also closes off the rung below:

    Medusa v2 has NO outbound webhooks. The only supported way to react to a commerce event is a Subscriber under src/subscribers/*.ts, which medusa build COMPILES INTO the application and runs in-process on the event bus. The ERPNext sync is therefore source that must be inside the image. There is no config flag to flip on someone else’s build, and no sidecar that can subscribe from outside the process.

    Note what that makes the medusa↔ERPNext integration: fork-carried on the Medusa side (compiled-in subscribers), rung 2 on the ERPNext side (a Frappe app). Neither half is a sidecar, and the Dockerfile says a sidecar could not have done it.

The cost of this rung is the fork: every rebase carries the patch, and the patch has to keep applying.

4. A compose sidecar in that app’s own deployment

Section titled “4. A compose sidecar in that app’s own deployment”

The bottom rung: a container with its own image, config and ledger, declared in the plug-side app’s own compose.yaml so it shares that deployment’s lifecycle. Take it when neither app admits an extension you can reach, and accept that you now own an image.

The nextcloud-picker was the stateless reference and is retired, deleted 2026-08-21 along with its two consumer plugs (apps/RETIRED:93). It is worth reading as the precedent for this rung’s cost rather than as an example to copy: the direct frappe↔nextcloud edge superseded it, because apps/flagship/frappe/nextcloud.plug.yaml consumes the nextcloud socket itself and “linking a Nextcloud file into a Frappe doctype no longer needs a third app standing between the two.” The retirement note also records the honest cost: Huly lost the feature, since the superseding edge does not cover it.

huly/bridges/huly-nextcloud is the stateful shape and the honest cost of it: Huly transactor → Redpanda → bridge → Nextcloud CalDAV/CardDAV, with Postgres holding the deterministic UIDs, origin stamps and ETag loop-break that keep the mirror from fighting itself. It ships in this tree as the hulync sidecar on apps/flagship/huly (the standalone huly-nextcloud app was retired 2026-08-25, SHC-244).

✗ Not a rung: a bridge as its own top-level deployment

Section titled “✗ Not a rung: a bridge as its own top-level deployment”

Ruled out (owner, 2026-08-23). A bridge is code, not a deployment. A bridge deployment is still a deployment: it gets a backup policy, a quota charge, an ingress question, a place in shc ls, and a lifecycle that can drift from both apps it joins. That is the tax the four rungs above avoid, and it is the reason the ladder is ordered.

apps/standalone/huly-nextcloud was that exception and is gone: the app was retired 2026-08-25 (SHC-244) after the fold into apps/flagship/huly. The remaining same-shape debt is gitlab-huly and huly-matrix-notifier.

An identity-broker is not a bridge and is not covered by this. It converges realm-keyed facts with no consumer edge, so it has no side to live on and is legitimately its own component; apps/standalone/identity-broker is that shape. So is an external-* socket shim, which joins nothing to nothing: it publishes coordinates for a service outside SHC. The full direction and placement rules are in integration doctrine.


One rule, and it decides the shape of the whole integration:

A master credential (one secret that opens every user’s data) may live in exactly one admin-scoped location that users cannot read. If the storage is per-user-readable, you may not put a master credential in it; you must mint per-user credentials instead, which makes the consumer an app-dialect socket consumer.

The catalogue has both branches side by side.

Master, admin-scoped. apps/collaboration/mailcow/imap.socket.yaml publishes masterUser/masterPassword: one dovecot master login that opens every mailbox over IMAP/POP3/Sieve. Nextcloud Mail’s admin provisioning row is a legal home for it: the row is server-side admin config, the value is stored crypto->encrypt-ed, and jsonSerialize() only ever returns the placeholder ******** (lib/Db/Provisioning.php, MASTER_PASSWORD_PLACEHOLDER), so the read-back can assert the placeholder and nothing more, which is the verifiable limit. No user, and no per-user surface, can read it.

Per-user, minted. apps/collaboration/mailcow/mailcow.socket.yaml is the other branch: a consumer that must act as individual people plugs the app-dialect socket (mailcow, the bare app name: it is mailcow’s REST dialect, not a substitutable wire protocol) and mints one app password per user with add/app-passwd. The socket’s contract is explicit that this is a broker capability, not a socket one: the plug schema takes a label, never a mailbox list.

The picker shows the same fork as a runtime switch. AUTH_MODE: bearer acts as the signed-in user via a Keycloak token, storing nothing; the comment on the alternative says why it is the fallback:

app-password is the fallback and stores a per-user credential, which is the thing bearer mode exists to avoid.

Two corollaries the catalogue enforces:

  • Say what the credential actually is. Both grant sockets refuse to call their key scoped: mailcow’s API key “is NOT a scoped capability, and calling it one would be a lie … A holder can mint app passwords AND delete mailboxes”; Nextcloud’s occ-minted admin token carries the admin account’s full authority because occ exposes no --scope. Narrowing either needs a change upstream, not a comment.
  • A service account that provisions itself is a loop, not a boundary. nextcloud.socket.yaml rejects a dedicated service user for exactly this: it has the same blast radius and it puts a synthetic account into the very user set its consumer enumerates, “which the mailbox reconciler would then mint a mailbox for”.

Four nouns, four questions. Answer them in order; the first yes wins.

If the answer isYou wantBecause
”a fixed number of facts or grants, settled when two deployments are wired together”a socket (+ plug)The cardinality is per edge, so it fits the deploy clock and the integration row.
”a sub-resource of one parent deployment, one per tenant/team/project, that an operator asks for by name”a vAppIt has no containers of its own, it is quota-charged and visible in shc ls, and it is created by shc install <parent>/<vapp>; see vApps.
”a small generic capability one app must expose so somebody else’s loop can converge”a helper vAppIt is a vApp by mechanism (a plugin instance on a store-app parent) but exists only to widen the provider’s API. It must expose ONE generic capability (“create a subscription for user X pointing at URL Y”) and never pair logic, or the provider has started switching on consumer type.
”one artifact per user, or per object, that keeps changing after install”a brokerCardinality tracks the user set, so it needs a ledger and a loop; see the four gaps above.

Three tie-breakers for the cases that actually get argued:

  • A plug never installs a top-level app. A store app that is the mechanism of an edge may be installed by a plug job (nextcloud’s user_oidc, integration_openai); user-facing functionality is a vApp instance declared with vapp:. Full rule in plugs and sockets.
  • Not everything cross-cutting is a socket. Telemetry is the standing counter-example: there is deliberately no otel socket. Read telemetry fabric before proposing one.
  • A broker still needs a socket. It is not an alternative to the edge; it is what spends what the edge granted. If your broker has no plug, ask what authorised it.

Worked example 1: NC↔mailcow mail, the broker you did not need

Section titled “Worked example 1: NC↔mailcow mail, the broker you did not need”

Verdict: native upstream loop. No broker. Live-proven 2026-08-07.

The requirement looks exactly like a broker’s: every Nextcloud user should open the Mail app and find their mailbox already configured, with no per-user setup and no per-user credential handling in Nextcloud, and users appear at first SSO login, long after any deploy-time job has run.

Scope it precisely first, because the scoping is what dissolved it. This is attaching each user to a mailbox that exists. Creating the mailboxes is a different problem with a different answer: worked example 3.

It resolved at rung 1 because two upstream mechanisms met in the middle:

  • The provider side is one credential, not N. mailcow ships a static dovecot master user, so the consumer opens any mailbox with <user's email>*<masterUser> and never mints anything. That is imap.socket.yaml contract (3), and the socket publishes exactly two extra scalars for it.
  • The consumer side already has the loop. Nextcloud Mail’s ProvisioningMiddleware provisions each user on their own first request against an admin-configured row.

So the entire integration is one admin config row, which is a per-edge fact, and a per-edge fact is a plug job. apps/collaboration/nextcloud/imap.plug.yaml writes it, and deletes it again on detach.

What that plug had to get right is instructive, because every item was verified against upstream source rather than assumed:

  • No occ command exists for it. Mail registers 21 commands and none is a provisioning writer; CreateImapAccount creates one user’s account and stores a per-user password this plug must never hold. The surface is the admin HTTP endpoint (settings#createProvisioning, POST /apps/mail/api/settings/provisioning).
  • The CSRF pass is OCS-APIRequest: true, and a cookie-jar dance cannot substitute: with TLS on, compose pins OVERWRITEPROTOCOL=https, the session cookie becomes Secure, and curl will not replay it over the job’s plain-HTTP loopback dial: every write 412s.
  • The request body must be wrapped {"data": {...}}, because Nextcloud’s AppFramework binds createProvisioning(array $data) by request-parameter name. A flat body is HTTP 400.
  • emailTemplate must be %EMAIL%, not %USERID%. With SSO the user id is a Keycloak sub UUID, and a UUID FROM address gets postfix’s “Sender address rejected: not owned by user”.
  • The sslMode vocabularies are inverted between the two sides. The socket says tls|starttls|none where tls is implicit TLS (993); NC Mail says none|ssl|tls where ssl is implicit TLS and tls is STARTTLS. Writing the socket’s value through unmapped asks for STARTTLS on 993 and hangs every login, so the job maps tls|ssl→ssl, starttls→tls, none→none.
  • Every master key is | default('')-guarded, because external-imap serves the same stem and publishes none of them; the job then no-ops loudly rather than writing a half-filled row.

Two live findings are worth carrying forward as broker-adjacent lessons. nextcloud/mail#9175 (master-password provisioning reportedly breaking for passwordless users) does not apply, and the source says why: the catch in beforeController covers CredentialsUnavailableException|PasswordUnavailableException and therefore skips only updatePassword, which master mode never needs; provisionSingleUser has already run. And the shipped app originally hard-set DOVECOT_MASTER_USER/PASS to the empty string (regenerating a random master on every dovecot restart), which is what had made the whole approach look impossible; making the master static is the change that unlocked it.

The generalisable rule: before designing a broker, find out whether the consumer already provisions per user on its own clock, and whether the provider can be opened by one admin-scoped credential instead of N minted ones. If both are yes, you have a plug, not a broker.


Worked example 2: the frappe→nextcloud calendar bridge

Section titled “Worked example 2: the frappe→nextcloud calendar bridge”

Verdict: edge-broker, consumer-side. PROPOSED: the write path is not chosen and the component is not built.

The shape (docs/proposals/2026-08-08-identity-and-edge-brokers.md §2): a frappe-side bench app watches ERPNext calendars via doc-event hooks, serves per-user tokenised ICS feeds, and writes one calendar subscription per user into Nextcloud, when and only when that Nextcloud user exists. It is rung 2 by the bake rule: image-owned, activated through vars.activateApps, never a vapp: dependency.

It is an edge-broker and not an identity-broker because the fact is pair-specific (this ERPNext’s calendars, that Nextcloud’s users), and it runs consumer-side because that is where both dialects are already spoken.

What is settled and verified. The observation half is done: nextcloud.socket.yaml grants an opt-in, label-scoped admin app password on the admin account and publishes ocsUrl, so a broker can answer “does an NC account exist for this ERPNext user, and what is its uid” via GET <ocsUrl>/cloud/users/details. That is the precondition the loop converges on, and it is what the grant is for.

The write surface exists too, and its semantics were read off the pinned image rather than guessed:

  • occ dav:create-subscription <user> <name> <url> [<colour>]: documented upstream alongside dav:delete-subscription and dav:list-subscriptions.
  • It dedupes on the source URL, not the name, and it fails (exit 1, Subscription for url <…> already exists for this user) rather than no-opping. A converger must treat “already exists” as success and must never key its ledger on the display name.
  • There is no update command. Changing a feed URL is dav:delete-subscription <uid> <uri> then a fresh create, and the uri the delete takes is the one dav:list-subscriptions <uid> prints in its first column, not the display name.
  • Refresh cadence is occ config:app:set dav calendarSubscriptionRefreshRate, an ISO-8601 duration, unset by default with a code default of P1D, so an ERPNext event can take a day to appear. The socket deliberately does not write that knob (it is instance-global, so an edge-clocked write lets the last consumer to bind silently retune everyone else’s feeds); it reads the effective value back and publishes it as calendarRefreshRate so a broker can size its own cache against the truth.

What is genuinely open, and why it is a probe rather than an authoring decision. dav:* is occ (CLI inside the provider’s container), and the edge-broker runs consumer-side. The admin app password does not close the gap either, because CalDAV authorises per principal: admin credentials cannot MKCOL into another user’s calendar home. Three candidate write paths remain:

  1. per-user bearer via Keycloak token exchange (the picker’s proven pattern), writing the subscription as the user over CalDAV; oidcClientId is published for exactly this;
  2. a provider-side occ job, for day-one backfill only;
  3. a helper vApp: a small admin-authed endpoint in a Nextcloud app, installable as a plugin vdeployment, exposing one generic capability (“create subscription for user X → URL Y”) and never pair logic.

Until one is proven, the socket grants observation and says so. That refusal to guess is the point: publishing a write key the provider cannot honour would make every consumer’s install go green with the feature broken: the exact failure the sso socket’s SAML face produced and which plugs and sockets records.


Verdict: identity-broker. UNBUILT, and its destructive half is gated on an owner ruling.

The shape (docs/proposals/2026-08-08-identity-and-edge-brokers.md §1): a standalone catalogue app consuming -i mailcow=<mc> plus an identity-provider edge, running a level-triggered loop that ensures every realm user (filtered by group policy) has a mailcow mailbox (add/mailbox, authsource: keycloak; keycloak is one of the three authsources mailcow accepts, per apps/collaboration/mailcow/oidc.plug.yaml), ledgered in its own volume.

The identity half has no socket to bind yet. The proposal writes -i keycloakAdmin=<kc>, and at the pinned ce the keycloak addon serves oidc and saml and nothing else: there is no keycloakAdmin socket in apps/, in this repo’s addons/, or in ce. apps/security/midpoint hit the same wall and documents it (“A plug naming it could never bind, so the credentials are yours to supply”); apps/security/midpoint-classic/tests/tests.bats pins the absence. So the reconciler’s realm-side input is operator-supplied credentials until that socket exists, and any design that assumes the plug is designing against a stem that is not served.

It is an identity-broker, not an edge-broker: the fact is realm-keyed (“every user in this group has a mailbox”) and there is no consumer whose dialect it must speak. That is why it is a standalone deployment rather than a bridge riding one side.

The creation side is mechanical. add/mailbox takes authsource: keycloak; the admin API key’s username field is gated by a hasDomainAccess check that an API-key caller short-circuits, so one key provisions for any mailbox with no user present, which is precisely what “artifacts needed before first use” requires. Mail arrives for people who have never logged in, so the native-loop trick that solved worked example 1 is unavailable here by construction.

The deletion side is the reason it is not built. Two options, and they are not equivalent:

OptionWhat happensReversible
revoke-and-retainmailbox deactivated, app passwords revoked, mail keptyes
deletemailbox and mail gone; app_passwd rows FK-CASCADE awayno

Retain is the safe default. Delete must never be improvised, by an agent or otherwise: it is the one broker action in this estate that destroys user data on a schedule, and it needs an explicit ruling before it is authored.

The gap-3 argument is the whole justification for the ledger this component carries: without one, a mailbox that exists for a user the realm no longer lists is indistinguishable from a local account the reconciler must not touch.

Prerequisites already in place: the mailcow singleton socket and its grant-admin-api mint (with the preflight that proves headless minting works on every bind), the label discipline, and the standalone mailcow-revoke-app-passwords sweep. What is missing is the loop, the ledger, and the ruling.


Worked example 4: the Nextcloud runtime-subscription surface

Section titled “Worked example 4: the Nextcloud runtime-subscription surface”

Verdict: BUILT. The nextcloudWebhooks socket (apps/collaboration/nextcloud/nextcloudWebhooks.socket.yaml), the receiving loop in identity-broker 0.2.0 (src/subscription/nextcloud.ts in the fork, digest-pinned in apps/security/identity-broker/vars.yaml), and the nextcloudWebhooks plug binding them. Statically guarded and container-proven; a registration against a live estate Nextcloud is still owed.

This was the piece the doctrine above named as missing: brokers are level-triggered convergence loops, and until now a broker learned about a new Nextcloud user only when its full poll next walked the realm. The event surface closes the latency gap. It does not, and at this pin cannot, close it the obvious way, and the measurement that proves that is the part worth keeping:

  • Nextcloud 34.0.2 (the pinned image) ships webhook_listeners 1.6.0, default-enabled, with an admin-only OCS registration API and delivery via a queued background job. Verified inside the image, not from docs.
  • The registerable classes at this pin are files-node, calendar-object and system-tag events: exactly 22, enumerated by reflection over the classmap. OCP\User\Events\* and OCP\Group\Events\* do not implement IWebhookCompatibleEvent, so no registration can name “user created”.
  • What every delivery does carry is the acting session user in the envelope (user.uid). So the surface is consumed as a level trigger: an event arrives → that uid was just active → if the map does not know it, converge that one person now. A design that waited for upstream to make user events webhook-compatible would have shipped nothing; a design that pretended the event class was the fact would have been wrong the first time someone replayed a payload.

The doctrine points this example adds:

The event payload is a hint, never a fact. Every converge re-reads Keycloak and re-resolves against Nextcloud before writing a row. That is what makes one shared delivery secret proportionate auth: a forged delivery can cause work, not a wrong identity. Any event consumer that writes what the payload says (instead of what the sources show when poked) has built a spoofing surface, not a broker.

Events are latency; the poll is the floor. Delivery rides the provider’s background-job queue, whose clock is backgroundjobs_mode: in ajax mode (upstream’s default, and this app sets nothing else) jobs run on page loads, so a browser SSO login self-delivers within the session while a WebDAV-only writer may fire nothing for hours. The reconcile loop stays, unchanged, as the level-triggered completeness path, the same split frappe-medusa and the Keycloak admin-events outbox already embody. A socket job reports the delivery clock at bind time and refuses to write it: the knob is instance-global, and an edge-clocked write is the last-binder-wins failure the calendar example already refused once.

Registration is runtime state, so the consumer’s loop owns it. webhook_listeners ships no occ create/delete; the write is OCS with admin credentials the broker already holds. More importantly, oc_webhook_listeners is data: a provider restore rewinds it, and a bind-clocked registration would stay silently gone until the next deploy. The broker’s ensure pass re-converges its registrations on its own clock, which also makes secret rotation self-healing: the read-back never returns the stored secret, so each registration carries a fingerprint of it in the plain headers, and a mismatch is a delete-and-recreate. The socket’s integration job does what deploy-time is actually good for: it proves the surface (app enabled, migration applied, the one occ command answering JSON) so a bind against an instance that cannot serve subscriptions fails loudly at bind time.

The uninstall leak is documented, not hidden. Detaching the edge cannot deregister (the runner never saw the rows, and there is no occ delete to script); the socket header carries the manual sweep. The cost of a dead callback is one logged provider-side warning per matching event: bounded, and strictly cheaper than an unintegration job holding admin HTTP credentials it has no other reason to have.


No. A socket integration job runs to completion inside the deploy; the deploy waits on it. A job that loops is a deploy that never finishes, and a job that runs once is not a convergence loop. The discovery phase is the sanctioned bounded wait (max_retries × retry_interval) and it exists to prove readiness, not to converge state.

A broker needs a credential the socket minted. Where does it read it?

Section titled “A broker needs a credential the socket minted. Where does it read it?”

Seal it. The socket’s integration job mint-and-returns the value; the consumer’s post_install/post_update hook is the only render that can see it (gap 2); that hook re-emits it as ::shc-hook:secret:<key>=<value>::; PersistHookMintedVars routes it through the vault and persists a ref+vault://…?autogen ref into vars.yaml. From then on it is an ordinary var and the broker’s compose reads it like any other. apps/devops/gitlab/gitlab-seal-sentry-dsn.job.yaml is the reference implementation and carries the ce line references.

Does detaching the edge revoke what the broker created?

Section titled “Does detaching the edge revoke what the broker created?”

No. unintegration tears down what the bind built; per-user artifacts are not bind-time state and the runner has never seen them. Revocation is a label-keyed sweep the operator runs by name, deliberately standalone so that detaching one edge cannot revoke credentials another surviving edge still uses. Say so in your socket header the way mailcow.socket.yaml and matrixAppservice.socket.yaml do. A silent leak and a documented one are different operational facts.

Should the broker be one component per pair, or one for the estate?

Section titled “Should the broker be one component per pair, or one for the estate?”

Per pair for edge-brokers (that is what “the bridge IS the broker” means); one for the estate for identity-brokers (the realm is a singleton). Resist the merge in the first direction: a single “integration service” that switches on pair type is the provider-side placement failure wearing a different hat, and every new pair becomes a change to a component every other pair depends on.

When is a ledger needed, rather than plain idempotence?

Section titled “When is a ledger needed, rather than plain idempotence?”

Idempotent creation covers the create side. Ask what happens when the artifact is absent and the loop did not just create it. If “do nothing” and “recreate” are both defensible answers, the fact that decides between them is missing, and that fact is the ledger.


  • Integration doctrine: the direction test and the ruling that a bridge is never its own top-level deployment
  • Plugs and sockets: the deploy-clocked half, and where a broker’s capability comes from
  • vApps: the other noun in the decision table
  • Lifecycle hooks: the ten deploy-clocked events, and the hook render a mint-and-return lands in
  • Credential reconciliation: making vault the source of truth for a persisted credential
  • docs/proposals/2026-08-08-identity-and-edge-brokers.md: the proposal this page promotes; components 1 and 2 there are still unbuilt
  • docs/proposals/2026-08-06-cert-delivery-broker.md: the same argument made for certificate renewal; read its status header before citing it
  • docs/proposals/2026-08-08-in-stack-exec-broker.md: a differently-shaped “broker” (request-time verbs in a sibling service); superseded by the docker socket’s own-container view (SHC-3333)