App definition
Layer 6. Services. Services
An app in SHC is a directory containing one app.yaml plus a set
of supporting files. Source of truth: the app-dir validator behind
shc validate (modules/app/validate.go, ce module). There is no
single AppDefinition struct; each consumer reads the keys it owns
(variables: modules/app/var_types.go, hooks: modules/app/hooks.go,
plugs/sockets: modules/app/integration_service.go).
Directory layout
Section titled “Directory layout”apps/myapp/├── app.yaml # required: top-level metadata & configuration├── shc.yaml # required, or compose.yaml: the app's services, SHC's own schema├── compose.yaml # the import path in place of shc.yaml: Jinja-templated Docker Compose├── vars.yaml # optional: non-declared default variable values├── backup.yaml # required once the app has a named volume: the mounts a backup copies, and the ones it keeps├── *.socket.yaml # optional: sockets this app provides├── *.plug.yaml # optional: plugs this app consumes├── *.vsocket.yaml # optional: virtual sockets├── *.vapp.yaml # optional: virtual app groupings├── *.vplug.yaml # optional: virtual plugs├── *.vvars.yaml # optional: variables injected by a vApp└── *.job.yaml # optional: job definitions referenced by hooks / crons / integration phasesAll app.yaml extra keys are allowed (model_config = ConfigDict(extra="allow")): SHC ignores fields it doesn’t
recognize rather than rejecting them. This is intentional so apps can
carry custom metadata (e.g. for a UI) without SHC needing to know
about every key.
app.yaml top-level fields
Section titled “app.yaml top-level fields”Identity & discovery
Section titled “Identity & discovery”| Field | Type | Description |
|---|---|---|
name | str (required) | App name; must match the directory name. |
version | str | Default "1.0.0". Displayed in listings; doesn’t gate anything in SHC. |
description | str | One-line human summary. |
category | str | Free-form category (cms, database, collaboration, …), used for filtering in the HUD. |
icon | str | Local path (icon: icon.svg, a .svg/.png file beside app.yaml, ≤128 KiB, no script) is the servable form: the daemon stamps it into the launcher’s apps.json and materialises the bytes for the Traefik plugin. A remote URL is display metadata only, never fetched at runtime (see traefik/plugin/icons.go), so prefer mirroring the art into the app. Apps with no local icon fall to a category glyph, then a monogram. Guarded by tests/unit/catalog/app_icon_declaration_test.go. |
homepage | str | Upstream project URL. |
source | str | Source-code URL (used by app-source-resolution; see source-resolution). |
tags | list[str] | Free-form tags for search. Whatever the appr typed: there is no vocabulary and no validation. Contrast features below, which is the same shape of chip and the opposite kind of claim. |
features | list[str] | map | What the APPLICATION supports (SSO, LDAP, email, webhooks and twelve more) from a CLOSED vocabulary; a value that is not in it is completely ignored. Tri-state (supported / paid / absent), and unknown is the default. Never describes what this app wires: that is derived separately. See App features and openness. |
hidden | bool | Three-state. Overrides the catalog’s derived visibility default; see Catalog visibility below. Visibility only: never affects publication or install. |
private | bool | Default false. Excludes the app from publication entirely; see Publication below. |
house_only | bool | Default false. Publishes the app to the house registry only, never to the public catalogue on selfhostedcloud.org; see Publication below. |
flagship | str | The league this app is SHC’s single best-in-class pick for; see Flagship below. Owner-assigned; absent for the overwhelming majority of apps. |
gallery | list[map] | Storefront screenshots; see Gallery below. Every entry carries a REQUIRED source: provenance discriminator. Opt-in; almost every app declares none and renders exactly as it did before this key existed. |
license | map (required) | The licence of the software this app deploys; see License below. Every shippable app must declare one; unverified is a legitimate answer. |
Features
Section titled “Features”features: is what the application supports. It is the sibling of tags:
and the opposite kind of claim: a tag is free-form and a feature comes from a
closed vocabulary, so that a feature chip is something a shopper can filter
on rather than something somebody typed. A value not in the vocabulary is
completely ignored: not rendered, not scored, not an error.
features: [sso, email_outbound] # list form: every element is `supported`, # everything else stays `unknown`features: # map form: the full tri-state sso: absent mfa: {state: paid, note: "MFA is Premium"}It never describes what this app wires. An app that has SSO declares
sso: supported whether or not the app ships an oidc.plug.yaml, because
the app is not at fault for SHC not having caught up. The SHC side is derived
from the app’s own files and is never declared; make catalog/capabilities
computes both and prints the gap.
unknown is the default and must never be read as absent. The full
vocabulary, the state and via tokens, the license.openness axis and the
storefront seam are all in
App features and openness; the generated two-axis matrix is
App capabilities.
Gallery
Section titled “Gallery”gallery: is the app’s storefront screenshots. It is opt-in: almost
every app declares none, and an app without one renders exactly the page
it rendered before this key existed: no empty carousel, no placeholder.
For how gallery: and landing: fit together (i.e. everything an app author
controls on their marketplace page, and the one surface they deliberately do
not), see Storefront pages.
gallery:- path: gallery/dashboard.webp # 1200px canonical, inside the app dir source: captured # captured | vendor | none (REQUIRED) captured_from: "1.24.3" # the version of the frame's SUBJECT width: 1200 height: 750 caption: The project dashboard # this is the image's ALT TEXT| Key | Required | Meaning |
|---|---|---|
path | for captured/vendor | Repo-relative to the app directory, .webp only. Produced by scripts/encode-gallery-asset.sh, which also writes the -960w/-640w srcset rungs beside it. |
source | always | captured (we took it, on an instance we run) · vendor (upstream granted it) · none (we looked and have nothing; carries NO path, and the caption is where the reason goes). |
captured_from | for captured | The version of the frame’s subject when it was taken. |
width / height | for captured/vendor | Intrinsic pixels. The storefront reserves the box from this ratio so a lazily-arriving frame cannot reflow the page. |
caption | always | What the frame shows. Rendered as alt: a gallery of unlabelled frames is inaccessible, not decorative. |
addon | optional | Names the add-on this frame depicts (<vapp>/<name>, or a bare <name> for an add-on that declares no vapp:). Absent = the frame is of the product. |
grant_url / grant_read_on | for vendor | The page whose terms permit the use, and the day it was read. |
source: is the load-bearing field.
docs/proposals/2026-08-16-marketplace-screenshots.md
§0.9 measured why: 68 of 90 apps carry license: {id: unverified}, a code
licence does not grant screenshot or trademark rights in any case, and SHC is
commercially deployed. An asset with no source: is an image nobody has said
anything about; the catalogue guard refuses it, the seeder refuses to publish
it, and the freshness verifier calls it BROKEN rather than merely unverified.
An add-on’s frames are declared on the owning app, not on the add-on.
A gallery: key inside a *.addon.yaml is the obvious-looking home and is
currently unshippable: ce’s v-document lint closes the add-on document to a
fixed key set and refuses anything else (X400341VDCUNK), correctly, because
a key no loader reads is parsed and dropped. So the entry lives in app.yaml
and names its subject with addon:.
Guards. tests/unit/catalog/app_gallery_declaration_test.go holds the
declarations (provenance, resolvable paths, the srcset rungs, the per-asset
and per-listing byte budgets, and that no EXIF/XMP/ICC survived the encode;
these frames come off live estates). make catalog/gallery-freshness
(cmd/verify-gallery-freshness, also run by make lint) compares
captured_from against the subject’s current version and reds when a frame
trails by a major, because a major is exactly when a UI is redrawn.
Catalog visibility
Section titled “Catalog visibility”hidden: is the author’s catalog-visibility override. By default the
catalog derives visibility itself (an item is listed iff it is
installable with no required free-form input), and hidden overrides
that verdict in either direction: hidden: true takes an item off
the wall the rule would list, hidden: false forces one on that the
rule would hide. Absent means “let the rule decide”, which is the
right choice for almost every app.
name: bucketapp: s3type: vapphidden: true # a bucket is provisioned by an integration, never picked off a listingHidden affects nothing but the listing. A hidden item still
publishes to the registry, still syncs to every node, and still
installs by exact name: shc install s3/bucket works exactly as
before. If you need an app kept out of the registry itself, that is
a different declaration: private:.
The same key is available on a vApp definition; see vApps.
Publication
Section titled “Publication”private: true declares that an app is not published at all: an
E2E fixture, an internal building block, anything that must never
reach the app registry:
name: test-simpleprivate: true # E2E fixture, not a product appIt is enforced where the archives are built: the archive builder
skips any app that declares it, so a private app never reaches
a node through a synced repo. That is what replaced the builder’s old
test- name-prefix rule. A name is not a declaration, and an app
legitimately called test-runner now publishes like any other.
Private is not disabled. shc install test-simple still resolves
the app by exact name from a local apps tree, which is what the
e2e suite depends on.
The two keys are deliberately separate: hidden answers “is this a
choice on the catalog wall?”, private answers “does this exist in
the repository at all?”. An author who hides an app keeps it
installable everywhere; an author who marks it private keeps it out of
the repository altogether.
house_only: true sits between the two. The app is built and published
to the house registry like any other, and scripts/publish-apps-site.sh
leaves it out of the public catalogue on selfhostedcloud.org. It is for
an app whose image or source only the house can reach, or that is a
product integration rather than a catalogue app:
name: multiplatformhouse_only: trueThe publisher refuses any value but true or false, so a typo never
publishes a house app to the world.
The store seed (frappe/apps/marketplace/scripts/import_catalog.py)
treats house_only: true like hidden: true: the app’s listing and its
add-ons’ listings are unpublished, and the seed refuses to run when a
house-only row would be published
(tests/unit/marketplace/seeder_public_seed_test.go).
Until the house has a private index of its own (FZ-8130), house_only
also takes the app out of the only index Ogden reads, so a house app that
Ogden runs from the catalogue stays hidden: true and keeps
house_only off until that index exists.
Flagship
Section titled “Flagship”flagship: <league> declares that this app is SHC’s single
best-in-class pick for one function (its league: git, mail,
knowledge, …). The full tier definition, the requirements a flagship
must meet, and the exception process live in
the flagship-tier decision.
name: forgejoflagship: git # owner-assigned; exactly one app per leagueThe label is owner-assigned: do not add it in an app PR without an
owner ruling. Its mechanical half is enforced by
scripts/arch-fitness.sh’s flagship-contract check (hard-fail in
make lint): a labeled app must carry an oidc/saml/ldap plug or
a documented userless access: posture, backup.yaml,
tests/tests.bats, and no two apps may claim the same league. The
league token must be an unquoted lowercase ^[a-z][a-z0-9-]*$ word.
A -classic app can never carry the label: classic lines are
unmaintained upstream by definition and the maintained-upstream
requirement has no exception process (owner ruling 2026-08-06).
A second hard check, flagship-league-roster, pins the leagues the
owner has actually RULED (today analytics → superset, errors →
glitchtip, replay → openreplay, uptime → uptime-kuma, and storage
→ VACANT) as a set equality with the catalogue. So the two ways
this key goes wrong in an app PR both fail by name: deleting or
moving a ruled declaration (the contract check cannot see a label
that is not there), and inventing a league token that no roster row
carries. Adding or moving a row is how an owner ruling enters the
machine; it is not an app-PR edit. A labelled app must also live in
apps/flagship/ (TestFlagshipLabelLivesInFlagshipBucket): the
folder is integration depth, the label is best-in-class, and only the
label→folder direction holds. See
App buckets.
The label rides the ordinary metadata path: app.yaml →
shc catalog build’s meta → the per-version manifest.json → the
registry index.yaml head (flagship: beside category:; omitted
when unlabeled). Older consumers ignore the key.
License
Section titled “License”license: records the licence of the software this app deploys,
not the licence of the packaging, which is SHC’s own. Every shippable
app must declare one:
license: id: EPL-2.0 source: docs/guide/apps/custom-builds.md| Sub-field | Type | Description |
|---|---|---|
id | str (required) | An SPDX license expression, or the literal unverified. |
source | str | The in-repo file that verifies the claim. Required unless id is unverified. |
note | str | Anything an operator needs to know: an obligation, a carve-out, a caveat. |
components | list | Bundled third-party software whose licence differs from the app’s own. Same name/id/source/note shape. |
no_saas | str | The restriction the licence puts on offering the software to third parties as a hosted service, in one or two sentences: O’Saasy, SSPL, Dify’s multi-tenant clause. Absent means the licence carries none. The store shows it on the app page. |
noncommercial | str | Set when the licence permits noncommercial use only: one or two sentences on what commercial use needs instead. The store shows it on the app page as “Noncommercial use only”. |
openness | map | ★ Editorial. How OPEN the app is to somebody who wants to run it: model (foss/open-core/source-available/proprietary), pricing (free/flat/per-user/per-resource), seat_cap, and the required reviewed_by/reviewed_on that mark it a human judgement. Orthogonal to id in both directions. See App features and openness. |
The store’s app page shows three of these to a shopper:
requires_paid_license with its requires_paid_license_reason (see
app buckets), openness.seat_cap with the
openness.note that says what happens at the cap, no_saas and
noncommercial. An unverified id is never shown.
The id grammar is SPDX license-expression syntax: bare tokens
joined by OR, AND or WITH: Apache-2.0, AGPL-3.0-or-later,
MIT AND GPL-3.0-only AND Apache-2.0. Do not quote it. A licence with no
SPDX identifier rides the SPDX-standard LicenseRef- prefix, so
source-available and one-off vendor terms stay in the same grammar as
everything else and a consumer needs one parser rather than two:
license: id: LicenseRef-RSAL-2.0 OR SSPL-1.0 OR AGPL-3.0-only source: apps/standalone/redis/vars.yamlThat is a real OR: Redis 8 is tri-licensed and the licensee elects,
holding exactly what they already have. -only rather than -or-later
because Redis 8’s per-file headers offer “(c) the GNU Affero General
Public License v3 (AGPLv3)” and nothing about later versions.
Several of these terms do have SPDX identifiers and should use them
rather than a LicenseRef-: SSPL-1.0, BUSL-1.1, Elastic-2.0.
Use the -only / -or-later forms. Bare GPL-2.0, GPL-3.0,
LGPL-* and AGPL-3.0 are deprecated SPDX identifiers: SPDX split
each one because the bare token cannot say whether the project’s “or (at
your option) any later version” election applies, and that election is a
different licence. The answer is in the project’s per-file headers, not
in its LICENSE. The LICENSE carries the same GNU text either way.
OR means the licensee may elect, and nothing else. Do not use it to
express that an app can be configured to deploy a different image
under different terms. That is not an election a licensee can make from
what they hold, and an OR there conceals which arm actually ships,
which is exactly what it did on openobserve until 2026-08-07. Declare
the arm the default render takes, as a single token, and put the opt-in
alternative in note:. See
licensing postures, “Two images,
one app”.
unverified is a legitimate answer, and it is the required one when
you do not know. It means no in-repo source establishes this app’s
licence, not “probably MIT”, not “the README says Apache”. Do not read
an upstream licence off the internet and record it here, and do not
infer one from a project’s reputation: a confident wrong answer is worse
than a counted gap, because nobody re-checks it.
license: id: unverifiedAn unverified entry is a work item for a human, and the queue is a
one-line query:
grep -rc 'id: unverified' apps --include=app.yamlResolving one means finding or writing an in-repo source (a review
under docs/reviews/, a decision record, a section in
licensing postures, or a verified
statement in the app’s own vars.yaml) and then citing it in
source:. That citation is the whole point: it is the difference
between an answer and a guess that reads like one.
Bundled components. Where an app ships third-party software under
different terms, record it rather than flattening it into the app’s own
id:
license: id: EPL-2.0 source: docs/guide/apps/custom-builds.md components: - name: elasticsearch:7.14.2 id: Elastic-2.0 OR SSPL-1.0 source: docs/guide/apps/licensing-postures.md - name: redpandadata/redpanda:v24.3.6 id: BUSL-1.1 source: docs/guide/apps/licensing-postures.mdThis matters most for the no-SaaS family (SSPL, RSALv2, Elastic 2.0, BUSL 1.1). None of them stop an operator self-hosting for their own estate, but they do constrain offering the component itself to third parties as a service; see licensing postures for the catalogue’s standing posture on each.
The enterprise marker: requires_paid_license. id: records what
licence the code is under; it can never say whether an SHC estate has to
pay to run the app (or to get the SSO an estate needs). The two are
orthogonal in both directions (an AGPL app can paywall SSO; an
SSPL-family app can be free to run). The ruled marketplace schema
(marketplace architecture §5)
adds two keys inside the license: block:
license: id: unverified requires_paid_license: true # ★ does an estate have to PAY? requires_paid_license_reason: >- Class (b). SAML, JWT and OIDC SSO are paid-tier features …| Sub-field | Type | Description |
|---|---|---|
requires_paid_license | bool | true iff the app requires a paid licence to run, or the SSO an estate needs is behind a paid tier with no free equivalent. Absence means false. |
requires_paid_license_reason | str | Required when the boolean is true: what specifically costs money, and for whom. Refused without the boolean. |
The marker is a biconditional with the apps/enterprise/ bucket:
tests/unit/catalog/app_bucket_taxonomy_test.go’s
TestEnterpriseFolderAndDeclaredFieldAgree fails either record without
the other, a true with no reason, a reason with no true, and (the
spelling ratchet) the retired pre-ruling key names. Marketplace listings
badge from this field (“commercial license, available through us”), so
a wrong answer here is a wrong storefront claim. See
app buckets §3 for what counts as enterprise: a
source-available licence (SSPL/BSL/ELv2/RSALv2/FCL) restricts
redistribution, costs a self-hoster nothing, and is not enterprise.
Provisioning the upstream licence: a plain var. The marker above
says whether money changes hands; how the purchased licence reaches
the deployment is an ordinary @secret deployment var the app
declares and its compose template if-guards on: windmill’s
license.key (shc install windmill --var license.key=<key>, exported
as LICENSE_KEY on server and worker) is the worked example. There is
deliberately no license magic socket: one existed briefly (the
2026-08-14 marketplace ruling §5 scaffolding, a master-tenant license
provider behind -i license=LICENSE), and it was amputated by owner
ruling 2026-08-18. Cluster-side licence enforcement lives in shcd, and
licence management (purchase, custody, vendor APIs) lives on the
portal’s Frappe backend beside the marketplace, not in an estate app
(docs/decisions/2026-08-18-builtin-vs-deploy-placement.md). When the
custodial machinery lands it arrives portal-side; an app should only
ever declare the var its software actually reads.
What the fitness check enforces. scripts/arch-fitness.sh’s
app-license-declared check (hard-fail in make lint) requires the
declaration, the mapping shape, the id grammar, and a source: on any
non-unverified claim. It walks apps/** recursively and skips only
private: true apps, mirroring the archive builder’s definition of
shippable.
Built-ins are a second corpus, and you edit their source. The five
apps/BUILTINS apps (postgres, keycloak, openobserve,
traefik, branding) ship inside the binary rather than through the
catalog. make modules/stage assembles them into apps/<app> as a build
product nothing commits, and make lint is hermetic (no configure
prerequisite), so in CI apps/ never contains one: the catalog walk
above had never seen them. They are checked separately by
addon-license-declared, which reads the source trees the same way
make modules/stage does (addons/<app>/ in shc, else the ce module’s modules/<app>/ at
the committed pin) and refuses by name if any of the five fails to
resolve. A local ce checkout is deliberately not consulted: the subject is
the payload the deb ships, and that payload is assembled from the pin. Add a
license: to the source app.yaml; an edit to a staged apps/<app> copy is
overwritten by the next make.
It deliberately does not check that a declaration is true: no grep can read an upstream LICENSE file. A declaration can still be wrong; what cannot happen any more is an app shipping with no answer at all, and that is the part that made licence questions unanswerable before the field existed.
Requirements
Section titled “Requirements”requires: declares what the app needs from the host node.
requires: runtimes: [docker, podman] storage: [local, nfs] # storage-provider names this app can target architectures: [amd64, arm64] platforms: [linux]requires.runtimes: is an ORDERED preference list
Section titled “requires.runtimes: is an ORDERED preference list”The first entry is the app’s preferred runtime; later entries are acceptable
fallbacks. runtimes: [docker] restricts the app to Compose. An absent list
expresses no preference. Each node runs one runtime, and the install gate
refuses a runtime the app does not accept.
The reader is modules/app/required_runtimes.go in CE; enforcement runs in
required_runtimes_gate.go before install records durable intent, with a
render-time check for sources that the early gate cannot read.
Listening ports and publication
Section titled “Listening ports and publication”expose: and ports: in compose.yaml declare the ports a container listens on. They do not publish the deployment. Publication is --expose on shc install and shc update (modules/app/commands/install.go). A bare --expose <hostname> routes to the service marked x-shc-primary.
Host roles (host_roles)
Section titled “Host roles (host_roles)”host_roles: dtrack-apiserver: apihost_roles maps compose service name → role, and names the services a
browser must not be sent to. It exists for one consumer: the app
launcher draws one tile per deployment, so a deployment routing more
than one host has to choose which host that tile links to.
Its fallback for choosing is the operator’s --expose declaration order
(published by the daemon as the primary flag on the apps.json record),
and that order does not survive Terraform. shc_deployment.routes is a
map; the provider sorts its route keys before lowering them to --expose
argv; and - (0x2D) sorts before . (0x2E). So <name>-api sorts before
<name> every time, primary lands on the API host, and the deployment’s
single tile opens raw JSON in a new tab. Nothing warns, and the operator
chose no order at all; a map has none.
Rules worth knowing before you write one:
apiis the only role. Browser-facing is spelled by leaving the service out; a literalbrowser:row is inert. An unrecognised role resolves to browser-facing too, so a typo can only ever fail to demote a host, never hide one.- It is an app declaration, not an operator knob: the same rule
ungated_pathsfollows. Which of an app’s images answers browsers is a fact about the software the app deploys and is identical on every estate; the hostname is the operator’s. - The join is on the expose’s service half. The daemon matches this map
against each routed host’s
service=half (host_mapping.service_name). A host installed with no service named carries no role and stays browser-facing. A deployment whose hosts front different services must name them anyway: the unique-exposer rule refuses an ambiguous target. - It can only demote, never delete. A deployment whose every routed
host is marked
apistill gets a tile, pointing at itsprimaryhost, because the marker reorders candidates within a deployment rather than removing them. - It is re-read from the catalogue copy of
app.yamlby the apps-menu sweep, so repackaging an app changes the marker without a reinstall.
The launcher consults it before declaration order; see the launcher flow.
Launcher placement (launcher_placed)
Section titled “Launcher placement (launcher_placed)”launcher_placed: falselauncher_placed: false declares that the app is not a launcher
destination at all: the daemon still writes its routed hosts to
apps.json (stamped "placed": false) so the plugin can resolve the
host’s tenant, and the plugin omits it from the tile list. It exists for
apps whose routed host exists to be called, never opened. The
canonical declarer is gitlab-huly, a webhook bridge with no page a
person uses. Sidecars that do have a URL a person might open from a
parent menu use belongs_to instead.
Rules worth knowing before you write one:
- Absent means placed. The key can only ever remove a tile, and the
daemon’s read is bool-typed: a quoted
"false", a typo, an unreadable file all degrade to placed, so a mistake can never silently cost an estate a tile. - It is an app declaration, not an operator knob: the same rule
host_rolesfollows. Whether an app is a destination or an embed is a fact about the software the app deploys. (A scopedlauncherPlacedconfig key for operators is planned but not yet built.) - It is not
host_roles: api. The role marker chooses which of a deployment’s hosts the one tile links to and can never delete the tile; this key says there is no tile. Different sentences, different keys. - An unplaced app’s own host still gets the launcher, because the
plugin resolves a host’s tenant from the same
apps.jsonentries and only omits the unplaced app from the tile list. - The declarer list is closed by a catalogue guard
(
tests/unit/catalog/launcher_placed_test.go): removing a tile is an estate-visible act, so a new declarer must be added there deliberately.
Launcher parent (belongs_to)
Section titled “Launcher parent (belongs_to)”belongs_to: nextcloudbelongs_to names the parent catalogue app. The launcher shows one
icon per deployment; a sidecar of that deployment does not get its own tile. It
appears on the parent tile’s right-click (or long-press) menu.
The canonical declarer is nextcloud-picker. The three Frappe benches
(bit, erpnext, ffx) are three deployments of one catalogue app and must
not declare this: folding on frappe would be ambiguous across them.
The daemon stamps "belongsto" on the apps.json entry, and the stamp is
the plugin’s only source: it carries no table of sidecars.
Native clients (native_clients)
Section titled “Native clients (native_clients)”native_clients: user_agents: [HulyTracker] globals: [__HULY_NATIVE__]The launcher stays out of a page the app’s own native client opens, since
a native app has its own navigation. Android’s ; wv) and iOS’s missing
Safari/ are caught by the platforms’ own marks; a desktop shell, or a
WebView that sends a stock user agent, is known only by what it sets
itself. user_agents are tokens the client appends to its user agent,
globals are window properties it sets to true; either one keeps the
launcher off that page. The daemon stamps them on every apps.json entry
of the app’s deployments, and the plugin reads them per host. A token is
a letter or digit followed by letters, digits and ._/-, a flag is a
JavaScript identifier, each at most 64 characters; anything else is
dropped.
Gate exemptions (ungated_paths)
Section titled “Gate exemptions (ungated_paths)”access: openungated_paths: - /accounts/loginungated_paths names path prefixes that skip the shc-oidc forward-auth
middleware while the rest of the host stays gated. It only has an effect on
a deployment whose access axis resolves to gated; an open deployment
renders no exemption routers at all.
Gating is binary. An app is public and guards itself with its own login, or
it is gated whole. An app whose API clients, mobile apps or agents cannot
sign in through the gate ships access: open; it does not carve paths out
of the gate. Exactly two carve-outs exist, each a single caller that cannot
carry the gate cookie: docuseal’s signing links (/s/<token>) and
paperless’s bare /accounts/login, where swift-paperless fetches its
csrftoken. TestOnlyTheRuledCarveOutsDeclareUngatedPaths
(tests/unit/catalog/public_or_gated_test.go) fails the unit job when any
other app declares the key.
How a carve-out renders:
- The prefix is written in the app’s own URL space. A route mounted at
--expose docs.example.com/paperlessrendersPathPrefix(/paperless/accounts/login). "/"is refused, as is any entry not rooted at/and any entry containing a backtick.- Exempt routers are stamped below the OIDC-callback router, so an
exemption cannot steal
/.shc/auth/callbackfrom the forward-auth sidecar. - An exempt router carries no gate and no launcher. The request reaches the
app’s own container with its
Authorizationheader untouched.
The Traefik-host reconcile reads this from the deployment’s verified
declaration (the stored package, else the catalogue copy), so a repackaged
exemption reaches a deployment on its next shc update. An unreadable
declaration yields no exemptions: the unreadable case fails shut.
Default exposure (exposure)
Section titled “Default exposure (exposure)”exposure: internetexposure is the exposure a deployment of this app gets when shc install
does not name one. It takes lan or internet, trimmed and in any case;
absent means lan. --public on shc install overrides it to internet
(ResolveExposure, ce modules/app/inject_traefik.go). The install result
prints the value it resolved as exposure: lan or exposure: internet.
What it changes in the rendered ingress:
- The LAN allowlist. Under
lanEnforcementallowlist, the default, every router a non-internetdeployment renders gets the deployment’sipAllowListmiddleware first in its chain: the gate exemptions, the gRPC and TCP routers and the auth callbacks included. Aninternetdeployment gets none. The test is “notinternet”, so an unset or misspelled value is restricted. SeelanEnforcement. - The certificate. HTTP-01 ACME is requested only for an
internethost whose public entry is reachable, because the CA has to dial port 80 of that host. Alanhost gets a certificate by DNS-01 when a DNS-01 connection owns its zone, and the SHC leaf otherwise. - The DNS record. An
internetdeployment publishes the public address ladder and never readsdns.lan_addressordns.public_address; alandeployment publishes those first. See the cluster entrypoint.
Rules worth knowing before you write one:
- It is read at install and stored on the deployment.
shc updatekeeps the stored value unless--publicor--privatechanges it, so a repackaged default does not move a deployment already installed. - It is a default, not a limit:
--publicraises alanapp, andshc update --privatewalks aninternetone back. - It is the exposure axis only.
internetdoes not ungate a gated app; the access axis isaccessand--gated/--open. See access control.
In the catalogue, openreplay declares exposure: internet: its tracker’s
audience is off the estate, and a LAN-only recorder records nothing. Most
other declarers say lan. The worked example is
Your own ingress, whose app
installs as internet with no --public typed.
gRPC prefixes (grpc_paths)
Section titled “gRPC prefixes (grpc_paths)”grpc_paths: - /pkg.Service/ - path: /management.ManagementService/ service: netbird-management port: 80grpc_paths lists the path prefixes under the app’s hostnames that carry
gRPC. An entry is a prefix, or a map naming the prefix (path) and the
compose service and container port that serve it. Each prefix gets a
sibling router, <router>_grpc_<n>, on every host the deployment routes,
on the websecure entrypoint with the rule
Host(<host>) && PathPrefix(<route prefix><entry>). Its service dials over
h2c://, cleartext HTTP/2, instead of http://: the named service and
port, or the route’s own container and port for an entry that names none.
The prefixes that dial the same upstream share one service (ce
modules/app/grpc_paths.go, inject_traefik.go).
Name the service whenever a host of the app is fronted by a service that does not serve the prefix. netbird’s dashboard is its primary host and serves static files, so an entry naming no service sent the management API’s gRPC calls to the dashboard there, which answered HTML.
It exists because Traefik proxies an http:// backend over HTTP/1.1, and a
gRPC server that shares its listener with a REST mux cannot recognize an
HTTP/1.1 request as gRPC. It answers 404 while every other path on the host
looks healthy. NetBird was the measured case: its dashboard answered,
its API answered 401, and no peer ever finished enrolling.
Rules worth knowing before you write one:
- A gRPC router carries no gate and no launcher. The gate cannot
evaluate the bearer a gRPC client brings, and the launcher’s body
rewriter corrupts gRPC frames. It keeps the platform’s gate-cookie strip
and, on a
landeployment, the LAN allowlist. - It is rendered for every access mode,
openandgatedalike. - The prefix is written in the app’s own URL space, as
ungated_pathsis: a route mounted at--expose h.example.com/apiserves/api/<entry>. - An entry not rooted at
/, the bare/, and any entry containing a backtick are dropped. So is an entry naming aservicewithout aportor the reverse, a service containing whitespace, a backtick,/or:, and a port outside 1 to 65535. The first entry of a prefix wins, and declaration order is kept so the router names stay stable across sweeps. - The named service joins the ingress network at install. The fabric
grants the system Traefik what a host binding names, so the service:port
an entry names should be bound by one of the deployment’s hosts too, as
netbird’s API hosts bind
netbird-management:80. - The gRPC routers sit above the
ungated_pathsrouters, so a prefix in both lists still dials h2c, and below the OIDC callback router. - The Traefik-host reconcile re-reads it every sweep from the deployment’s verified declaration (the stored package, else the catalogue copy), so repackaging adds or removes a gRPC route without a reinstall. An unreadable declaration yields no gRPC routes.
In the catalogue, netbird declares /management.ManagementService/,
served by netbird-management on port 80. The
worked example is Your own ingress,
which calls a gRPC server through the ingress with grpcurl.
Cloud discovery (discovery)
Section titled “Cloud discovery (discovery)”SHC serves /.well-known/cloud/v1 on every host it routes: a document a
Based Linux first boot reads to learn which services a domain has and which
provider signs each one in. An app adds its rows with discovery:; an app
without one adds nothing, and the reader runs its own discovery for what the
document does not state.
discovery: service: synapse # optional: only this compose service's host is the entry point kinds: - kind: chat # mail, files, calendar, contacts, chat or vpn type: matrix # the protocol or product within the kind method: bearer # the credential the client ends up holding login: own # optional: the app holds its own accounts realm_when_federated: true # sign in at the realm when gated or oidc-federated server_name: true # stamp server_name from the host at: true # stamp the entry point from the host mint: {service: apppass, path: /apppass} # a route that mints app passwords over a realm token broker: {service: apppass, path: /apppass, endpoint: /v1/app-password, scope: broker-mail} issuer: {service: mas, id: matrix, label: Matrix account} # an issuer the app runs in front of the realm plugin: passwords # published only with this add-on installed opt_in: true # published only once the estate switches the kind on device: # instead of kinds: the document's device_management member kind: fleet broker: {service: fleet-enroll-broker, path: /enroll-broker, endpoint: /v1/enroll, scope: broker-fleet}A route the block names (mint, broker, issuer) counts only when the
same deployment publishes it. A kind outside the six above is published only
with opt_in, because a reader refuses a whole document that names a kind it
does not know. mailcow, nextcloud, matrix, netbird and fleet in the catalogue
are worked examples.
Raw TCP on the system Traefik (tcp_ingress)
Section titled “Raw TCP on the system Traefik (tcp_ingress)”tcp_ingress: - name: ssh service: gitlab port: 22tcp_ingress names a service that speaks raw TCP and the system Traefik
entrypoint it wants. name is the entrypoint, service the compose
service, and port the port it listens on in its container. ssh is the
only entrypoint the traefik module opens today (ce
modules/app/tcp_ingress.go).
The app declares one half. The platform holds the other: the public port,
and the ONE deployment the entrypoint forwards to, as two vars of the
traefik system app. The entrypoint is armed only when both are set and
well formed; the deployment is <tenant>/<environment>/<stack>. The port
is static Traefik configuration, so the vars take effect when the system
app is updated in the master tenant it lives in:
shc config set modules.traefik.vars.ssh.port 2222shc config set modules.traefik.vars.ssh.deployment prod/prod/gitlabshc -t master update traefikThe system Traefik comes back with the entrypoint, and the next ingress
sweep gives the deployment a TCP router, HostSNI(*) on that entrypoint,
forwarding to the service’s fabric address and port. One deployment per entrypoint
is the point: a TCP router without TLS cannot tell two deployments apart,
so a second would receive the first’s traffic.
Rules worth knowing before you write one:
namematches^[a-z][a-z0-9-]*$, andwebandwebsecureare refused: aHostSNI(*)router there would capture every TLS connection no HTTP router claims.serviceis non-empty with no whitespace, backtick,/or:.portis 1 to 65535. An entry that fails is dropped, and the first entry of a name wins.- On a
landeployment the TCP router carries the same LAN allowlist as its HTTP routers, so the port admits exactly the networks the web UI admits. - A declaration with no binding renders nothing: the app can say what it speaks, never which port the edge gives it.
- The armed binding is also the fabric policy’s word. The system Traefik
reaches the declared service on the declared port the way it reaches a
routed host, and nothing else of the deployment; clear the binding and
the grant goes with it.
shc -t master cluster fabric explain traefik own:echo --target-tenant prod --target-environment prod --port 2222reads it as an ingress route.
In the catalogue, gitlab declares ssh on its gitlab service, port 22,
for git over SSH. The worked example is
Your own ingress, which binds
the entrypoint and reads a line over it from two nodes.
Delivered certificates (certs)
Section titled “Delivered certificates (certs)”certs: - reload: dovecot-mailcow: restart postfix-mailcow: restartA service that ends TLS itself asks SHC for a hostname’s certificate by
binding /etc/shc/tls.serve/<host> read-only in its compose file. The
mount is the request, so certs names no host. SHC delivers the chain
and key under current/ (current/fullchain.pem,
current/privkey.pem); a rotation writes a new serial directory and
flips the current symlink in one rename, keeping the previous serial on
disk for a reader that opened it just before (ce
modules/certbroker/materializer.go).
certs is the one thing the mount cannot say: the reload contract,
what to do to each service when the material rotates. Per service, either
restart or an argv list to run in the running container:
certs: - reload: postfix-mailcow: restart dovecot-mailcow: ["doveadm", "reload"]Rules worth knowing before you write one:
- No
certsblock means restart every service that binds the mount. A service that binds it but is missing fromreloadis restarted. - A deployment may bind only a host the platform already routes to it.
Binding another deployment’s host is refused with
X403471CRTSCP, so a package cannot grant itself a sibling’s key by editing its manifest. - The contract is read from the deployment’s own copy of its package, not the catalogue, so a deployment reloads the way the package it runs says.
- A scalar other than
restart, an empty argv list or an empty service name logsX400470CRTDCL, and the rotation falls back to restarting every binding service: delivered less gently, never withheld. privkey.pemis mode 0600 and root-owned through the bind, so a process running as another uid cannot read it. Run the TLS terminator as root, as postfix and dovecot do, or copy the pair at entrypoint.shc cert deliveryshows each delivered host, the class its resolver requests against the class delivered, and the expiry. The material is on the node that runs the deployment: the listing asks every node and each row names the node that holds it.
In the catalogue, mailcow declares it for postfix and dovecot, which bind the SMTP hostname. The worked example is Your own ingress, whose nginx ends TLS with the delivered certificate.
Variables
Section titled “Variables”Two sources of variable declarations:
- Inline under
vars:inapp.yaml - File:
vars.yamlnext toapp.yaml(same structure)
# Inline in app.yamlvars: adminUser@string!: WordPress admin username adminPassword@secret!: WordPress admin password mysql: hostname@string!: MySQL hostname port@port!: MySQL portEach leaf uses type-annotated syntax:
<name>@<type>[:<role>][!]: <description>.
| Annotation | Meaning |
|---|---|
! (trailing) | Required: install fails if not provided |
@string | String value |
@secret | Stored in Vault; never logged, never in events |
@port | Integer 1-65535 |
@int | Integer |
@bool / @boolean | Boolean |
@email | Validated email address |
@url | URL |
@path | Filesystem path |
@json | A JSON object or list. --var name='<json>' and a deployfile list reach templates as the parsed list or map; install and update refuse anything else with X400153VARJSN. An empty value is unset |
@uuid | UUID in the 8-4-4-4-12 hex form; install refuses anything else with X400158VARUID |
@select(a|b|c) | One of the listed options, offered as a pick by every form; install, update and shc config set modules.<app>.vars.<var> refuse anything else with X400156VARSEL. An empty value is unset |
@json(a|b|c) | A JSON object whose every value is one of the listed options, set one entry per path (names.User-Agent=keep); install, update and shc config set refuse an entry outside them with X400156VARSEL. Its entries are never sealed into the vault |
A select states its options inside the type, before any role:
metaStore@select(sqlite|postgres)! or level@select(debug|info):identifier.
An option may hold a colon or a space; | and ) end one. A select with no
options, an empty option or a repeated one is refused with X400157VARSLD, and
the catalogue sweep refuses a package whose vars.yaml default is not one of
its options.
A var that holds a vault ref is checked by the value the ref resolves to, at install and update. The refusal names the ref, never that value.
Variables nested under a group (mysql:) become addressable as
{{ vars.mysql.hostname }} in compose.yaml, hook jobs, integration
jobs: everywhere SHC renders user templates.
Full type list lives in
modules/app/var_types.go
(VarDecl / the declared-variable parser).
Roles: what a HOST MOVE does to a URL-bearing var
Section titled “Roles: what a HOST MOVE does to a URL-bearing var”The type says how to parse a value. The role says what a move does to it, and those are different questions with three opposite answers:
| Role | A move… | Examples |
|---|---|---|
endpoint | replaces it with the new origin | keycloak.issuer, gitlabUrl, backendUrl, mautic’s site_url |
allowlist | appends the new origin and keeps every existing member | keycloak.extraRedirectUris, allowedParentOrigins, csrfTrustedOrigins, nextcloud’s trusted_domains |
identifier | never touches it | matrix serverName, frappe siteName, SAML entityIDs, every Keycloak clientId |
vars: siteName@string:identifier!: Frappe site name (FQDN or short name) keycloak: issuer@url:endpoint: Keycloak OIDC issuer URL extraRedirectUris@string:allowlist: Additional redirect URIs, comma-separatedThe role is a qualifier rather than a new type because allowlists and
identifiers are routinely @string: they are comma-joined lists or bare
hostnames, not single parseable URLs. A colon is safe inside the key: YAML
splits a mapping entry on ": " (colon space), so issuer@url:endpoint!:
parses as one key with the description as its value.
Getting a role wrong has a measured cost, one per role:
- endpoint mistaken for identifier: mautic’s
site_urlwas left athttp://prod.mautic.mautic, the deployment’s in-cluster service alias, so every tracking pixel and unsubscribe link pointed at a name that resolves nowhere outside the tenant network. - allowlist mistaken for endpoint: a rewrite replaces the membership rather
than extending it. Nextcloud’s persisted
trusted_domainsheld four entries against one in the render; replace semantics would take three live hostnames offline, and Nextcloud answers an untrusted Host with a full-page 400. - identifier mistaken for endpoint: rewriting matrix’s
serverNamechanges every user id on the homeserver; rewriting frappe’ssiteNamedetaches the bench from its own site directory. Neither is undone by a second flip.
Two refusals keep the vocabulary honest, for the same reason @boolean
accepts exactly one spelling: a typo must not degrade to “no role stated”,
because the degrade is invisible until a move, months later.
| Refusal | Fires when |
|---|---|
| unknown-role | the role qualifier is not one of the three |
| url-var-missing-role | a var declared @url states no role |
| identifier-rewrite | a move, rename or import proposes to change an identifier |
The three carry X-codes, deliberately not named here yet: the codes ship
with ce and this page is checked against the error registry of the ce version
this repo currently pins (ghost-error-codes in scripts/arch-fitness.sh).
Naming a code before its guard is shipped “reads to an operator as a guard that
runs”, which is exactly the fault that rule exists to stop. The codes go in
this table in the commit that bumps the ce pin.
Seed-once origins
Section titled “Seed-once origins”There are three places a stale origin can hide, and only the first two are reachable by the verbs an operator reaches for:
- Route: what the edge serves.
--exposeowns it; a re-election fixes it. - Render: what the app is configured with (compose env, a mounted
config, a plug/socket job argument).
--varplus a re-render fixes it. - Seed-once: what the app copied into its own state at first-run setup and never revisited. No re-render reaches it: the render already carries the right value and the app has stopped reading it. It surfaces as a broken login, a dead tracking link, or an SSO callback the IdP rejects.
An app declares its class-3 values so an import or move runs the rewrite as a step instead of an operator discovering it weeks later:
seed_once: - key: site_url # as the APP spells it kind: self_origin # self_origin | foreign_endpoint role: endpoint # the vocabulary above; default endpoint store: "file:/var/www/html/config/local.php" rewrite: "job:mautic-init" # job:<name> | exec:<svc>:<cmd> | none probe_path: /mtc.js # the surface that EMITS the value probe_service: mautic note: >- How it was measured and what breaks when it is stale.kindseparates the app’s own advertised origin (self_origin) from another service’s URL the app captured (foreign_endpoint): a wrong self_origin breaks every link the app emits, a wrong foreign_endpoint breaks the handshake it makes.rewrite: noneis a legitimate, deliberate value: fleet and wordpress have no rewrite path today, and a declared hole beats an undeclared one. An absentrewriteis refused, because an omission cannot be told apart from an author who never considered the question.probe_pathmatters more than it looks. Measured across 32 routed host pairs on a live estate, most app roots emit no absolute URL at all (a JS shell, a relative redirect, or a gate), so a probe of/learns nothing. mautic’s/mtc.jsemitssite_urlverbatim beside the request host, which is what makes the two distinguishable in one fetch.
Find members with shc doctor origin-probe <stack>: it fetches one surface
under several Host values and classifies every origin that comes back as
derived (tracks the request host: a move fixes it for free), rendered-pin
(constant and present in the render: shc update re-points it), seed-once
(constant, one of this deployment’s identities, and absent from the render)
or foreign (a CDN or project link: reported, never a fault).
hooks: post_install: - setup-database - notify-admin pre_backup: - flush-cacheEach value is a list of job names, not inline commands. Jobs
are defined in *.job.yaml files in the same app directory. See
lifecycle-hooks for the full hook model.
crons: daily-cleanup: "0 2 * * *" hourly-sync: "@hourly"Maps job name → cron expression. The named job is scheduled on
the node that owns the deployment. Jobs are defined in *.job.yaml, same
as for hooks. See architecture/modules/schedule.
Timeouts
Section titled “Timeouts”There is no per-app timeouts: block in app.yaml. Operational
timeouts are platform config: the timeouts.<role> config keys
(defaults and role list in core/timeouts/timeouts.go, ce module)
bound compose convergence, container stop, backup shell-outs, and the
other pipeline phases. Per-service stop behavior uses compose’s native
stop_grace_period.
Plugs and sockets
Section titled “Plugs and sockets”Plugs and sockets are declared in standalone *.plug.yaml /
*.socket.yaml files next to app.yaml; you do not hand-write
them in app.yaml. See
plugs-and-sockets.
Connections
Section titled “Connections”connections: declares the app’s connection dependencies: typed,
admin-owned credential backends the app is bound to at install time.
The declaration lives in app.yaml (not a parallel file: the
credential schema per type is baked into SHC), keyed by binding name:
connections: aws: type: aws # Same `required:` grammar as plugs: true / false (default) / # a Jinja condition against the merged vars. The conditional form # expresses the either-or: mandatory UNLESS direct credentials # were supplied via vars. required: "{{ not vars.masterPassword }}"A required connection that is not bound fails the install fast with
X400999CONREQ (before anything is deployed), naming the declarations
and the -c remedy.
A declaration with many: true is bound any number of times under
sub-names: -C mirror.apt=cloudflare -C mirror.dev=hetzner. Each gets
its own row, its own mint and its own vault key. The render namespace nests
them one level (connections.mirror.apt.endpoint), so a template walks
the family with {% for name, t in connections.mirror | dictsort %}.
The one policy: renders per binding with binding (the sub-name) in
its context, and required: on the family is satisfied by any one
member. The bare family name is not a binding.
connections: mirror: type: s3 many: true required: falseshc.yaml or compose.yaml
Section titled “shc.yaml or compose.yaml”An app’s services are written in shc.yaml, SHC’s
own strict schema, which is the recommended form. compose.yaml is the
import path, for an app you already have as a compose file, and the rest
of this section describes it. When both files exist, shc.yaml is the
app and compose.yaml is not read. The template context below is the
same for both files.
compose.yaml is standard Docker Compose, but run through Jinja before
handing to Compose. All variables, integration data, and derived names are
injected as template context.
The install render is strict about undefined references: a
{{ vars.x }} (or integrations.… / connections.…) lookup that
misses and reaches the output with no | default(...) guard fails the
install with X400998TPLUND, naming every miss, instead of silently
rendering an empty string into container config. Guard idioms stay
free: | default(...), branches dropped by {% if %} /
is defined, and condition expressions never trip it.
services: web: image: wordpress:{{ vars.version | default('6.9') }} environment: WORDPRESS_DB_HOST: "{{ integrations.mysql.socket.hostname }}:{{ integrations.mysql.socket.port }}" WORDPRESS_DB_USER: "{{ integrations.mysql.socket.username }}" WORDPRESS_DB_PASSWORD: "{{ integrations.mysql.socket.password }}" volumes: - wp-content:/var/www/html/wp-content expose: - "80"
volumes: wp-content:Available template variables:
| Expression | Source |
|---|---|
{{ vars.<key> }} | Merged variable values |
{{ integrations.<plug>.socket.<key> }} | Provider socket data for each connected plug (see plugs-and-sockets) |
{{ integrations.<plug>.plug.data.<key> }} | What this app sent on that plug |
{{ stack }} | the deployment’s stack name (-s / --stack) |
{{ tenant }} | Tenant name |
{{ environment }} | Environment |
Render logic: the Jinja engine in core/jinja/ (ce module), driven from
the install pipeline (modules/app/install_runner.go,
jinja.RenderCollect).
SHC directives (x-shc-*)
Section titled “SHC directives (x-shc-*)”x-shc* is a reserved namespace. Compose ignores any x--prefixed
key without complaint, so SHC is the only thing that can tell you a
directive was not understood, and it does: the install render refuses
any x-shc* key it cannot honour as written, rather than dropping it.
Do not park unrelated values under this prefix.
x-shc directives lists every directive, the ones an app writes and the ones SHC writes itself, with its fields, its refusals and what the render does with it.
x-shc-primary: the app’s front door
Section titled “x-shc-primary: the app’s front door”--expose takes <hostname>[/<path>][=<service>[:<container port>]] for
an HTTP route, and the selector has always been optional. It could only
be omitted, though, on an app with exactly ONE routable
(service, container-port) candidate,
and almost no real app has one. A valkey sidecar exposing 6379 is a
candidate; a metrics port is a candidate; keycloak alone has five. So
--expose app.example.com was X400937HSTAMB on 44 of the shipped apps,
and every operator had to know a service name and a port number to bind
the obvious hostname.
x-shc-primary lets the PACKAGE answer that, once, where the author
already knows it:
services: windmill-server: x-shc-primary: true # the front door; take its first declared port expose: ["8000"] windmill-lsp: expose: ["3001"] # absent = not primary| Value | Meaning |
|---|---|
true | This service is the primary ingress target; the port is the FIRST it declares |
<port> | Primary and the port, pinned: the scalar doubles as the port selector |
false | Explicitly not primary. Identical in effect to leaving the key out; it records a decision, and it does not make the service unroutable |
| absent | Not primary |
Which port true means. expose: is read first and ports: second:
expose: is the more intentional declaration, and the seven catalogue
services that declare both use ports: for a NON-http listener published
to the host (keycloak’s LDAP, gitea’s SSH, kubo’s libp2p peer port).
Only container/target ports count; a published host port is never a
candidate, and non-TCP entries are skipped rather than stopped at. Ranges
EXPAND before “first” is taken, so expose: ["8000-8010"] means 8000.
When to pin the number instead of writing true. Whenever the
service declares more than one port. true follows list order, and list
order is one editor away from moving your front door: mail’s mailpit
lists SMTP 1025 before the 8025 web UI, and pmm-server lists its
self-signed 8443 before the 8080 Traefik should target. Both pin.
Two services may not both declare it. That is a hard refusal
(X400938HSTPRM) naming every declaring service, not a pick. services:
is a YAML mapping and the parse behind resolution is a Go map, so
“take one” would be the map seed choosing your route: the same compose
file could route differently on two consecutive renders, silently.
Unmarked, the default service is the first one the file lists. A bare
--expose <hostname> or --expose <port> on an app with no marker goes to
the first service in the file and its first declared port. SHC reads that
order from the file itself, never from a map, so it does not move between
renders.
A bare hostname follows the front door. A hostname exposed with no
target means this app’s front door, and SHC reads the front door again
from the definition at every install and every update. Move
x-shc-primary to another service, run a plain shc update, and the
hostname goes to that service. A hostname exposed with a target
(--expose app.example.com=web:8080) is the operator’s pin, and no
update moves it. shc ingress shows which each route is under TARGET,
front door or pinned.
| Code | Cause |
|---|---|
X400938HSTPRM | Two or more services declare x-shc-primary |
X400939HSTPRP | The declaring service pins a port it does not declare, or declares no TCP port at all |
What it does not do. It only ever fires where X400937HSTAMB fired
before. It never overrides an explicit --expose selector, never changes a
resolution that already had exactly one answer, and never turns a
multi-host app into a single-host one: matrix, netbird, supabase and
blockscout still need an --expose per public surface. It settles ONE
question: which service a bare hostname lands on.
Groups: the {% members %} block
Section titled “Groups: the {% members %} block”A service scales by interchangeable replicas behind one virtual IP. A group scales by stamping numbered members, each an ordinary service with its own name, virtual IP and volumes. A database with one writer and two standbys is a group of three members, not a service with three replicas.
A group is written one way only: a {% members %} block inside
services:. There is no x-shc-groups and no x-shc-member; both are
refused as unknown directives (X400330XSHUNK).
services:{% members "db", default=3, max=9 as m %} pg-{{ m.n }}: image: postgres:17 volumes: ["data-{{ m.n }}:/var/lib/postgresql/data"] environment: PGLOCATION: "{{ m.location }}" exporter-{{ m.n }}: image: prometheuscommunity/postgres-exporter network_mode: service:pg-{{ m.n }}{% endmembers %} web: image: example/webSHC renders the body once per member. Everything a pass writes belongs
to that member, so pg-2 and exporter-2 are member 2: they are placed,
started and retired together. SHC declares data-1, data-2, data-3
itself; write no top-level volumes: loop.
| Part | Meaning |
|---|---|
"db" | The group’s name, a string literal: lowercase letters, digits, -, _. It is what --scale names |
default=<n> | Members a first install starts with when no --scale names the group. Default 1 |
max=<n> | The ceiling. No max means none |
separate="required" | No node ever runs two members. The default, "preferred", lets them share a node when the cluster has too few; see Groups |
as m | The member inside the body |
m.n | The member’s number. It never changes while the member exists |
m.location | The location SHC assigned the member, always a leaf, or "" when the scale names none |
Scale the group, not its services.
shc install pgha --scale db=5shc update -s pgha --scale db=2@home,1@fardb=5 runs five members anywhere. db=2@home,1@far holds two members
to home and one to far. db=3@incus names a level: SHC gives each
member a leaf under it, spread over its children, and keeps it there,
because a member holds data and never floats inside a level. A group
takes no <n>/<kind> shape.
Numbers are stable. They are a stored list per deployment. Scaling 3 to 5 adds members 4 and 5 and leaves 1 to 3 exactly as they were: same rendered services, same spec digests, no restart. Scaling 5 to 3 retires 5 and 4 and keeps their durable volumes, and scaling back up brings them back by number.
Only --scale <group>=... changes a group. An update that names no
value for the group keeps every member it has, in its location. Changing
default in a new app version does not grow or shrink a deployment
that already runs the group. One exception: a block inside an {% if %}
that renders false has no members in that render and keeps no list, so
when the branch turns true again the group starts over from its
--scale value, else its default.
A pass is its own YAML. What the body writes for one member has to
read on its own as a mapping of whole services, and the rest of the file
has to read on its own without the block. Write each member’s services
whole inside the block; nothing in it may add to a service outside it or
to another member’s. An anchor written outside the block before it, such
as x-common: &common, may be merged inside (<<: *common). An anchor
written inside the block is for that member’s services only.
Members prefer separate nodes. A member is placed on a node no other member of its group holds when there is one; with too few nodes members share a node rather than wait. A member that had to share stays where it is when a node joins later: nothing moves members apart yet. A member with a location is placed only there.
One copy per member, for now. A member’s service that states
deploy.replicas or scale other than 1 is refused
(X400099GRPREP). Replicas inside a member are not supported yet.
The block may sit inside an {% if %}. It may not sit inside another
members block, a {% for %}, a {% macro %}, a {% call %} or a
{% filter %}, and a group is written once per template.
shc status -s <deployment> lists each member of its groups: group,
number, location, readiness, the service selector named for the group it backs and services. shc logs and shc restart take a
member’s service name (pg-2).
| Code | Cause |
|---|---|
X400512JNJMBR | The tag is malformed: a name that is not a literal, an unknown argument, default over max, nesting, or the group written twice |
X400093GRPMAX | --scale asks for more members than max |
X400094GRPSHP | --scale <group>=<n>/<kind> |
X400095GRPSVC | --scale names a service the group stamps (pg-2, or pg when no service is called pg); scale the group |
X400096GRPDUP | A group shares a name with a service, or a member pod name (db-2) is a service outside the group |
X400097GRPEMT | A pass is not a YAML mapping of whole services on its own, the file does not read without the block, an anchor written inside the block is used outside it, or a pass wrote no service |
X400098GRPLNK | A member’s service shares a namespace or volumes_from with a service outside its member |
X400099GRPREP | A member’s service states a replica count other than 1 |
vars.yaml
Section titled “vars.yaml”vars.yaml is the file form of the vars: block. Same
annotated syntax; use whichever is more convenient. If both are
present, inline vars: in app.yaml takes precedence on conflict.
adminEmail@email!: Administrator emailapiKey@secret: API key for external serviceport@port: Application portbackup.yaml
Section titled “backup.yaml”Lists the mounts that hold data worth backing up, and says how a
deployment’s state is captured and restored. A mount is named the way the
deployment knows it: a compose volume (uploads), a group member’s volume
(pgdata-1), or the host-... name a host-path data volume gets at install.
Anything the file doesn’t list is never captured
(the ruling).
A path the image declares as a VOLUME that no mount of the service covers
is a mount too. A new install reads the image’s config from its registry and
mounts auto-<service>-<path> there, so /var/lib/mysql of service db is
auto-db-var-lib-mysql, and the lists name it like any other. A service
that ran before shc gave these paths a volume keeps them in volumes of the
node’s own, which no backup or move reaches, and an update doesn’t change
that. Mount the path in the compose file when its data matters.
include: - uploads - repo-dumpskeep: # Repositories. The pre_backup hook packs them into repo-dumps. - repos# valkey-data and cache aren't listed, so they're scratch.SHC reads these lists from 0.1.22, so a package that ships backup.yaml
declares it in app.yaml, and shc app validate refuses one that doesn’t:
requires: shc: "0.1.22"A package without that declaration is read with the meaning v0.1.21 gave its file, so a deployment made before 0.1.22 captures what it did until it’s updated onto a package that declares the lists (the ruling).
include is what a backup copies. Every capture path reads the same list:
the orchestrated pass, paused-tar, a failed converter’s fallback, the
daemonless CLI tar and the restic path.
keep is persistent data another path protects, such as a tree a dump hook
packs into an included mount, or one the image owns. A relocation carries it
and shc uninstall --data names it (X200975DURDEL), but a backup never
reads it and it never counts toward the per-node refusal (X409899BKPRPL).
keep is for data protected by another path (a dump hook, an image-owned tree), not a way to skip backups.
Every keep entry carries a comment, on its line or the line above, saying
what protects it. tests/unit/catalog fails a catalogue package whose entry
has none, and it reads keep only in the block style shown here.
Every mount in neither list is scratch. It’s never captured, it’s recreated
empty wherever its replica lands, and doctor’s storage.durable-on-node-disk
never reports it. A mount in either list is durable: its reach is a hard
placement rule, and block storage holds it where the estate has a provider
(block storage by default).
A host-path data volume neither list names is kept by default (SHC-2583). A
move carries it and a delete names it, but a backup copies it only when
include names it.
An empty file, comments only or include: [], says the app has nothing to
back up. A backup of it carries the deployment config and the vault bundle,
and succeeds. The system traefik ships one:
# Nothing to back up. logs is the access log, and static is rendered from the# environment at every start, so both are scratch.The file is required once the app has a named volume. Install refuses an app
that declares the lists, has one and ships no backup.yaml (X400601BKPUND), and
tests/unit/catalog refuses the same catalogue package. An app with no named
volume needs no file, though it may ship one for converters or jobs.
Install refuses a file it can’t read, too. exclude.mounts, the old
include.mounts mapping and the durability: block (critical,
ephemeral) are gone and refused by name (X400345DURSHP), as is a list
that isn’t a list of mount names. A mount under both include and keep
refuses X400339DURCON.
# backup.yaml, abridged from the postgres appconverters: pgdata: # the converter's name image: "{{ vars.images.postgres }}" expect: [/data/db/PG_VERSION] user: "0" entrypoint: ["sh", "-c"] command: | # start postgres on the snapshot, then dump it su postgres -c "pg_dumpall -U {{ vars.postgres.username }} --clean --if-exists" > /output/pgall.sql volumes: - pgdata:/data/db # the snapshotted named volume - "@output:/output" # what the converter writes is archived
jobs: restore: - name: restore_from_dump image: "{{ vars.images.postgres }}" command: | su postgres -c "psql -f /backup/pgall.sql" volumes: - "@pgdata:/backup" # the pgdata converter's output - pgdata:/data/db
include: - pgdata # the converter's mount is always listedconverters is a map keyed by converter name, never a list. Each one runs
its image in its own container against the snapshot, with no network
unless it names one, and its fields are rendered with Jinja2 against the
deployment’s render context before the run. A volume source of
@output is where the converter writes; a restore job reads that output
back as @<converter>. There is no top-level volumes: key: include is
the one list of what a backup copies. See
flow: backup/restore for the
full capture/apply order.
Optional expect lists absolute container paths under the converter’s input
volume targets; it cannot name @output, relative paths or .. segments.
Any one match permits the converter to start: an entry ending in / requires
a directory that exists, and any other entry requires a regular file that
exists, even if empty.
The engine checks the resolved frozen copy as the container engine sees it
before starting the converter, and refuses malformed declarations at install,
update and capture; access errors fail loudly.
If none matches, the result is the same as exit 65: raw frozen bytes stand in
with X200683CNVEMP when they exist, or the backup fails with X409691CNVNDT
when the frozen copy is empty; omitting expect preserves script checks.
A converter’s mount must be under include, and install refuses one that
isn’t (X400603BKPCNV). The converter stands in for its mount only on the
orchestrated pass. A backup that doesn’t run it (paused-tar, a failed
converter, the daemonless tar) copies the mount itself, and only a listed
mount is ever copied. A hook that dumps into a volume lists the dump target
under include and the live source under keep: gitlab’s pre_backup packs
its repositories into gitlab-backups, which it includes, and it keeps
gitlab-data.
A converter over a replicated group’s data says which member it reads
with from:, a selector in the language --where reads, for example
from: group=postgres or from: group=postgres & role!=primary. It is
written for member 1’s volume, reads one member chosen when the backup
runs, and the group’s volumes it covers are captured once, through it.
An app’s from: uses only group, member, service and the labels
the app writes itself, such as postgres’s live label role. See
Database apps, HA by scale.
Volumes belonging to a bundled cache or broker go in neither list, so they’re scratch. House rule, stated in full (with the one documented exception) in bundled services:
include: - uploads# valkey-data isn't listed: the cache rebuilds from the database on boot.How an app definition is loaded
Section titled “How an app definition is loaded”There is no single LoadAppDefinition loader: each pipeline stage
reads the app-dir files it owns (all in the ce modules/app app):
app.yamlis read via safe-YAML by the consumer that needs it: the validator (validate.go), variable typing (var_types.go,vars_validate.go), ingress injection (inject_traefik.go).vars.yaml(if present) merges into the declared-variable set (var_types.go).*.socket.yaml/*.plug.yamlare globbed by the integration machinery (integration_service.go,integration_render.go);*.vapp.yaml/*.vsocket.yaml/*.vplug.yaml/*.vvars.yamlby the vapp runner (vapp_runner.go).- Declared-variable parsing turns every annotated leaf into a
VarDeclrecord available for schema validation (var_types.go). - A missing or invalid
app.yamlfails at the consumer that first needs it: install rejects,shc validatereports.
Validation rules beyond struct tags
Section titled “Validation rules beyond struct tags”Some rules aren’t expressible as field types. Examples:
- A
--exposetarget must name a service incompose.yaml. Checked when the deployment is rendered. - Required plugs must be connectable: enforced by install’s plug-matching step.
- Required variables must be provided: install rejects with
X400903INVAPPif a!var is missing. - Job names in hooks / crons / integration phases must resolve to
a
*.job.yaml: logged as a warning at runtime (X500908HKFAIL) rather than rejected at install time, so a typo inpost_uninstalldoesn’t block an install.
How files are processed
Section titled “How files are processed”Each file in an app directory goes through a different transform pipeline before SHC hands it to Docker:
| File pattern | Processing |
|---|---|
app.yaml | metadata only, no templating |
vars.yaml | vault references resolved |
compose.yaml | Jinja2 → vault → enrichment |
*.socket.yaml / *.plug.yaml | 3-part split (discovery, data, jobs) |
*.vapp.yaml / *.vplug.yaml / *.vsocket.yaml | Jinja2 → vault |
*.vvars.yaml | vault references resolved |
*.export.yaml | Jinja2 → vault |
backup.yaml | converters, jobs, and the include and keep lists; converter fields rendered with Jinja2 |
Templating order
Section titled “Templating order”For files that go through Jinja2, the order is strict:
1. YAML load → parse, preserve !ref+vault:// as SecretRef objects2. Jinja2 templating → expand {{ vars }}, {{ tenant }}, {{ integrations }}, …3. Vault resolution → resolve !ref+vault:// and !ref+env://Jinja2 runs before vault resolution, so vault paths can be dynamic:
secret_name: admin_password
# compose.yaml: Jinja2 firstpassword: !ref+vault://{{ vars.secret_name }}# After Jinja2: !ref+vault://admin_password# After vault: the-actual-secretTemplate context
Section titled “Template context”Available variables in any Jinja2-processed file:
| Variable | Description |
|---|---|
tenant | tenant name |
stack | the deployment’s stack name, the app-identity axis |
environment | environment name |
vars | merged + resolved variables from vars.yaml |
integrations | integration socket values (compose context) |
integration | single integration context (plug/socket context) |
paths | SHC path locations |
gated | true when every published way into the deployment passes the SSO gate: its access resolves to gated, it serves at least one hostname route, and it publishes no raw port. A package uses it to turn its own login off behind the gate (Uptime Kuma’s uptime-kuma-disable-auth job) |
shc.runtime | the runtime of the node the replica runs on: docker, podman, incus or apple |
shc.ingress_sources | the addresses the system Traefik reaches the deployment from, one per node; see trusting the ingress |
replica.ordinal | the replica’s number, from 0 for each service (compose template only, printed) |
replica.node | the id of the node the replica runs on (compose template only, printed) |
replica.location | the location that node states, "" when it states none (compose template only, printed) |
The deployment id is {tenant}_{environment}_{stack}. It is not a template key. Identity in the render is stack, environment and tenant (modules/app/render_context.go buildRenderContext).
paths.* resolves to:
| Path | Description |
|---|---|
paths.base_dir | project root (dev mode only) |
paths.state_home | state directory (database) |
paths.cache_home | cache directory (git repos) |
paths.runtime_dir | runtime directory (PID, socket) |
paths.logs_dir | logs directory |
paths.sqlite | SQLite DB path (single-node) |
Trusting the ingress
Section titled “Trusting the ingress”Each node keeps one fabric address for the system Traefik, and no other
container is given it. shc.ingress_sources lists them, one per node, so an
app that takes the client’s address from X-Forwarded-For or X-Real-IP
trusts those headers from Traefik and from nothing else:
services: web: image: nginx:1 environment: REAL_IP_FROM: "{{ shc.ingress_sources | join(' ') }}"The list changes only when a node joins or leaves the cluster, and the next
update renders it again. An app that reads its trusted proxies at run time
can ask ingress-sources.shc.internal instead (see
private DNS), which follows a node
joining with no update. When the addresses cannot be read the list is empty,
never a wildcard, so the app trusts no forwarded header until the next
render.
A service can also learn the current addresses of a sibling service of its
own deployment, such as a daemon it must trust as a proxy, from
_replicas.<service>: one A record per ready replica, never the VIP the
service’s own name answers.
Branching on the runtime
Section titled “Branching on the runtime”shc.runtime is the runtime of the node each replica runs on. Branch on it
only where a runtime has no equivalent of a key: SHC translates what it can
for each runtime (see compose keys), so most apps never
need it. Apple runs every container as its own VM and refuses sysctls, so
this Redis sets them everywhere else:
services: redis: image: redis:7 {% if shc.runtime != "apple" %} sysctls: net.core.somaxconn: 1024 {% endif %}A deployment runs on one runtime, so its replicas share one value, except a
deploy.mode: global service’s, which can land on a node of another
runtime. When the template reads shc.runtime, the install renders it once
for each runtime and keeps every service key the renders write differently,
and a replica is finished with its node runtime’s value, on install and on
every move. A branch may change only keys SHC leaves as written: one that
changes a key SHC rewrites (volumes, ports, networks, environment,
labels, deploy and the like), or adds or drops a service, is refused at
install with X400797RTMKEY.
Printing the replica
Section titled “Printing the replica”A service can print which replica it is: its number, the node it runs on and that node’s location. A peer list, a node-local cache key or a zone label are the usual reasons:
services: web: image: ghcr.io/example/web:2 environment: PEER_ID: "web-{{ replica.ordinal }}" NODE_ID: "{{ replica.node }}" ZONE: "{{ replica.location }}"The template is rendered once for the deployment, and each replica’s
document is finished with its own values when it is placed. A replica that
moves gets its new node’s values, and so does every replica on a node whose
location changes (shc node update --location). No other replica restarts.
replica.* can only be printed, as {{ replica.ordinal }},
{{ replica.node }} or {{ replica.location }}, into a service’s
environment, command, entrypoint, healthcheck, hostname,
working_dir or labels. A branch, a filter or arithmetic on it, or a
replica in a hook, job, plug, socket or backup file, is refused with
X400714RPLUSE. A replica value in a key SHC reads before a replica is
placed (image, ports, volumes, networks, deploy, a service name)
is refused with X400715RPLKEY. Numbered members that need arithmetic, such
as a port per member, are a {% members %} group (see Groups above) and
m.n.
Vault references
Section titled “Vault references”Secrets use !ref+vault:// (not Jinja2 {{ secrets.* }}):
# Simplepassword: !ref+vault://db_password
# Scoped lookuppassword: !ref+vault://db_password?scope=tenant
# Auto-generate if missingpassword: !ref+vault://db_password?autogenpassword: !ref+vault://db_password?autogen&length=64&charset=alnum
# Environment variable (with default)api_key: !ref+env://API_KEY?default=testParameters:
| Parameter | Description |
|---|---|
scope | tenant, stack, environment, deployment |
autogen | auto-generate and store if missing |
length | generation length (default 32) |
charset | named set or character range (see below); default hex |
default | default value if env var not found (!ref+env:// only) |
Named character sets:
| Charset | Characters |
|---|---|
alnum | A-Za-z0-9 |
alpha | A-Za-z |
num | 0-9 |
hex | 0-9a-f (default) |
base64 | A-Za-z0-9+/ |
Custom regex-style character ranges are also accepted (a leading or
trailing - is a literal):
password: !ref+vault://key?autogen&charset=a-z0-9token: !ref+vault://key?autogen&charset=0-9a-fkey: !ref+vault://key?autogen&charset=A-Za-z0-9_-When the charset spans more than one character class (lowercase, uppercase, digits, specials), the generated value always contains at least one character from each covered class, so class-counting password policies are satisfied on every mint.
type mints a private key instead of a string, as a PKCS#8 PEM
(-----BEGIN PRIVATE KEY-----). Use it for a key the app’s stored data
depends on, a signing or token key, so the key has the vault’s lifetime
rather than an app volume’s:
| Type | Key |
|---|---|
rsa | RSA, 2048-bit |
ec-p256 | ECDSA on NIST P-256 |
ed25519 | Ed25519 |
rsaKey: !ref+vault://rsa_key?autogen&type=rsatype needs autogen and takes no length or charset. A ref that breaks
either rule, or names another type, fails the install, whether or not the
vault already holds the key. The app derives the public
half itself. A PEM spans lines, so hand it to an app as a file where the app
reads one.
type also mints an x509 certificate, as one CERTIFICATE PEM. x509-ca is a
self-signed CA. x509-cert is a leaf signed by a CA the vault already holds.
Each certificate’s private key is a vault key of its own, named by key,
which the mint creates the way ?autogen&type=<keytype> would when the vault
does not hold it yet. A CA key so never has to sit in a compose or a container.
| Parameter | On | Meaning |
|---|---|---|
key | both, required | vault key of the certificate’s private key |
keytype | both | rsa, ec-p256 (default) or ed25519 |
cn | both | subject common name; default the ref’s key, or a leaf’s first DNS name |
days | both | validity; default 3650 for a CA, 825 for a leaf, never past its CA |
ca | x509-cert, required | vault key of the CA certificate |
cakey | x509-cert, required | vault key of the CA’s private key |
usage | x509-cert, required | server or client |
san | x509-cert | comma list of DNS names and IP addresses; a server needs one |
secrets: ca: x-shc-vault: ref+vault://tls/ca?autogen&type=x509-ca&key=tls/ca-key server: x-shc-vault: ref+vault://tls/server?autogen&type=x509-cert&key=tls/server-key&ca=tls/ca&cakey=tls/ca-key&usage=server&san=db,10.0.0.5 server-key: x-shc-vault: ref+vault://tls/server-key?autogen&type=ec-p256One deployment’s refs mint in order, every other ref first, then CAs, then
leaves, so a CA and its leaves can be declared together in any order. A leaf
whose CA certificate or CA key the vault does not hold is refused
(X404986AGNCAM) and mints nothing. A cakey that is not the ca’s key is
refused (X400978AGNCRT), and ca and cakey can’t name the estate’s own CA.
A certificate is minted once and kept, like every autogen. To rotate one,
remove it with shc vault rm, and its key too for a new key pair, and update
the deployment. Leaves signed by a rotated CA go the same way. Nothing
watches expiry.
Minimal complete example
Section titled “Minimal complete example”name: helloversion: "1.0.0"description: Minimal hello-world examplecategory: example
license: id: unverified
vars: message@string!: Message to display port@port: Port to listen onservices: web: image: nginx:alpine environment: MESSAGE: "{{ vars.message }}" ports: - "{{ vars.port | default(8080) }}:80"shc install hello \ --stack hello-world \ --var message="Hello, SHC!" \ --var port=9000Related
Section titled “Related”- Creating apps: tutorial walkthrough
- Lifecycle hooks: hook/job contract
- Plugs & sockets: integration system
- vApps: virtual apps and vSockets
- Source resolution: where SHC finds apps on disk
- Architecture: app module: the consumer side
- Flow: install: where
app.yamlis read and validated