Source resolution
Layer 11. Lifecycle. Lifecycle
SHC installs apps from three source types: repository catalog, local filesystem paths, and git URLs. Archive tokens (.tar, .tar.gz, .tgz, .zip, .bak, .bkg, .exp) are detected as a fourth kind but currently refused at install time; see Archive sources. This document describes the architecture for detecting, fetching, caching, and tracking app sources.
Overview
Section titled “Overview”The source resolution system enables flexible app installation workflows:
- Repository apps - Curated apps from synced catalogs (existing behavior)
- Local path apps - Install directly from a local directory
- Git URL apps - Clone and install from any git repository
All sources result in the same deployment structure once installed.
Repositories are remote-only. A repo (shc repo add) is a git or
http(s) URL. File-path repos (file:// crawl-in-place) were removed:
their catalog rows replicated cluster-wide while the source directory
stayed node-local, so installs failed on every other node
(X400927REPFIL is shc repo add’s refusal).
Local directories are still first-class install sources (shc install ./path/to/app); they are just not registrable as repos. A local bare
git repository can be registered with the git+file:// scheme: it
goes through the real clone/fetch path like any remote.
Two repo behaviors keep the catalog usable cluster-wide:
- Shipped catalog. The daemon crawls the shipped-app roots
(
/usr/lib/shc/apps,/usr/share/shc/apps) into an implicitshippedrepo at boot and on everyrepo syncsweep, soshc searchlists shipped apps with no bootstrap step. The name is reserved (X400928REPRSV) and the repo cannot be removed (X403902REPBLT). - On-demand source materialization. The app catalog is
cluster-replicated but repo clones are node-local. When an install
resolves a catalog app whose source directory is absent on the
installing node, the daemon clones the owning git repo synchronously
(bounded by the git-clone timeout) and installs from the local
clone, so search-visible apps install on any node. A failed clone
surfaces
X500902REPMATwith ashc repo sync <name>hint.
Source Detection
Section titled “Source Detection”When running shc install <source>, the system detects the source type from the provided argument.
Detection Rules
Section titled “Detection Rules”| Pattern | Source Type | Example |
|---|---|---|
./, ~/, or / prefix | Local path | ./apps/myapp, ~/apps/myapp, /home/user/apps/myapp |
.tar, .tar.gz, .tgz, .zip extension | Local archive | myapp.tar.gz, /tmp/myapp.zip |
git+https:// | Git HTTPS URL | git+https://github.com/user/myapp.git |
git@* | Git SSH URL | [email protected]:user/myapp.git |
git:// | Git protocol | git://github.com/user/myapp.git |
https://*.git | Git HTTPS URL | https://github.com/user/myapp.git |
https://github.com/* | Git HTTPS (GitHub) | https://github.com/user/myapp |
https://gitlab.com/* | Git HTTPS (GitLab) | https://gitlab.com/user/myapp |
https://bitbucket.org/* | Git HTTPS (Bitbucket) | https://bitbucket.org/user/myapp |
https://*.tar.gz, https://*.zip | Remote archive | https://releases.example.com/myapp.tar.gz |
Other https:// | Attempt git clone | https://git.example.com/myapp |
<parent>/<vapp> (exactly one slash, no other pattern match) | vApp install: a vdeployment into the parent’s deployment | postgres/database |
| Plain name | Repository app | nextcloud, wordpress |
Repo Qualification and Collisions
Section titled “Repo Qualification and Collisions”A slash in a catalog token means one thing: <parent>/<vapp>. A
bare token with exactly one slash that matches no other detection
pattern (no path prefix, scheme, git@ form, or archive/backup suffix)
is vApp grammar: it installs a vdeployment into the parent’s
deployment, which is that vdeployment’s host. An explicit
--into host_stack --instance instance is likewise always vApp grammar.
Naming a repo is the -r/--repo flag, never a token spelling:
# two repos both carry "plane"; name the one you meanshc repo add community https://apps.example.com/communityshc repo add internal https://git.example.org/internal-apps.gitshc repo syncshc install plane -r community-r restricts resolution to that repo’s catalog rows only; a known
repo missing the app fails with X404012APPDIR naming the repo
searched. (The positional <repo>/<app> spelling is retired: one
slash grammar, one meaning; the flag is the explicit form and it
outranks every resolution heuristic.)
Unqualified collisions resolve by declared policy, loudly when tied.
A plain name (shc install plane) resolves through the static roots
first, then the catalog, where visibility comes before precedence:
candidates come only from the requesting tenant/environment scope plus
the system scope; repo priority ranks the survivors; scope
specificity breaks a priority tie. Only a same-scope, same-priority tie
refuses: X409039APPAMB lists every candidate repo and says the tie
is operator-breakable: set a repo priority, or qualify with -r.
Names carried by a single repo keep resolving unchanged.
App Name Resolution
Section titled “App Name Resolution”The app name is determined in priority order:
namefield fromapp.yaml(if present)- For git URLs: last path segment, excluding
.gitsuffixhttps://github.com/user/myapp.git->myapp[email protected]:user/my-app.git->my-app
- For local paths: directory name
./apps/myapp->myapp/home/user/custom-app->custom-app
Source Flow Diagram
Section titled “Source Flow Diagram”flowchart TB install["shc install <source>"] --> resolver["DetectInstallSourceKind<br>(detect type)"] resolver --> repo["repo"] resolver --> path["path"] resolver --> git["git"] repo --> catalog["Catalog Lookup"] path --> validate["Validate Path<br>(ResolvePathSource)"] git --> clone["Clone to cache<br>(materializeGitSource)"] catalog --> extractor["Subpath selection<br>(handle --path)"] validate --> extractor clone --> extractor extractor --> copy["Copy to deploy directory"] copy --> tracker["SourceData<br>(persist metadata)"] tracker --> deploy["Normal deploy flow continues"]Components
Section titled “Components”| Component | Source (ce module) | Responsibility |
|---|---|---|
DetectInstallSourceKind / IsExplicitInstallSource | modules/app/source_resolver.go | Detect source type (catalog / path / git / archive), parse URL/path |
materializeGitSource / ParseGitURLFragment / InjectGitToken | modules/app/source_resolver.go | Clone, fetch, checkout, token injection |
ResolvePathSource | modules/app/source_resolver.go | Validate and resolve a local path source |
SplitRepoQualifier / ResolveAppDirVersioned | modules/app/source_qualifier.go | The -r/--repo qualifier and versioned catalog dirs |
SourceData | modules/app/types.go | Store/retrieve source metadata for upgrades (Cached* fields set by the daemon) |
Git Source Handling
Section titled “Git Source Handling”Clone Behavior
Section titled “Clone Behavior”- Shallow clone (
--depth 1) by default for efficiency - Full clone only when needed (e.g., specific commit reference)
- Respects system git credentials (SSH keys, credential helpers)
- Supports
--tokeninjection for private HTTPS repositories
Token Injection
Section titled “Token Injection”For private HTTPS repositories, the --token flag injects authentication:
Original: https://github.com/user/private-appInjected: https://{token}@github.com/user/private-appTokens are:
- Used only on the wire: the clone and every fetch name the credentialed URL
- Never recorded: not in the database, the sidecar, or the cached clone. The clone’s origin is the remote without its userinfo (FZ-7075), and a boot pass takes the credential out of clones cached before that release
- Replaced if the URL already contains credentials
A token is not replayed. A later shc update of the deployment reads the
remote with its own --token, or with a credential the node holds: shcd runs
git as root, so a read-only deploy token in root’s ~/.netrc (mode 0600)
answers every update and every re-render on any node. Without either, the
update of a private source fails with X500905REPGIT and the deployment keeps
running what it ran.
Subdirectory Support
Section titled “Subdirectory Support”The --path flag extracts apps from subdirectories within a repository:
shc install https://github.com/user/monorepo --path apps/myapp --stack myappThe entire repository is cloned to cache, then only the specified subdirectory is used for the app definition.
Git URL Fragment Syntax
Section titled “Git URL Fragment Syntax”Git URLs support a fragment syntax for specifying ref and subpath in a single argument:
git+https://github.com/user/repo.git#<ref>:<path>| Component | Required | Description |
|---|---|---|
ref | No | Branch, tag, or commit: main, v1.0.0, abc123def |
path | No | Subdirectory within repo: apps/myapp |
Examples:
# Clone and checkout tag v1.0shc install git+https://github.com/user/app.git#v1.0
# Clone and use subdirectoryshc install git+https://github.com/user/monorepo.git#:apps/myapp
# Both ref and pathshc install git+https://github.com/user/monorepo.git#v2.0:apps/myapp
# SSH with fragmentParsing logic:
- Find
#separator in URL - Split fragment into
ref:pathat first:after# - If no
:, treat entire fragment asref - If starts with
:, treat entire fragment aspath
Archive sources
Section titled “Archive sources”Archive tokens are detected but not installable. DetectInstallSourceKind
classifies a source ending in .tar, .tar.gz, .tgz, .zip, .bak,
.bkg, or .exp as archive, and the resolver then refuses it with
X400002VALINP (“archive install sources are not implemented yet; use a
directory path or git URL”). The detection exists so slash-carrying archive
names never misroute into the vapp dispatch path.
shc install --auth is a casualty of the same gap: the flag is registered
and reaches the daemon as the request’s auth field, but the ONLY code that
reads it is this archive branch, which refuses either way: with --auth set
you get “authenticated remote archive install is not implemented yet” instead
of the plain message. There is no source kind it can authenticate. Private
git sources use --token, which is wired.
Packaged app archives still reach the platform two other ways:
- App repos: http(s) repos serve
.apparchives through the repo catalog (modules/repo/httpindex.go); installs go through the normal catalog path. - Backup archives:
.bak/.bkg/.expfiles are backup/export artifacts consumed byshc backup import, not install sources.
Caching Behavior
Section titled “Caching Behavior”Git Repository Cache
Section titled “Git Repository Cache”Git repositories are cloned to a cache directory for:
- Future upgrades (re-pull without re-clone)
- Subdirectory access
- Offline reference
Cache Location
Section titled “Cache Location”{cache_home}/app-sources/└── {key}/ ├── .git/ ├── app.yaml ├── compose.yaml └── ...The key is a truncated SHA-256 of remote + "\n" + ref, where remote
is the clone URL without its userinfo (normalizeGitCacheKey,
modules/app/source_resolver.go), so each (remote, ref) pair caches
independently and an update without --token finds the clone an install with
it made. {cache_home} resolves through
core/paths/paths.go per the daemon’s path mode (dev / root / non-root).
Cache Operations
Section titled “Cache Operations”A cached clone (.git present) is reused and fetched on every render: git fetch <remote> <ref> --depth 1 + checkout FETCH_HEAD, where <remote> is
origin, or the credentialed URL when a token or URL userinfo is in play. A
source with no ref fetches the remote’s HEAD the same way (FZ-7075; before
it, a refless clone was never fetched again and every bare update read the
commit it was installed at). A missing or broken cache directory is re-cloned
from scratch. A render holds the clone’s lock and stamps its mtime as its last
use. The node’s self-clean sweep removes a clone no deployment’s kept spec
(current or rollback) names once that stamp is over an hour old, and shc doctor’s node.self-clean reports the cache’s bytes and how much of it no
deployment names (FZ-9873).
Local Path Handling
Section titled “Local Path Handling”Local path sources are not cached. On each install or upgrade, files are read directly from the source path. This allows rapid iteration during development.
Source Tracking
Section titled “Source Tracking”Source metadata is persisted on the deployment row itself so upgrades can replay the original install source without a separate side table.
Database schema
Section titled “Database schema”The relevant columns on DeploymentModel (modules/app/models.go):
type DeploymentModel struct { // ... other deployment columns ...
RepoID *int64 `db:"repo_id"` // FK to repo.id, nullable RepoURL *string `db:"repo_url"` // String(500), nullable}The richer SourceInfo shape (SourceType, URL, Ref, Path, …)
lives in modules/app/types.go and is reconstructed from the
deployment row + repo metadata when needed; it is not a separate
database table.
Tracked Metadata
Section titled “Tracked Metadata”Git sources:
source: type: git url: https://github.com/user/myapp ref: v1.0.0 path: apps/myapp # subdirectory, if any cached_at: 2026-01-15T10:00:00ZPath sources:
source: type: path path: /home/user/apps/myappRepository sources:
source: type: repo url: shc-apps # repository name path: nextcloud # app nameUpgrade Behavior
Section titled “Upgrade Behavior”Default Upgrade (Re-pull)
Section titled “Default Upgrade (Re-pull)”Running shc update mystack without --from uses the stored source:
- Git source:
git fetch+git checkout {ref}in cache, then copy to deployment - Path source: Re-read files from the stored path
- Repo source: Check repository for newer version (existing behavior)
Source Change on Upgrade
Section titled “Source Change on Upgrade”--from renders another source for ONE update. It does not rewrite the
deployment’s recorded source, so the next bare update renders the recorded
one again (measured on v0.1.26: a path deployment updated --from a git
remote went back to the path on its next bare update).
# Render a git remote for this update onlyshc update mystack --from https://github.com/user/myapp --ref v2.0.0
# Move a git deployment to another ref, which it recordsshc update mystack --ref v2.0.0To move the recorded source itself, from a path onto a git remote at a ref, repoint it. It renders nothing; the next update renders the remote:
shc repoint mystack https://github.com/user/myapp --ref v2.0.0 --path deploy/myapp --diffshc repoint mystack https://github.com/user/myapp --ref v2.0.0 --path deploy/myappshc update mystackThe cluster keeps the repoint until the next update renders the new source,
whichever node runs it. shc update --check and --plan read that pending
source from any node.
shc repoint mystack --back moves it back onto the path, before that update
or after it: the update keeps the record of the move while it renders the
remote the repoint named.
A deployfile names a git source as git:, and shc deploy does the repoint
and the update together:
stack: mystackgit: url: https://github.com/user/myapp.git ref: 0a1b2c3d4e5f60718293a4b5c6d7e8f901234567 path: deploy/myappenvironments: prod: {}shc update mystack --check and shc deploy --dry-run read a git
deployment’s remote at its ref and path, in a clone of their own beside the
one a render reads. A branch that moved shows as stale and as a package line,
and a file whose ref changes shows a source line even when the bytes are the
same, since a bare update replays the ref the deployment records.
Upgrade Flow
Section titled “Upgrade Flow”flowchart TB upgrade["shc update <stack> [--from <source>] [--ref <ref>]"] --> load["Load existing source from DB"] load --> override["Apply --from override if given"] override --> resolve["Resolve source<br>(same as install)"] resolve --> fetch["Fetch/copy new app files"] fetch --> track["Update source tracking record"] track --> continue_["Continue normal upgrade flow"]Deployment Storage
Section titled “Deployment Storage”Regardless of source type, app definition files are copied to the deployment directory:
{state_home}/tenants/{tenant}/stacks/{stack}/environments/{env}/app/├── app.yaml├── compose.yaml├── vars.yaml└── ...This ensures consistent behavior once deployed - the source is only relevant for upgrades.
Error Handling
Section titled “Error Handling”Every code below is in the generated
error reference. There is no git-specific
family: every failure of the underlying git process (clone,
fetch, a ref that does not resolve, a private remote that refuses the
credential, a network timeout) comes back as the one code git’s own
non-zero exit produces, with the argv and the captured output attached.
| Error | Code | Message |
|---|---|---|
| Repository app not found | X404012APPDIR | app {name} not found, looked under {candidates} |
| Ambiguous repository app | X409039APPAMB | app {name} exists in more than one repo, found in {repos} |
| Source clone failed on this node | X500902REPMAT | failed to materialize source for repository {name} {error} |
| File-path repo (removed form) | X400927REPFIL | file-path repositories are no longer supported |
Path not found: bad --path, missing subdirectory, bad local source | X404064SRCNFD | source path not found {path} |
Any failing git invocation: clone, fetch, checkout of an unresolvable ref, a private remote refusing the credential | X500905REPGIT | git {args} failed {error} {output} |
app.yaml present but unreadable in the resolved source | X500953APPYRD | app.yaml read failed for app {name} {error} |
app.yaml readable but not valid YAML / not a valid app | X400010APPYML | app.yaml parse failed for app {name} {error} |
Install source is ambiguous between <repo>/<app> and <parent>/<vapp> | X400075SRCAMB | install source {source} with stack name {stack} is ambiguous, {reason}… |
X500905REPGIT scrubs URL userinfo out of {args}, {error} and
{output} before the error is built, so a PAT spliced into a clone URL
never reaches a log line.
Security Considerations
Section titled “Security Considerations”- Token handling: Tokens are used only for clone operations, never stored
- SSH keys: Follow system git configuration
- Path validation: Paths are validated to prevent directory traversal
- URL normalization: URLs are normalized before hashing to prevent cache poisoning