shc export / import
shc export runs an exporter an app ships against one deployment and packs
what its jobs write into a .exp artifact. shc import runs the same
exporter’s import jobs over an artifact. An export takes what the app says it
can hand over, one database or one realm, where a backup takes every data
mount.
| Backup | Export | |
|---|---|---|
| Defined by | SHC | the app, in <exporter>.exporter.yaml |
| Takes | every data mount | what the exporter’s jobs write, chosen by its parameters |
| Runs | converters over a snapshot | jobs against the running deployment |
| Use | disaster recovery | moving data between deployments, a selective copy |
Two exporters ship today:
| Exporter | Deployment | Export parameters | Import parameters |
|---|---|---|---|
postgres | a built-in postgres | database (default postgres), format (default custom, or tar, plain, directory) | database (default postgres) |
keycloak | the system Keycloak | realm (every realm when omitted) | override (default true: a realm that exists is overwritten; false skips what it already has). The master realm is never imported. |
A parameter the exporter does not declare for that direction is refused with
X400528EXPUND before any job runs, and the refusal names the ones it does
declare:
$ shc import postgres --stack pa -p target_database=copy -i /var/lib/shc/import/pashc: [X400528EXPUND] exporter postgres declares no import parameter target_database, its import parameters are database, nothing was run
$ shc export postgres -s pa -p databse=postgres --output-dir /root/pa-typoshc: [X400528EXPUND] exporter postgres declares no export parameter databse, its export parameters are database, format, nothing was runpostgres’s import restores every format its export writes.
shc export
Section titled “shc export”shc export postgres -s mydb -p database=customersshc export postgres -s mydb -p database=orders -p format=plainshc export postgres -s mydb -p database=customers --output-dir /srv/handovershc -t master export keycloak -s keycloak -p realm=my-companyThe exporter is the first argument and the deployment is -s and -e, as on
every verb that names one deployment. Parameters come as repeated
-p key=value, or all at once as --params-json '{"database":"customers"}',
where a -p wins over the same key in the JSON. Typed on a terminal with no
stack named, the command asks for the stack and the environment.
On success it prints one line, the artifact’s path:
$ shc export postgres -s pl5 -p database=postgres --output-dir /root/learn-export-pl5-aexported "postgres" from stack "pl5" to /root/learn-export-pl5-a/4a956b9418d8ee2b.expWith --output-dir the artifact is copied into that directory, which is made
if it is missing. Without it the path printed is in the node’s artifacts
directory, /var/lib/shc/tmp/exporters/artifacts. An artifact is named for its
export id, sixteen hex characters.
An export whose jobs all exit 0 and write nothing is refused with
X500520EXPMTY: there is no import from an empty artifact.
shc export ls
Section titled “shc export ls”Lists the artifacts of one deployment, newest first. -s is required.
$ shc export ls -s pcNAME SIZE BYTES CREATED AT4d3e99feb39e1644.exp 1258 2026-10-08T02:09:11Z2882588ff0cbaef7.exp 1256 2026-10-08T02:09:05ZEvery export runs on the node that holds the deployment’s render (below), and
that node keeps a copy of every artifact. It answers shc export ls wherever
the command is typed, so the list is the same on every node. A node that typed
an export without --output-dir keeps its own copy too. A deployment with no
export yet:
$ shc export ls -s pcno exports of stack "pc" in tenant "prod"There is no command that lists the exporters an app defines: read the app’s
<exporter>.exporter.yaml, or the table above. An exporter is read only from
the app the named deployment was installed from, and a name that app does not
ship fails with X404500EXPNFD. Keycloak’s exporter asked of a postgres:
$ shc import keycloak --stack pbshc: [X404500EXPNFD] exporter keycloak not found in stack pbWhen typed on a node that holds only the deployment’s mounts, shc import,
shc export and shc export ls run on the node holding the render. If no
render holder can be selected, they refuse with X409543EXPNRN, naming the
local node and explaining why its directory cannot run the command.
shc export mount / umount
Section titled “shc export mount / umount”Unpacks an artifact into a directory you can read, after checking its payload checksum:
$ shc export mount /root/learn-export-pl5-a/4a956b9418d8ee2b.expmounted 4a956b9418d8ee2b.exp path: /var/lib/shc/tmp/shc-mount-export-2642592006 id: AiNDdciaB_bRVgLMnKL8TQ readable by the daemon user; the payload may carry credentialsrelease with: shc export umount AiNDdciaB_bRVgLMnKL8TQ
$ shc export umount AiNDdciaB_bRVgLMnKL8TQreleased AiNDdciaB_bRVgLMnKL8TQThe directory is a full copy of the payload, mode 0700. It stays until
shc export umount deletes it, or until it has sat idle for --idle-ttl
seconds, 1800 by default, and the daemon reclaims it.
shc import
Section titled “shc import”shc import postgres --stack mydb -p database=customers -i /var/lib/shc/import/4a956b9418d8ee2b.expshc import postgres --stack mydb -i /var/lib/shc/import/handovershc import postgres --stack mydbshc -t master import keycloak --stack keycloak -p override=false -i /var/lib/shc/import/kc/08b875b761e082d5.exp--stack (or -s) is required. Which artifact is read:
-i/--inputnames an artifact file, or a directory. A file is imported whatever deployment it was exported from, which is how data moves from one deployment to another. A directory is searched for the newest artifact this exporter made of this deployment. The path must sit under the import directory,/var/lib/shc/import; anything else, a..climb or a symlink that leaves it, is refused withX400522IMPDIR. Copy an artifact there first.- With no
-iand no--wrap, the node that holds the deployment’s render takes the newest artifact this exporter made of this deployment from its artifacts directory. -w/--wrap PATH[@SLOT]packs a plain file or directory under the import directory into an artifact first. With no slot the file, or the directory’s contents, sits at the root of@input; with one, under@input/<slot>. The shipped importers read/input/dump.pgdumpand/input/realms/, so wrap a pg_dump file nameddump.pgdumpas it is and a directory of realm JSON files with the slotrealms:
shc import postgres --stack mydb --wrap /var/lib/shc/import/wrap/dump.pgdumpshc -t master import keycloak --stack keycloak --wrap /var/lib/shc/import/kc-realms@realmsA directory, or the render node’s artifacts directory, that holds no export
this exporter made of this deployment is refused with X404540IMPNON, and
nothing runs. A bare import into a postgres that was never exported:
$ shc import postgres --stack pbshc: [X404540IMPNON] no .exp artifact found in /var/lib/shc/tmp/exporters/artifacts that exporter postgres made of deployment prod/prod/pb, so nothing was imported, name the artifact to import with --input or --wrapEach import job that writes output prints it under its name, then the result:
$ shc -t master import keycloak --stack keycloak -p override=false -i /var/lib/shc/import/kc/08b875b761e082d5.exp[realm-import]Skipping master realm (managed by keycloak bootstrap)Realm prod exists; partial-import strategy=SKIPImport complete: 1 realm file(s) processed, 1 skippedimported "keycloak" into stack "keycloak"
$ shc import postgres --stack pl5 -p database=postgres -i /var/lib/shc/import/628e72f1490a6493.expimported "postgres" into stack "pl5"
$ shc import postgres --stack pc --wrap /var/lib/shc/import/wrap/dump.pgdumpimported "postgres" into stack "pc", wrapped from /var/lib/shc/import/wrap/dump.pgdumpA job that fails ends the import with the job’s own error:
$ shc import postgres --stack pc -p database=nosuchdbshc: [X500901OPFAIL] import failed postgres job "pg-restore" failed: exited with code 1: pg_restore: error: connection to server at "postgres" (10.224.0.10), port 5432 failed: FATAL: database "nosuchdb" does not existpostgres’s pg-restore job restores into the database you name: with
pg_restore --clean --if-exists for a custom, tar or directory dump, and
with psql for a plain one, which the export writes with --clean --if-exists too. Either way, what that database holds and the artifact also
holds is replaced.
An artifact with no files in it is refused with X400521IMPMTY.
Where it runs
Section titled “Where it runs”shc export can be typed on any node of the cluster. It runs on the node that
holds the deployment’s render, and each job runs on the node that runs the
replica of the job’s service. The jobs of one run share @output wherever each
runs. The artifact is written on the node the command was typed on: into
--output-dir when one is given, otherwise into that node’s artifacts
directory. A node the export needs and cannot reach is named, and the export
fails.
shc import can be typed on any node too. An artifact named with --input or
--wrap is read on the node the command was typed on, inside that node’s
import directory, and travels with the import. With neither, the node that
holds the deployment’s render reads its own artifacts directory, which holds
every export of the deployment. The import’s jobs run on the node that runs the
replica of each job’s service, and @input goes with them.
A job runs on the engine of the node it lands on. A context: service job runs
on every runtime. A context: container job runs on a docker node and on a
podman node through that node’s own engine, and on an incus or apple node
through SHC’s executor for that runtime, in a project of its own that is
removed when the job ends. On apple the job runs once the address apple gave
its container is the job’s pin. On either, a job that states
network_mode: none joins none of the deployment’s networks, only its own
project’s.
shc export
Section titled “shc export”| Flag | Default | Description |
|---|---|---|
<exporter> | exporter name, as the app ships it | |
-s, --stack | $SHC_STACK, then shc ctx -s, then asked on a terminal | the deployment to export from |
-e, --environment | $SHC_ENVIRONMENT, then shc ctx -e, then the tenant’s default environment | its environment |
-p, --param | key=value, repeatable | |
--params-json | parameters as one JSON object | |
--output-dir | the artifacts directory | where the artifact is copied |
shc import
Section titled “shc import”| Flag | Default | Description |
|---|---|---|
<exporter> | exporter name, as the app ships it | |
-s, --stack | required | the deployment to import into |
-e, --environment | the tenant’s default environment | its environment |
-i, --input | the render node’s newest artifact of this deployment | an artifact file, or a directory holding this deployment’s export, under /var/lib/shc/import |
-w, --wrap | PATH[@SLOT], a plain file or directory under /var/lib/shc/import packed as the artifact | |
-p, --param | key=value, repeatable | |
--params-json | parameters as one JSON object |
The generated pages carry every flag as the binary registers it: shc export, shc export ls, shc export mount, shc export umount, shc import.
The artifact
Section titled “The artifact”A .exp is a Unix ar archive of two members:
| Member | Holds |
|---|---|
metadata.json | what was exported, and the checksum of the data |
data.tar.gz | everything the export jobs wrote to @output, gzipped |
{ "id": "4a956b9418d8ee2b", "exporter": "postgres", "stack": "pl5", "tenant": "prod", "environment": "prod", "timestamp": "2026-10-08T00:17:10Z", "version": "1", "params": { "database": "postgres", "format": "custom" }, "data": { "compression": "gzip", "checksum": "d41802763f5cb1979cebe3517eef5660f07f8817e2d1edcb208fed0e1d0cf21e", "size": "735" }}checksum is the sha256 of data.tar.gz; an import or a mount whose data does
not match it is refused. A wrapped artifact also carries source,
wrapped_from, wrapped_by and wrapped_at.
An ar member holds at most 9999999999 bytes. A larger data.tar.gz is
written as numbered members, data.tar.gz.000, data.tar.gz.001 and on, each
that size but the last; checksum and size describe them joined in order.
To unpack one by hand:
$ ar x 4a956b9418d8ee2b.exp$ cat data.tar.gz.* > data.tar.gz$ tar -xzf data.tar.gzWriting an exporter
Section titled “Writing an exporter”An app ships each exporter as <exporter>.exporter.yaml beside its
compose.yaml; the file’s name is the exporter’s name. SHC reads it only from
the app the named deployment was installed from: the deployment’s own copy of
the app first, then that app in the catalogue. postgres’s, cut to its shape:
schema: export: database: type: string required: false default: "postgres" description: Database name to export format: type: string required: false default: "custom" description: "pg_dump format: custom, plain, directory, tar" import: database: type: string required: false default: "postgres" description: Database name to import into
jobs: export: - name: pg-dump context: container image: "{{ vars.images.postgres }}" networks: [public] environment: PGPASSWORD: "{{ vars.postgres.password }}" command: | clean= case "{{ params.export.format }}" in p | plain) clean="--clean --if-exists" ;; esac pg_dump -h postgres -U "{{ vars.postgres.username }}" \ -d "{{ params.export.database }}" -F "{{ params.export.format }}" \ $$clean -f /output/dump.pgdump test -s /output/dump.pgdump volumes: - "@output:/output" timeout: 3600
import: - name: pg-restore context: container image: "{{ vars.images.postgres }}" networks: [public] environment: PGPASSWORD: "{{ vars.postgres.password }}" command: | if [ -d /input/dump.pgdump ] || [ "$$(head -c 5 /input/dump.pgdump)" = PGDMP ] || [ "$$(dd if=/input/dump.pgdump bs=1 skip=257 count=5 2>/dev/null)" = ustar ]; then pg_restore -h postgres -U "{{ vars.postgres.username }}" \ -d "{{ params.import.database }}" --clean --if-exists /input/dump.pgdump else psql -h postgres -U "{{ vars.postgres.username }}" \ -d "{{ params.import.database }}" -v ON_ERROR_STOP=1 -q -o /dev/null \ -f /input/dump.pgdump fi volumes: - "@input:/input" timeout: 3600A $ the job’s shell should see is written $$, as in a compose file.
schema.export and schema.import declare each direction’s parameters:
type is string, int, float, bool or array, and a required one
that is missing is refused before any job runs. jobs.export and
jobs.import run in order and the first that fails stops the run.
Every string in a job is rendered first, against the deployment’s render
context: vars.* as its update renders them, params.export.* or
params.import.*, hosts.<service>, tenant_name and stack_name.
Import container jobs may declare vault: {encryption_key: encryption_key} and
mount "@vault:/vault". Each map entry names a deployment vault key and a plain
file name in that job’s private mount. The deployment’s app/vars.yaml must
reference each key as ref+vault://<key> (query options are allowed), at
deployment scope: a ref that names a broader ?scope= is refused with
X400559IMPVSC before any job runs, since other deployments read that row. Missing
or empty files fail the import. Values are trimmed and sealed only after all jobs
succeed; a failed write restores earlier writes. Each sealed key leaves one
vault change row naming the deployment and the key, with the reason “imported
by exporter shc vault set, the seal changes the row
at once, and the deployment’s containers read it from their next shc update.
Output jobs must write files without printing secrets; their stdout and stderr
are withheld from results.
The mount is removed after the run. Export jobs cannot declare vault outputs.
@output mounts the directory the artifact is packed from, and @input the
artifact’s unpacked data. A job takes one of two contexts:
context: containerruns a one-off container ofimage. It runs in the network namespace of one of the deployment’s running replicas: the replica of the job’sservicewhen it names one, otherwise the first by name. The job reaches the deployment’s services by the names the replicas use (postgres,keycloak), and thenetworksit lists are not joined. A job that statesnetwork_modekeeps it. When no replica of the deployment runs on the node, the job joins the networks it lists instead, which is enough for a job that only reads a volume.context: servicerunscommandinside the running container ofservice.@outputand@inputare copied in and out around it; any other volume is refused, since the container’s mounts are already fixed.
A job also takes environment, user, working_dir, labels, timeout
in seconds and stall_after in seconds.
timeout is the most a job may take. stall_after is how long an export job
may go without progress: its @output not growing, and its own fabric address
carrying less than 4 KiB in each 30 s sample (an idle connection’s keepalive
probes are not progress). At half of it the job warns with X200557JOBSLO, and at
all of it the job is stopped and the export fails with X504556JOBSTL, naming the
job, its output, the bytes it holds and since when. A context: container
export is watched over 10 minutes when it declares no stall_after; a
context: service export only when it declares one, since its @output is
copied out once it has ended; an import never. A job with a slow phase that
writes and sends nothing, a lock wait before its first row, declares a
stall_after longer than that phase.
Every job container the fabric gives a pin of its own, an export’s, a hook’s
or an app’s, starts with net.ipv4.tcp_keepalive_time 60,
net.ipv4.tcp_keepalive_intvl 10 and net.ipv4.tcp_keepalive_probes 6 in its
network namespace, on docker, podman and incus. A client that asks for
keepalive, as libmariadb and libpq do, learns of a server gone without a reset
in about two minutes, not the kernel’s two hours. A value the job sets in its
own sysctls: is kept.