Huly
Huly is an all-in-one project management platform (a Linear / Jira / Slack /
Notion alternative). SHC packages it as a 13-service stack behind a bundled
nginx that path-muxes every platform service, so one --expose serves the whole
product.
Installation
Section titled “Installation”Huly’s database and blob store are integrations, not bundled services.
Create a bucket at the hosted provider, then connect and grant its key as
shown in Object storage for apps.
Use that connection as <connection> and the bucket you made as <bucket>.
The first line selects the tenant Huly goes into, and every command on this
page then runs in it:
shc ctx -t <tenant>shc install postgres --stack huly-dbshc install external-s3 --stack huly-blobs -C s3=<connection>
shc install huly --stack huly \ -i postgres=huly-db \ -i s3=huly-blobs \ --var s3.bucket=<bucket> --var s3.accessKey= --var s3.secretKey= \ --expose huly.example.comHuly passes s3.bucket as rootBucket, so all workspaces share the bucket
with a separate object-key prefix for each workspace. The empty key overrides
make Huly take the connection’s key pair from the front.
Optional integrations:
-i oidc=<stack> # native OIDC on the huly sign-in page -i mail=<stack> # outbound signup/invite mail through mailcow or the mail appTo point at a database or object store SHC does not manage, set
vars.postgres.hostname / vars.s3.endpoint instead of attaching the plug:
each plug is required only while its var is empty.
What lives where
Section titled “What lives where”| Component | Where | Why |
|---|---|---|
| Accounts, people, workspaces, issues, documents, the tx log, hulykvs | integrated postgres stack | Durable. Needs backing up, so it is an integration. |
| Uploads, attachments, avatars, collaborative-document blobs | hosted provider bucket through the external-s3 front huly-blobs | Durable. Back up the bucket separately. |
elastic | bundled, own volume | Derived fulltext index, rebuildable from the database + blobs. |
redpanda | bundled, own volume | Work-distribution queue. Not the system of record, but see the caveat in apps/business/huly/backup.yaml: it holds the fulltext consumer’s offsets. |
What shc status huly is claiming
Section titled “What shc status huly is claiming”Every service in the stack declares a healthcheck that makes one real request
on its own surface, so healthy means the service answered, not merely that a
container is running. Each probe was chosen by measuring the pinned image:
| Service | Probe | Healthy answer |
|---|---|---|
nginx | GET /_healthz, a location nginx serves itself | 200 |
front | GET / (the SPA) | 200 |
account | GET / (the RPC mount is POST-only) | 405 |
transactor | GET /api/v1/health | 200 |
collaborator | GET /api/v1/statistics with no token | 401 |
fulltext | GET /api/v1/search (the route is PUT-only) | 405 |
stats | GET /api/v1/manage (the route is PUT-only) | 405 |
rekoni | POST /toText with an empty body | 400 |
mail | POST /send with an empty body | 400 |
kvs | GET /api/health over bash’s /dev/tcp | 401 |
workspace | its own heartbeat in stats | entry fresher than 120s |
elastic, redpanda | cluster health / rpk cluster info | green-or-yellow / ok |
workspace is the odd one and worth knowing about: it has no HTTP surface at
all: no port, no routes, nothing listening but docker’s resolver. It is a
poller. Every huly service pushes metrics to stats every 5 seconds under its
own container id, and stats drops any service it has not heard from for five
minutes, so the probe reads that heartbeat back. If the poller’s timers stop,
the entry goes stale and the service is reported unhealthy, which is the
question that previously took hand-sampling pg_stat_activity to answer.
A transient stats outage does not flip it: the freshness window is 24x the
push interval and the retry budget is 5.
PWA icons and BRANDING_URL
Section titled “PWA icons and BRANDING_URL”The front image keeps its PNG icons under /huly/ (/huly/icon-192.png and
siblings). The shipped site.webmanifest asks for them at the origin root
(/icon-192.png). On an estate with BRANDING_URL unset the browser still
hits those root paths and they 404.
This package aliases the three manifest sizes to /huly/… on nginx, and
serves a host-keyed /branding.json that front’s BRANDING_URL points at
(the image’s own branding.json is keyed on huly.local and is a no-op
here). Bind -i branding=BRANDING to add the estate logo as an extra icon;
an unbound install stays on the stock /huly/ favicons.
Endpoints huly advertises but does not deploy
Section titled “Endpoints huly advertises but does not deploy”/config.json carries LOVE_ENDPOINT, CALENDAR_URL, GMAIL_URL and
TELEGRAM_URL because front refuses to boot without them. No service backs
any of the four in this stack, so nginx answers them with 501 Not
Implemented rather than letting them fall through to the SPA, which used to
answer 200 text/html and hand a client an app shell where it expected a
protocol reply.
Backups
Section titled “Backups”A single-stack backup of huly captures only elastic and redpanda. To
capture the database too, back it up with the connected stacks:
shc -s huly backup create --all-connected # -> <group>.bkgshc recover --file <group>.bkg -y # restores postgres and huly--all-connected takes the huly-blobs front with it. The front’s bucket
capture is off as shipped, so the group is recorded completed with the
cause degraded and holds no blobs: the bucket is the provider’s to keep. To put the blobs
in the archive, turn the front’s capture on first, and read what that costs
in disk (Object storage for apps).
PostgreSQL, not CockroachDB
Section titled “PostgreSQL, not CockroachDB”Upstream huly-selfhost ships CockroachDB. SHC runs huly on plain
PostgreSQL 16. Huly speaks the Postgres wire protocol either way, and at the
pinned v0.7.426 plain PostgreSQL is a first-class target:
accountandworkspacecarry a flavor table (dbTypes = { cockroach: {...}, postgres: {...} }) and choose a branch fromgetDbFlavor(), which runsSELECT version()and matches/cockroach/iagainst/postgresql/i. There is no scheme or env to set: detection is automatic. The postgres branch emitsTEXT/BYTEA/BIGINT, createspgcryptoand agen_random_bigint()helper in place of cockroach’sunique_rowid(), usesGENERATED ALWAYS AS (...) STOREDgenerated columns, and guards enum/constraint changes withDO $$ ... $$blocks.hulykvs(Rust) ships two refinery migration runners,migrations_crdbandmigrations_pg, and picks one from the detected backend.- The per-workspace platform tables are engine-generic (
text,bigint,bool,JSONB,USING ginontext[]), and the transaction-retry path keys off SQLSTATE40001, which both engines raise.
The one exception: the communication API
Section titled “The one exception: the communication API”The communication messaging backend is genuinely CockroachDB-only: its
package is literally foundations/communication/packages/cockroach and has no
flavor detection. Its migrations use DEFAULT unique_rowid(),
ALTER TABLE ... ALTER PRIMARY KEY USING COLUMNS, DROP INDEX <unique constraint> CASCADE and a schema-qualified RENAME TO, none of which
PostgreSQL accepts, and the runner rethrows on the first failure.
It is reached only through the COMMUNICATION_API_ENABLED gate in the
transactor and the fulltext service, which SHC pins to "false". That is what
upstream does too: its own dev/docker-compose.pg.yaml pairs postgres:16
with COMMUNICATION_API_ENABLED=false on transactor, fulltext and front,
while the cockroach compose sets it true.
Do not set it to true while the database is PostgreSQL. Schema init
would fail its second migration and retry-loop forever. This is not a
regression from the CockroachDB packaging; SHC never enabled it there either.
Migrating an existing install
Section titled “Migrating an existing install”An install created before huly’s database and blob store became integrations
runs a bundled CockroachDB (cr-data, cr-certs) and a bundled MinIO
(files). Those three volumes no longer exist, so a plain shc update will
not carry the data across. This is a deliberate, one-time data migration.
This runbook has not been executed against a production estate. Rehearse it on a copy before running it anywhere you care about, and keep the pre-cutover backup until the verification steps below pass.
Why not a SQL-level dump
Section titled “Why not a SQL-level dump”The obvious instinct (pg_dump the old engine, psql it into the new one)
does not work here, for three independent reasons:
cockroach dumpno longer exists. Verified against the pinned image:docker run --rm --entrypoint /cockroach/cockroach cockroachdb/cockroach:v24.2.10 dump --helpanswersERROR: unknown command "dump" for "cockroach". CockroachDB’s ownBACKUP/RESTOREonly restores into CockroachDB.- The schemas are not the same schema. The account service emits
different DDL per flavor:
key STRING AS (CONCAT(type::TEXT, ':', value)) STOREDon cockroach versuskey TEXT GENERATED ALWAYS AS (global_account.social_id_type_to_text(type) || ':' || value) STOREDon PostgreSQL,unique_rowid()defaults versusgen_random_bigint(), and so on. A row-level copy would fight the generated columns and the differing defaults. - The blob layout changes too. The bundled MinIO ran without
rootBucket, so huly minted a bucket per workspace with objects at the bucket root. The integrated store uses one bucket with objects keyed<workspaceDataId>/<name>. No database dump addresses that.
The correct vehicle is huly’s own logical backup tool, which reads and writes through the platform adapters and is therefore both engine-agnostic and storage-layout-agnostic. It solves all three problems in one pass.
The tool
Section titled “The tool”hardcoreeng/tool:v0.7.426 (same version train as the rest of the stack).
Relevant commands, and the env each reads:
| Command | Purpose |
|---|---|
backup <dir> <workspace> | Dump a workspace’s transactions and blobs into a portable directory. |
backup-restore <dir> <workspace> --accounts | Replay that directory into the workspace the tool is currently pointed at. |
create-account <email> -p <pw> -f <first> -l <last> | Recreate an account on the target account DB. |
create-workspace <name> <owner_social_id> -d <dataId> | Recreate a workspace record, pinning its dataId. |
fulltext-reindex <workspace> / fulltext-reindex-all | Enqueue a full Elasticsearch rebuild. |
Env contract: DB_URL, ACCOUNT_DB_URL (note: not ACCOUNTS_DB_URL),
ACCOUNTS_URL, SERVER_SECRET, STORAGE_CONFIG, QUEUE_CONFIG, and
FULLTEXT_URL for the reindex commands. Copy the values straight off the
running containers (docker inspect --format '{{range .Config.Env}}{{println .}}{{end}}' <deployment>-transactor-1) so the tool sees exactly what the stack
sees.
Pinning dataId in step 5 is not optional: it is both the old bucket name and
the new object-key prefix, so reusing it is what makes the restored blobs land
where the restored rows expect them.
Two shapes, and when to pick which
Section titled “Two shapes, and when to pick which”The database half is a cross-engine logical dump/restore either way; there is no shortcut for it. The blob half has two routes:
Route A: let the tool carry the blobs (default, fewest moving parts).
backup downloads every blob into the dump directory and backup-restore
re-uploads them through the storage adapter, which writes the new
rootBucket-prefixed layout automatically. Nothing else to do. The cost is
that every byte makes a round trip to local disk and back, and the dump
directory has to be big enough to hold the entire blob set.
Route B: sync the buckets directly (for large blob sets). The layout
change is a pure key remap: the old bundled MinIO ran without rootBucket, so
workspace X had its own bucket named X with objects at the bucket root;
the new store puts those same objects at <bucket>/X/<name>. That is a
per-workspace mc mirror. Use the endpoint and key held by the connection
for new, and the bucket you already made at that provider:
mc alias set old http://<old-minio>:9000 <old-ak> <old-sk>mc alias set new https://<connection-endpoint> <keyID> <applicationKey>
# one per workspace dataId from step 3mc mirror --preserve old/<data_id> new/<bucket>/<data_id>Then skip the blob domain in the dump so you are not moving the bytes twice;
huly’s blob domain is literally named blob:
... backup /backup/<ws-url> <ws-url> --skip blobRoute B keeps the byte movement server-to-server and can run before the
cutover while the old stack is still serving reads, which is the main reason
to choose it: it turns most of the copy into pre-work and shortens the
downtime window to the database migration alone. Re-run the mc mirror once
more after writes stop to pick up anything that landed in between.
Route B is the less-rehearsed path. If the instance is small enough that Route A finishes in an acceptable window, prefer Route A.
Procedure
Section titled “Procedure”Downtime: the whole window, from step 2 to step 8. Huly is not usable mid-migration; a partially-migrated workspace is worse than a stopped one. Budget roughly the time of one full workspace read plus one full write, plus the reindex; for a large workspace that is hours, not minutes. Announce it.
1. Take a safety net. Full group backup of the current stack, on the old packaging, before touching anything:
shc -s huly backup create --all-connectedKeep the .bkg. It is the only rollback that restores the old shape.
2. Stop writes. Scale the front door to zero so no one can mutate state while you are copying it, but leave the backends up: the tool needs the transactor, cockroach and minio reachable:
shc update huly --scale nginx=03. Enumerate the workspaces. From the old cockroach:
docker exec <old-deployment>-cockroach-1 \ cockroach sql --insecure -d huly \ -e "SELECT url, data_id, uuid FROM global_account.workspace;"Record url and data_id for every row. You need both later.
4. Dump each workspace with the tool pointed at the old stack. Attach
it to the old stack’s network and give it the old DB_URL / ACCOUNT_DB_URL
/ STORAGE_CONFIG:
mkdir -p /var/backups/huly-migrationdocker run --rm \ --network <old-deployment>_default \ -v /var/backups/huly-migration:/backup \ -e DB_URL='postgres://huly:<pw>@cockroach:26257/huly' \ -e ACCOUNT_DB_URL='postgres://huly:<pw>@cockroach:26257/huly' \ -e ACCOUNTS_URL='http://account:3000' \ -e SERVER_SECRET='<vars.secret>' \ -e STORAGE_CONFIG='minio|minio?accessKey=<ak>&secretKey=<sk>' \ -e QUEUE_CONFIG='redpanda:9092' \ hardcoreeng/tool:v0.7.426 backup /backup/<ws-url> <ws-url>Repeat per workspace. Verify each dump before moving on:
docker run --rm -v /var/backups/huly-migration:/backup \ hardcoreeng/tool:v0.7.426 backup-check /backup/<ws-url>Do not proceed past a failing backup-check.
5. Stand up the new providers and reinstall huly. Create the destination bucket and grant the connection first, as in Object storage for apps. Use the same bucket as the copy destination above.
shc install postgres --stack huly-dbshc install external-s3 --stack huly-blobs -C s3=<connection>
shc uninstall huly -yshc install huly --stack huly \ -i postgres=huly-db \ -i s3=huly-blobs \ --var s3.bucket=<bucket> --var s3.accessKey= --var s3.secretKey= \ --expose huly.example.comThe new database uses postgres_password. The empty s3.accessKey and
s3.secretKey overrides make Huly use the connection’s key pair. Carry over
vars.secret (the shared inter-service secret) and the Keycloak client secret
if you want existing sessions and the existing OIDC client to keep working:
shc vault get huly/secret # on the old stack, before uninstallshc vault set huly/secret <value>6. Recreate accounts and workspaces on the new database, reusing the
dataId values from step 3. Use the connection’s endpoint, region and key
pair below. rootBucket must match the s3.bucket set on the new Huly stack:
docker run --rm --network <new-deployment>_default \ -e DB_URL='postgres://huly:<pw>@<pg-host>:5432/huly' \ -e ACCOUNT_DB_URL='postgres://huly:<pw>@<pg-host>:5432/huly' \ -e ACCOUNTS_URL='http://account:3000' \ -e SERVER_SECRET='<vars.secret>' \ -e STORAGE_CONFIG='s3|https://<connection-endpoint>:443?accessKey=<keyID>&secretKey=<applicationKey>®ion=<connection-region>&rootBucket=<bucket>' \ -e QUEUE_CONFIG='redpanda:9092' \ hardcoreeng/tool:v0.7.426 \ create-workspace '<ws-name>' '<owner_social_id>' -d '<data_id>'Accounts come back with --accounts on the restore in the next step; use
create-account only for accounts that own no workspace and therefore appear
in no dump.
7. Restore each workspace with the tool pointed at the new stack (same env as step 6):
... hardcoreeng/tool:v0.7.426 \ backup-restore /backup/<ws-url> <ws-url> --accounts8. Rebuild the fulltext index. Nothing does this automatically after an out-of-band restore:
... -e FULLTEXT_URL='http://fulltext:4700' \ hardcoreeng/tool:v0.7.426 fulltext-reindex-all9. Bring the front door back.
shc update huly --scale nginx=1Verifying success
Section titled “Verifying success”Do all of these before you delete the old volumes.
-
Rows landed. The account schema exists and carries the expected identities:
Terminal window docker exec <pg-deployment>-postgres-1 psql -U postgres -d huly -c \"SELECT count(*) FROM global_account.workspace;"docker exec <pg-deployment>-postgres-1 psql -U postgres -d huly -c \"SELECT type, value FROM global_account.social_id ORDER BY value;"The workspace count must equal step 3’s, and every user’s email must be present.
-
The postgres branch ran, not the cockroach one. A direct fingerprint: this function only exists on the PostgreSQL migration path:
Terminal window docker exec <pg-deployment>-postgres-1 psql -U postgres -d huly -tAc \"SELECT count(*) FROM pg_proc p JOIN pg_namespace n ON n.oid = p.pronamespaceWHERE n.nspname = 'global_account' AND p.proname = 'gen_random_bigint';"Expect
1. Expectkvsininformation_schema.tablestoo; that proves hulykvs tookmigrations_pg. -
Blobs are keyed, not bucketed. One bucket, workspace-prefixed keys:
Terminal window mc ls new/<bucket>/Use the
newalias configured in Route B, even if you used Route A to copy the blobs. Expect one prefix perdataIdin the destination bucket. If those prefixes are missing, stop and re-checkSTORAGE_CONFIG. -
The product works. Log in as a real user, open a project that existed before the migration, open an issue with an attachment and confirm the attachment downloads (that is the blob path end to end), then search for a word from inside a document (that is the reindex).
-
Only then remove the old volumes.
Rolling back
Section titled “Rolling back”Before step 5, rollback is free: shc update huly --scale nginx=1. After step 5,
roll back by recovering the step-1 .bkg, which restores the old bundled
shape wholesale. There is no partial rollback; do not run a half-migrated
stack.
Before you install
Section titled “Before you install”These answers used to sit on this app’s store listing. They moved here on 2026-08-17: a shopper deciding whether to install is not reading six hundred words of Q&A, and the listing page is the pitch. The knowledge is no less true for being off the storefront: it is the part you want before you commit, which is what a guide is for.
Is this stock Huly?
Section titled “Is this stock Huly?”It is a soft fork, and it says so. Huly is EPL-2.0, which is file-level copyleft, so the changes are published as a numbered patch series against a pinned upstream release rather than disappearing into a private build. The patches add the meeting-provider picker, the Matrix bridge and on-behalf writes; everything else is upstream.
Do I need Matrix or MongoDB to run it?
Section titled “Do I need Matrix or MongoDB to run it?”No. Both are optional integrations. Without a Matrix application service the bridge is not deployed; without MongoDB and an inference endpoint the assistant is not deployed. The tracker, the documents, the chat and the files work on their own.
What does it replace?
Section titled “What does it replace?”The three subscriptions most teams carry side by side (an issue tracker, a wiki and a team chat), collapsed into one application, one database and one backup, on hardware you already pay for.