Komodo (deploy control plane)¶

Komodo is a web UI and API for running Docker across several servers. Here it does one job: take the compose files from the infra repo and keep every server running exactly what git says, then shout on ntfy when that fails.

Core, MongoDB and the local Periphery¶

All three run in one compose project on control-host. Keep this file in the infra repo (hosts/control-host/komodo/compose.yaml) even though Komodo doesn't deploy it, so Renovate sees its images and there's a reviewed history of changes.

services:
  mongo:
    image: mongo:8.3.13        # databases can also be pinned as tag@sha256:<digest>
    command: --quiet --wiredTigerCacheSizeGB 0.25
    restart: unless-stopped
    stop_grace_period: 60s
    volumes:
      - ./mongo/data:/data/db
      - ./mongo/config:/data/configdb
    environment:
      MONGO_INITDB_ROOT_USERNAME: ${KOMODO_DATABASE_USERNAME}
      MONGO_INITDB_ROOT_PASSWORD: ${KOMODO_DATABASE_PASSWORD}
    healthcheck:
      # ping needs no auth
      test: ["CMD", "mongosh", "--quiet", "--eval", "db.adminCommand({ping:1}).ok"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 30s

  core:
    # Keep core and EVERY periphery on the same version
    image: ghcr.io/moghtech/komodo-core:2.3.3
    init: true
    restart: unless-stopped
    depends_on:
      mongo:
        condition: service_healthy
    ports:
      - 9120:9120               # reverse proxy in front; don't expose this to the internet
    env_file: ./compose.env
    environment:
      KOMODO_DATABASE_ADDRESS: mongo:27017
    volumes:
      - ./core/keys:/config/keys
      - ./core/backups:/backups  # daily DB export lands here
      - ./certs/rootCA.crt:/usr/local/share/ca-certificates/rootCA.crt
    networks:
      - default
      - proxy
    healthcheck:
      # The image has no official HEALTHCHECK yet; this checks the UI answers
      test: ["CMD", "curl", "-f", "http://localhost:9120"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 20s

  periphery:
    # Outbound mode: connects to core, no listener to healthcheck. Komodo shows its state as a server.
    image: ghcr.io/moghtech/komodo-periphery:2.3.3
    init: true
    restart: unless-stopped
    depends_on:
      - core
    env_file: ./compose.env
    volumes:
      - ./core/keys:/config/keys
      - /var/run/docker.sock:/var/run/docker.sock
      - /proc:/proc
      # Stacks and repos are cloned here; host path and container path must be identical
      - ${PERIPHERY_ROOT_DIRECTORY:-/etc/komodo}:${PERIPHERY_ROOT_DIRECTORY:-/etc/komodo}
      - ./certs/rootCA.crt:/usr/local/share/ca-certificates/rootCA.crt

networks:
  proxy:
    external: true

# Database
KOMODO_DATABASE_USERNAME=komodo
KOMODO_DATABASE_PASSWORD=<LONG_RANDOM>
TZ=<TZ>

# Core
KOMODO_HOST=https://komodo.<YOUR_LOCAL_DOMAIN>   # used to build the webhook URLs shown in the UI
KOMODO_TITLE=Komodo
KOMODO_PERIPHERY_PUBLIC_KEY=file:/config/keys/periphery.pub
KOMODO_LOCAL_AUTH=true
KOMODO_INIT_ADMIN_USERNAME=<YOUR_USER>
KOMODO_INIT_ADMIN_PASSWORD=<ONLY_USED_ON_FIRST_START>
KOMODO_FIRST_SERVER_NAME=control-host
KOMODO_WEBHOOK_SECRET=<LONG_RANDOM>               # same value goes in the Forgejo webhook "Secret"
KOMODO_JWT_SECRET=<LONG_RANDOM>
KOMODO_DISABLE_USER_REGISTRATION=true

# Periphery (the container above)
PERIPHERY_CORE_ADDRESS=ws://core:9120
PERIPHERY_CONNECT_AS=control-host
PERIPHERY_CORE_PUBLIC_KEYS=file:/config/keys/core.pub
PERIPHERY_ROOT_DIRECTORY=/etc/komodo

The .env symlink exists so ${KOMODO_DATABASE_USERNAME} in the compose file is substituted without passing --env-file every time. Upstream's full list of variables is in their compose.env.

KOMODO_INIT_ADMIN_PASSWORD only seeds the admin account on the first start. Changing it later does nothing. Change the password in the UI, then delete the line so it doesn't linger in a file.

Komodo can't deploy itself¶

If Komodo redeployed its own stack, docker compose up would stop Core in the middle of the deploy that Core is running. So this compose is applied by hand:

# On control-host, after pulling main on your workstation and copying the file over
cd /opt/docker/komodo
cp compose.yaml compose.yaml.bak-$(date +%Y%m%d)
install -m 664 /tmp/komodo-compose.yaml compose.yaml
sudo docker compose config -q && sudo docker compose pull && sudo docker compose up -d
sudo docker compose ps        # mongo + core healthy, periphery Up

Expect a ~30 s UI/API outage while Core restarts. Deployed stacks keep running: they're plain compose projects and don't need Komodo to stay up. Rollback is copying the .bak-* back and running up -d again.

Renovate still opens PRs for these images (manual review). Merging one changes nothing on the host until you apply it.

Keep Core and Periphery in lockstep¶

Core and every Periphery must run the same version. Mixed versions show up as servers that won't connect or API calls that fail in odd ways. Upgrade in one window:

1. Read the release notes.
2. Merge the Renovate PR (core and periphery are grouped into one), apply the compose by hand as above.
3. Upgrade the systemd Periphery on every other host (re-run the install script below, it updates the binary).
4. Check Servers in the UI: all of them Ok on the new version.

MongoDB stays on its major version unless you choose to move it. Mongo majors are dashboard-approval PRs in Renovate.

Periphery on the other hosts¶

On server1, server2 and the workstation, Periphery runs as a systemd service, not a container. It needs no Docker image and survives anything that happens to Docker's containers. Install it with the upstream script, using an onboarding key created in Core's UI or API (it is shown only once):

curl -sSL https://raw.githubusercontent.com/moghtech/komodo/main/scripts/setup-periphery.py \
  | sudo python3 - \
    --core-address="https://komodo.<YOUR_LOCAL_DOMAIN>" \
    --connect-as="$(hostname)" \
    --onboarding-key="O-..."
sudo systemctl enable --now periphery

This installs /usr/local/bin/periphery, a periphery.service unit and /etc/komodo/periphery.config.toml. Re-running the script later upgrades the binary and keeps the config. The onboarding key isn't needed after the first connection.

The server then appears in Komodo. Its clones live under /etc/komodo/repos/ (shared linked repo) and /etc/komodo/stacks/ (per-stack clones).

Git access¶

Settings → Providers: add a git account for your Forgejo host (git.<YOUR_DOMAIN> or the LAN name) with the komodo bot user and its read-only token. Then create one Repo resource:

Linked repo vs per-stack clones¶

A Stack can either clone the repo itself (repo = "<YOUR_USER>/homelab") or point at the Repo resource (linked_repo = "homelab").

Use linked_repo for everything. The only stacks to leave on per-stack clones are ones that still keep state in a relative path like ./data; switching them would point the mount at an empty directory in the shared clone. Move that state to /mnt/appdata/<service> with an absolute path first, then switch.

Stack resources¶

One Stack per hosts/<host>/<stack>/ directory. The easiest way to add one is to copy an existing stack's config and change the server and path. The settings that matter:

Komodo can export resources as TOML (the API's ExportAllResourcesToToml, also available in the UI), which is handy as a record of these settings and as a starting point for a Resource Sync later.

The "Redeploy Changed Stacks" procedure¶

Procedures are a list of stages that Komodo runs in order, triggered by a webhook, a schedule or a button.

[[procedure]]
name = "Redeploy Changed Stacks"
[procedure.config]
schedule_format = "Cron"
schedule = "0 */15 * * * ?"       # Komodo cron has SIX fields: second minute hour day month weekday
webhook_enabled = true
failure_alert = true

[[procedure.config.stage]]
name = "Deploy Changed"
executions = [
  { execution.type = "BatchDeployStackIfChanged", execution.params.pattern = "*" },
]

• Webhook: the procedure's page shows its URL, https://komodo.<YOUR_LOCAL_DOMAIN>/listener/github/procedure/<ID>/main. Put it in a Forgejo push webhook (Forgejo). The /main at the end means pushes to other branches are ignored.
• Schedule: every 15 minutes, so a failed deploy retries without anyone pushing again. BatchDeployStackIfChanged is idempotent: when nothing changed, nothing happens.
• Pattern * means every stack. You could split by host or tag (pattern = "server1-*") if one host's failures shouldn't hold up another's.

Alerters to ntfy¶

Alerters send Komodo's alerts somewhere. Komodo has an ntfy endpoint type. Use several alerters so each kind of alert gets the right topic and priority:

Point them at ntfy over the internal Docker network (http://ntfy:80) so alerts still arrive when DNS or the reverse proxy is broken. Authenticate with a write-only ntfy token stored as a secret Variable (see Secrets).

Alerters support maintenance windows. If something pauses containers on purpose every night (a parity sync, a backup), put the affected alerter in maintenance for that window so it doesn't page you.

Backups of Komodo itself¶

New installs come with a Backup Core Database procedure (daily at 01:00) that writes a dated, gzipped export of the database into /backups, mounted from ./core/backups. It keeps the latest 14 by default.

Back up the whole /opt/docker/komodo directory with your host backup (it includes compose.env, the keys and those exports). Restore with the Komodo CLI: km database restore -y (latest export, or --restore-folder for a specific one). Restores don't clear the target database first, so restore into an empty Mongo.

Sources & further reading¶

• Komodo: Setup and Connect servers
• Komodo: Docker Compose stacks
• Komodo: StackConfig fields
• Komodo: Procedures, Schedules, Webhooks
• Komodo: Variables and secrets
• Komodo: Git and registry providers
• Komodo: Database backup
• Komodo upstream compose.env