Infrastructure & Deployment

Octopus runs on two Hetzner servers, both fronted by Cloudflare. Each server runs an independent Go backend + Next.js client under systemd, and both share the Docker data containers (Postgres/Valkey/RabbitMQ). The static sites and the claim worker live on Cloudflare (see Cloudflare).

The two servers

Master reference

Everything, per domain:

PROD vs SANDBOX namespacing

Both environments run on the same host family and share the Postgres/Valkey/RabbitMQ containers. Isolation is by name/prefix only:

All services run as octopus:octopus with Restart=always, RestartSec=5, LimitNOFILE=65535, LimitNPROC=4096, and hardening (NoNewPrivileges, PrivateTmp, ProtectSystem=strict, ProtectHome). The backend units run ExecStart=… octopus run-all (server + workers + crons). Sandbox is deliberately not After=octopus.service so a prod restart doesn't cascade.

systemd services

Files in deploy/:

• octopus.service — PROD backend. WorkingDirectory=/opt/octopus, env /etc/octopus/octopus.env, ReadWritePaths=/var/log/octopus /opt/octopus/uploads. Port from env PORT=:8081.
• octopus-sandbox.service — SANDBOX backend on :8082. Adds Environment=JOB_LOGS_DIR=/var/log/octopus-sandbox/job_executions (because ProtectSystem=strict makes /var/log/octopus read-only here).
• octopus-client.service — PROD Next.js. ExecStart=/usr/bin/node /opt/octopus-client/server.js on :3001, After=octopus.service.
• octopus-sandbox-client.service — SANDBOX Next.js on :3002.

nginx topology

Configs ship HTTP-only; certbot --nginx adds the 443 listener, cert paths, and HTTP→HTTPS redirect.

• nginx-octopus.conf — upstream →127.0.0.1:8081.
  • api.*: default_type application/json, client_max_body_size 50M. Allow-list ^/(api|auth|webhooks|client|g2a)(/|$); /status fast path; blocks dotfiles & /debug/pprof; catch-all returns JSON 404 (no HTML surfaces). Forwards X-Real-IP (app trusts only 127.0.0.1). proxy_buffering off.
  • warden.*: client_max_body_size 100M, 600s admin timeouts. / → 302 /admin/login. Allow-list ^/(admin|assets)(/|$); catch-all 404.
• nginx-octopus-sandbox.conf — same two blocks →127.0.0.1:8082 for sandbox-api/sandbox-warden.
• nginx-octopus-client.conf / -sandbox-client.conf — pure pass-through to :3001/:3002 (no path allow-list, no nginx caching — Cloudflare handles the edge). proxy_buffering off for SSR streaming.
• nginx/octopus-docs.conf — static file serving from /opt/octopus-docs (templated __DOCS_DOMAIN__); immutable caching for /_next/static/. (Self-hosted docs path; the live docs are on CF Pages.)
• nginx/signoz.conf — telemetry.* → :8080 (SigNoz UI, WebSocket upgrade) and otel.* → :4318 (OTLP /v1/traces|metrics|logs, client_max_body_size 50M).
• nginx/mypopupgiftcards.conf — white-label aliases: api./admin. → :8081, client. → :3001.

Docker data plane

Root docker-compose.yaml — single host, containers bind 0.0.0.0, data under ./.data/*:

Read-replica: deploy/replica/docker-compose.yaml runs pg_replica on a separate machine (pg_basebackup off slot replica_slot). See Database → replication.

Deploy & update scripts

Setup scripts (deploy/setup-*.sh) run on a fresh host; update scripts (deploy/update-*.sh) do safe in-place swaps with rollback. General flow: build → copy to /opt → install unit + nginx → migrate/seed → start.

• setup-octopus-service.sh (PROD, the primary bootstrap) — creates the octopus user + dirs, builds the Go binary, interactively generates /etc/octopus/octopus.env (openssl-random JWT_SECRET/APP_KEY/PG_PASS; prompts for feature flags, domain, OTel, LE email), brings up docker, runs migrations + 7 reference seeders, installs the unit + logrotate + nginx, and optionally runs certbot for api/warden.
• setup-octopus-sandbox-service.sh — reads the Postgres password from the running container, creates octopus_sandbox, writes /etc/octopus-sandbox/octopus.env (PORT=:8082, CACHE_PREFIX=sandbox:, QUEUE_EXCHANGE=app.sandbox, WEBAUTHN_RP_ID=octopuscards.io), installs the sandbox unit + nginx. No TLS (run certbot manually).
• setup-octopus-client-service.sh / -sandbox-client- — install Bun (build) + Node 20 (runtime), write the NEXT_PUBLIC_* env, build, install the standalone build + unit.
• update-octopus.sh — pre-flight checks → env drift-fix (appends missing keys) → verify Postgres reachable (never touches docker compose — restarting containers mid-update caused outages) → go build to tmp → backup running binary → stop → install → migrate up (rollback on failure) → refresh unit/logrotate → start → health check against :8081/status (rollback on failure) → prune backups (keep 5).
• update-octopus-sandbox.sh — same, health at :8082/status, and refuses to run if PG_DB=octopus (guard against migrating the prod DB).
• update-octopus-client.sh / -sandbox-client- — bun build → tar old build → install new standalone → restart → verify.
• uninstall-octopus-sandbox.sh — destructive, requires --yes-i-am-sure (supports --dry-run); removes sandbox units/dirs/nginx, drops octopus_sandbox, deletes sandbox:* Valkey keys and the app.sandbox exchange. Multiple guards cross-check every path against a prod denylist and refuse to drop a DB named octopus.

Log rotation: octopus.logrotate → /var/log/octopus/*.log daily, keep 14, compressed, copytruncate.

CI

.github/workflows/: test-backend-api.yml (Go 1.25 — lint → build → parallel test jobs incl. contract/orchestration tags against mocky-balboa.octopuscards.io), plus the frontend deploy workflows (grasshopper, grasshopper-sandbox, mocky-balboa, octopus-website, octopus-docs — all bun + wrangler). There is no Makefile; .githooks/pre-push handles git-side checks.