Backend Architecture

How the Octopus Go backend is structured — one binary, layered packages, the request lifecycle, auth, and configuration.

The backend is a single Go module (github.com/vnay92/octopus) that compiles to one binary with subcommands. It's built on Fiber v2, uses Squirrel for SQL, Goose for migrations, gocron for scheduling, and OpenTelemetry end-to-end.

One binary, many roles

There is no cmd/ directory — the whole CLI lives in main.go (~877 lines) using the Kong parser. .env is auto-loaded; the process forces time.Local = UTC.

Command	Function	What it does
`server`	`runServer`	Fiber HTTP server on `$PORT` (default `:8081`). `--debug` → pprof on `:6060`. Builds the Jet template engine from embedded `views/`, registers middleware, mounts routes. Graceful shutdown on SIGINT/SIGTERM.
`worker`	`startJobs`	RabbitMQ consumers from `processorRegistry`. `--name` filters. Currently a no-op (registry is empty — see below).
`cron [names]`	`startScheduler`	gocron scheduler; optional names filter which jobs run. Singleton mode prevents overlap; per-job metrics wired.
`run-all`	`runAll`	Workers + crons in background goroutines, then the server (blocking). The typical production entrypoint (`ExecStart=… run-all`).
`migrate <cmd>`	`handleMigrate`	Goose migrations against embedded SQL. `up/down/fresh/create/status/…`.
`seed <cmd>`	`handleSeed`	Seeders: `all`, `fresh`, `list`, `info`, `<name>`.
`pump-inventory`	`handlePumpInventory`	One-shot DB→Valkey voucher-inventory pump.

Gotcha: the queue is dormant

processorRegistry is initialized empty (main.go:165) and nothing is appended, so worker/run-all start zero RabbitMQ consumers today. The queue plumbing is fully wired and RabbitMQ is provisioned, but all real background work currently runs through the cron scheduler, not the message queue. Don't go hunting for queue workers that fulfil orders — they're cron jobs.

Cron registration is feature-flag-gated in init(). Always-on: export processor/cleanup, job-execution-cleanup, webhook-delivery. Vouchers, eSIM, Shopify, payouts, and topups each add their own tasks. All tasks are wrapped with execution tracking (scheduler.WrapTasksWithTracking). See Jobs & Observability for the full cron table.

Layered structure

The request flow is HTTP handler → repo (data) / services (business logic) → database / cache / queue, with cross-cutting middleware. Shared resources come from a singleton DI container.

Request lifecycle

Every /api/v1 request passes through the global middleware chain before it reaches a handler:

Top-level directory map

Dir	Purpose
`main.go`	Single entry point; Kong CLI, server/worker/cron bootstrap.
`config/`	Constants + helpers (time formats, cookie config, security headers/Helmet). Not a config loader.
`singleton/`	Global DI container (logger, DB, cache, queue, email, tracer).
`database/`	DB init, `models/`, `repo/`, `seeder/`, `migration/` (~244 Goose files), tenant manager.
`http/`	`handler/` (74 files), `routes/`, `entity/` (DTOs + AppError).
`middleware/`	Fiber middleware.
`services/`	Business logic; `external_vendors/`, `shopify/`.
`jobs/`	Background job implementations (retries, pumps, exports, webhooks, prefetch state machine).
`scheduler/`	Cron task wrappers implementing `ScheduledTask`, wrapping jobs with tracking.
`queue/`	Queue abstraction (RabbitMQ + in-memory) + `Processor` interface.
`pubsub/`	Pub/sub (Valkey + in-memory).
`cache/`	Cache abstraction (Valkey / in-memory / nil).
`processor/`	Import-pipeline processors (products, inventory, vendor products) — not queue workers.
`metrics/`	OTel instruments + cron-metrics store.
`errors/`	Central `AppError` catalog + sentinels + gRPC code mapping (legacy).
`email/`	SendGrid/SMTP delivery + embedded templates.
`common/`	Shared constants and `types/` (context keys like `TenantIDKey`).
`utils/`	`FetchEnv`, feature flags, auth/JWT service, tracer/metrics/log init.
`web/`	CORS origin allowlist.
`views/`	Jet templates — server-rendered admin UI + emails (ship with the binary).
`assets/`	Static assets served from embed.
`frontend/`	Separate frontend apps.
`deploy/`	systemd units, nginx configs, replica compose, setup/update scripts.
`test/`	Integration/e2e/load tests, fakes (`octopusfake`), fixtures.

Key patterns

Repository pattern + Squirrel. All SQL is built with Squirrel (PlaceholderFormat(squirrel.Dollar)). Every method hangs off one Repository struct carrying dbClient + cacheClient. Columns/scan targets come from per-model helpers (getOrderTableColumns / getOrderScanValues). sql.ErrNoRows → (nil, nil). Soft deletes via deleted_at IS NULL.
Handler pattern. Handlers are methods on the shared *Handler, returning error. parseQueryParams builds QueryFilters from the query string; AddPaginationHeaders emits X-Page, X-Per-Page, X-Total-Count, X-Total-Pages, X-Has-More.
Transactions. Repository.StartTransaction(ctx, opts) → *sql.Tx; convention is tx, _ := r.StartTransaction(...) then defer tx.Rollback(). Methods have …InTransaction variants for tx-scoped calls.
Tracing (OTel). Every repo/handler/service method opens a span: ctx, span := utils.Tracer.Start(ctx, "repo.get-order-by-id"); defer span.End(). Fiber via otelfiber, SQL via XSAM/otelsql, Valkey via valkeyotel.
Logging (zap). Central sugared logger from the singleton; request/task-scoped via middleware.Logger(ctx). When OTel is on, logs tee to both stdout and SigNoz.
Error handling. errors/errors.go holds a catalog of entity.AppError values with fluent .WithMessage() / .WithDetails(). Central Fiber handler is middleware.CustomErrorHandler.

Auth & route groups

Global middleware chain: PanicRecovery → otelfiber → Tracing → Metrics → CORS (strict allowlist, credentials) → Helmet → PanicLogging → ErrorLogging. The app trusts X-Real-IP only from 127.0.0.1/::1 (the nginx proxy).

Route group	Auth mechanism
`/auth` (`login`, `refresh`)	Public. `login` issues JWT access+refresh.
`/api/v1/**`	JWT bearer + IP whitelist. Token validated against signature, expiry, DB record (revocation), and must not carry a `user_id` (portal tokens rejected). Client must be active.
`/g2a/**`	G2A OAuth2 bearer issued by Octopus's own token endpoint. `oauth/token` public; rest behind `G2AAuthMiddleware` (stateless, ~15m TTL).
`/admin/**`	Cookie session (server-side, cache-backed). Password + optional TOTP 2FA. Write routes gated by `AdminWriteMiddleware`; super-admin by `AdminSuperUserMiddleware`.
`/client/**`	httpOnly cookie JWT — token must have a `user_id`; user + client active. Per-route permission checks (`RequireClientPermission`). Public sub-routes: login/refresh/logout, forgot/reset, 2FA verify, WebAuthn/passkey begin/finish.
`/webhooks/**`	No JWT — vendor-authenticated by signature or unguessable URL path-token.
`/pages/*`, `/status`, `/assets`	Public.

Auth building blocks: JWT (golang-jwt/jwt/v5, utils.NewAuthService), WebAuthn (go-webauthn/webauthn), TOTP (pquerna/otp, with backup codes), client permissions (CanCreateOrders, CanManagePayouts, CanManageUsers, CanAccessBilling).

Automation blocking

In production, requests with header X-Automation-Request: true are blocked (middleware/automation_blocker.go) — all of /admin/*, and write operations on /api/v1/*. This prevents automated agents from creating orders or calling vendor APIs against prod.

Configuration

Config is read ad-hoc via utils.FetchEnv / os.Getenv — the config/ package only holds constants. Feature flags are parsed once into utils.FeatureFlags. The important variables, by area:

Core: PORT, SERVICE_NAME, SERVICE_VERSION, ENVIRONMENT/ENV/APP_ENV, APP_NAME, APP_KEY, APP_BASE_URL, LOG_LEVEL, INSECURE_MODE.
Feature flags: FEATURE_VOUCHERS_ENABLED, FEATURE_PAYOUTS_ENABLED, FEATURE_SHOPIFY_ENABLED, FEATURE_VOUCHER_LINKS_ENABLED, FEATURE_TOPUPS_ENABLED, FEATURE_ESIM_ENABLED. (Shopify & voucher-links require vouchers.)
Database: DATABASE (postgres|sqlite), PG_HOST/PORT/USER/PASS/DB/SSLMODE, read replica PG_READ_*, pool PG_MAX_OPEN_CONNS/PG_MAX_IDLE_CONNS/PG_CONN_MAX_LIFETIME, PG_TENANT_CONFIG, DB_QUERY_LOGGING.
Cache/Queue: CACHE_TYPE, CACHE_PREFIX, VALKEY_ADDR, VALKEY_PASS, QUEUE_PROVIDER, QUEUE_CONNECTION, QUEUE_EXCHANGE.
Auth: JWT_SECRET, WEBAUTHN_RP_ID, WEBAUTHN_RP_NAME, WEBAUTHN_ORIGINS, COOKIE_DOMAIN, CLIENT_PORTAL_ORIGIN(S).
Observability: OTEL_ENABLED, OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_TOKEN.
Email/integrations: EMAIL_PROVIDER, EMAIL_DEFAULT_SENDER, SENDGRID_API_KEY, SMTP_*, XE_USERNAME/XE_PASSWORD (forex), TWILLIO_*, OPENAI_API_KEY, MOCKY_BASE_URL, VOUCHER_BASE_URL, GV_LINK_BASE_URL.

Tech stack (from `go.mod`)

Go 1.25 · Fiber v2.52 (+ otelfiber, Jet templates) · Kong CLI · Squirrel · Goose v3 · pgx v5 / lib/pq / go-sqlite3 (+ otelsql) · gocron v2 · RabbitMQ amqp091-go · valkey-go · JWT v5 / WebAuthn / TOTP · go-playground/validator · zap · OpenTelemetry (trace/metric/log + OTLP HTTP) · gobreaker · shopspring/decimal · segmentio/ksuid · SendGrid · svix-webhooks · excelize · testify / go-sqlmock / goleak.

Where to start reading

main.go → http/routes/routes.go (route wiring).
singleton/singleton.go, database/repo/repo.go (shared plumbing).
errors/errors.go, http/entity/error.go (error model).
Auth entrypoints: middleware/auth.go, middleware/client_cookie_auth.go, http/handler/admin_middleware.go, middleware/g2a_auth.go.

Backend Architecture

On this page