Cron Catalog
Every scheduled task — the actual cron string (source of truth), whether it's registered, what it drives, batch/timeout, and the two implemented-but-unregistered payout crons. Plus the stale-comment schedule traps.
Every gocron task, its actual schedule, and whether it runs. The scheduler wraps a custom scheduler.ScheduledTask interface; the schedule string passed to NewBaseTask(id, schedule, group) is what TimePattern() returns — so the 2nd arg of each NewBaseTask(...) is the source of truth, not the inline comment (several disagree).
Two things that bite
- Two payout crons are implemented but never registered —
payout-processorandpayout-status-sync— which is why the payout pipeline is dead. 2. Three schedule comments are stale (vendor-catalog-retry, vendor-balance-sync, pending-order-retry) — trust the cron string.
How it runs
- Framework: gocron v2 wrapping
scheduler.ScheduledTask(scheduler/scheduler.go). Tasks embed*BaseTask. - Registration:
baseTasksassembled inmain.go:170-242, gated by feature flag, thenWrapTasksWithTracking. - Single process: under
run-all, workers + crons + HTTP server all run in one OS process (main.go:284). See Deploy & Release. - Overlap protection:
gocron.WithSingletonMode(LimitModeReschedule)— a run is skipped/rescheduled if the previous run of the same task is still in flight (per-task, not fleet-wide). - Lifecycle:
PreExecute → Execute → HandleFailure(on error) → PostExecute+ Prometheus cron metrics. - Tracking: every task is wrapped by
TrackedTaskWithCallbackwhich writes ajob_executionsrow (items processed/succeeded/failed), allrecover()-guarded so tracking never breaks a job. Surfaced in the admin Job Executions UI (admin_job_executions.go); per-run log files underJOB_LOGS_DIR(default/var/log/octopus/job_executions). See Jobs & Observability.
gocron.CronJob(schedule, true) passes withSeconds=true for 5-field patterns (main.go:738) — it evidently parses 5-field fine (jobs schedule), but flagged for the handover. Feature-flag gating: a disabled feature's tasks are never registered.
The catalog
Schedule column = the actual TimePattern(). "Reg?" = registered in main.go. imp = ImportanceLevel.
Orders (vouchers)
| Task | Schedule | Reg? | What it does | Batch/timeout | file |
|---|---|---|---|---|---|
pending-order-retry | * * * * * | ✅ vouchers | Retries PENDING voucher orders past retry_after → PendingOrderRetryJob | Batch 50; MaxRetryCount 10 (comment says 5) | pending_order_retry_task.go:39 |
Top-up
| Task | Schedule | Reg? | What it does | Batch | file |
|---|---|---|---|---|---|
topup-order-retry | * * * * * | ✅ topups | Polls vendor status for pending recharges → TopupOrderRetryJob | 50 | topup_order_retry_task.go:39 |
eSIM
| Task | Schedule | Reg? | What it does | Batch | file |
|---|---|---|---|---|---|
esim-installation-poll | */5 * * * * | ✅ esim | Polls install/activation for DELIVERED-not-installed orders | 200 | esim_installation_poll_task.go:28 |
esim-order-retry | * * * * * | ✅ esim | Re-dispatches PENDING eSIM orders (transient errors, crash recovery, async) | 50 | esim_order_retry_task.go:28 |
esim-expiry | 0 2 * * * | ✅ esim | Flips elapsed-validity orders to activation_status=EXPIRED | 500 | esim_expiry_task.go:25 |
Vendor / catalog
| Task | Schedule | Reg? | What it does | Batch | file |
|---|---|---|---|---|---|
vendor-catalog-sync | 0 2 * * * | ✅ vouchers | Syncs vendor catalogs; skips if already synced today | — | vendor_catalog_task.go:33 |
vendor-catalog-cleanup | 0 3 * * 0 | ✅ vouchers | Deletes catalog snapshots > 30 days | — | vendor_catalog_task.go:181 |
vendor-catalog-retry | * * * * * ⚠️ | ✅ vouchers | Retries failed catalog snapshots | 10/run | vendor_catalog_retry_task.go:43 |
vendor-balance-sync | */15 * * * * ⚠️ | ✅ vouchers‖topups‖esim | Fetches + updates vendor wallet balances | 5 concurrent, 30s/vendor | vendor_balance_sync_task.go:41 |
Shopify
| Task | Schedule | Reg? | What it does | Batch | file |
|---|---|---|---|---|---|
fetch-shopify-orders | * * * * * | ✅ shopify | Pulls new Shopify orders (10-min lookback) → voucher orders | 60s timeout | fetch_shopify_orders_task.go:44 |
process-pending-shopify-orders | * * * * * | ✅ shopify | Fulfils pending Shopify orders; refund-cancels on non-recoverable failure | 50, 60s/order | process_pending_shopify_orders_task.go:44 |
detect-shopify-cancellations | */5 * * * * | ✅ shopify | Reflects Shopify cancellations (delivered vouchers NOT revoked) | 24h fallback | detect_shopify_cancellations_task.go:40 |
Payouts
| Task | Schedule | Reg? | What it does | Batch | file |
|---|---|---|---|---|---|
payout-webhook-delivery | */1 * * * * | ✅ payouts | Sends pending payout webhooks | 50 | payout_webhook_task.go:40 |
scheduled-payout-processor | */1 * * * * | ✅ payouts | Processes due scheduled payouts (FOR UPDATE SKIP LOCKED) | 20, 30s | scheduled_payout_processor_task.go:41 |
payout-processor | */1 * * * * | ❌ NOT REGISTERED | Would process queued payouts → provider | 10, 30s | payout_processor_task.go:41 |
payout-status-sync | */5 * * * * | ❌ NOT REGISTERED | Would sync non-terminal payout statuses with providers | 20, 15s | payout_processor_task.go:175 |
Webhooks
| Task | Schedule | Reg? | What it does | Batch | file |
|---|---|---|---|---|---|
webhook-delivery | */1 * * * * | ✅ always | Sends all pending webhook deliveries (orders, wallets, …) | 50 | webhook_task.go:40 |
Inventory / prefetch / import-export
| Task | Schedule | Reg? | What it does | Batch | file |
|---|---|---|---|---|---|
inventory-pump | 0 */6 * * * | ✅ vouchers | Pumps voucher inventory into the Valkey pools | 1000 | inventory_pump_task.go:32 |
prefetch-processor | */2 * * * * | ✅ vouchers | Processes pending prefetch jobs; exponential-backoff retry | 10, 60s/job | prefetch_processor_task.go:41 |
import-processor | * * * * * | ✅ vouchers | Processes product/inventory import jobs | — | import_processor_task.go:48 |
import-cleanup | 0 0 * * * | ✅ vouchers | Cleans old import jobs | — | import_processor_task.go:125 |
export-processor | * * * * * | ✅ always | Processes export jobs | 10 | export_tasks.go:35 |
export-cleanup | 0 2 * * * | ✅ always | Cleans old exports | 50 | export_tasks.go:102 |
G2A / maintenance
| Task | Schedule | Reg? | What it does | Batch | file |
|---|---|---|---|---|---|
g2a-reservation-expiry | @every 5m | ✅ vouchers | Flips RESERVED reservations past TTL to EXPIRED (frees no stock) | single UPDATE | g2a_reservation_expiry_task.go:23 |
job-execution-cleanup | 0 3 * * * | ✅ always | Summarizes then purges old job_executions (30-day retention) | 1000/delete | job_execution_cleanup_task.go:43 |
26 tasks total: 24 registered, 2 not. Import/export files (export_tasks.go, import_processor_task.go) each define two tasks.
Flagged issues
Implemented but not registered — dead crons
PayoutProcessorTask and PayoutStatusSyncTask constructors exist in scheduler/payout_processor_task.go but are never referenced in main.go (the payouts block registers only the webhook + scheduled-payout tasks). Their jobs never run on a schedule → payout state reconciliation and provider submission simply don't happen. This is the mechanism behind the payout pipeline being dead and the payout state machine.
Stale schedule comments — trust the cron string
| Task | Comment says | Actually runs |
|---|---|---|
vendor-catalog-retry | every 5 minutes | every minute (* * * * *) |
vendor-balance-sync | every 3 hours | every 15 minutes (*/15 * * * *) |
pending-order-retry | Max 5 retry attempts | MaxRetryCount 10 |
- No task-layer auto-retry or paging alert — most
HandleFailureoverrides only log (placeholder "could add alerting"). Retry/backoff lives inside the jobs (retry_after,MaxRetryCount, prefetchSchedulePrefetchRetry). Alerts are a separate query overjob_executionsin the admin API.
Key files
- Scheduler core:
scheduler/scheduler.go,scheduler/tracked_task.go; startupmain.go:284,628 - Tasks:
scheduler/*_task.go; jobs:jobs/*_job.go - Registration:
main.go:170-242 - Execution logging:
jobs/execution_log_manager.go,services/execution_log_service.go; adminhttp/handler/admin_job_executions.go
Jobs, Queue, Cache & Observability
The cron scheduler and full job inventory, the (dormant) RabbitMQ queue, the Valkey cache strategy, and the OpenTelemetry → SigNoz pipeline.
Vendors & Integrations
The vendor adapter architecture, the full adapter inventory, order orchestration, inbound sales channels (Shopify, G2A), and OCTO federation.