OctoWiki

Cron Catalog

Every scheduled task — the actual cron string (source of truth), whether it's registered, what it drives, batch/timeout, and the two implemented-but-unregistered payout crons. Plus the stale-comment schedule traps.

Every gocron task, its actual schedule, and whether it runs. The scheduler wraps a custom scheduler.ScheduledTask interface; the schedule string passed to NewBaseTask(id, schedule, group) is what TimePattern() returns — so the 2nd arg of each NewBaseTask(...) is the source of truth, not the inline comment (several disagree).

Two things that bite

  1. Two payout crons are implemented but never registeredpayout-processor and payout-status-sync — which is why the payout pipeline is dead. 2. Three schedule comments are stale (vendor-catalog-retry, vendor-balance-sync, pending-order-retry) — trust the cron string.

How it runs

  • Framework: gocron v2 wrapping scheduler.ScheduledTask (scheduler/scheduler.go). Tasks embed *BaseTask.
  • Registration: baseTasks assembled in main.go:170-242, gated by feature flag, then WrapTasksWithTracking.
  • Single process: under run-all, workers + crons + HTTP server all run in one OS process (main.go:284). See Deploy & Release.
  • Overlap protection: gocron.WithSingletonMode(LimitModeReschedule) — a run is skipped/rescheduled if the previous run of the same task is still in flight (per-task, not fleet-wide).
  • Lifecycle: PreExecute → Execute → HandleFailure(on error) → PostExecute + Prometheus cron metrics.
  • Tracking: every task is wrapped by TrackedTaskWithCallback which writes a job_executions row (items processed/succeeded/failed), all recover()-guarded so tracking never breaks a job. Surfaced in the admin Job Executions UI (admin_job_executions.go); per-run log files under JOB_LOGS_DIR (default /var/log/octopus/job_executions). See Jobs & Observability.

gocron.CronJob(schedule, true) passes withSeconds=true for 5-field patterns (main.go:738) — it evidently parses 5-field fine (jobs schedule), but flagged for the handover. Feature-flag gating: a disabled feature's tasks are never registered.

The catalog

Schedule column = the actual TimePattern(). "Reg?" = registered in main.go. imp = ImportanceLevel.

Orders (vouchers)

TaskScheduleReg?What it doesBatch/timeoutfile
pending-order-retry* * * * *✅ vouchersRetries PENDING voucher orders past retry_afterPendingOrderRetryJobBatch 50; MaxRetryCount 10 (comment says 5)pending_order_retry_task.go:39

Top-up

TaskScheduleReg?What it doesBatchfile
topup-order-retry* * * * *✅ topupsPolls vendor status for pending recharges → TopupOrderRetryJob50topup_order_retry_task.go:39

eSIM

TaskScheduleReg?What it doesBatchfile
esim-installation-poll*/5 * * * *✅ esimPolls install/activation for DELIVERED-not-installed orders200esim_installation_poll_task.go:28
esim-order-retry* * * * *✅ esimRe-dispatches PENDING eSIM orders (transient errors, crash recovery, async)50esim_order_retry_task.go:28
esim-expiry0 2 * * *✅ esimFlips elapsed-validity orders to activation_status=EXPIRED500esim_expiry_task.go:25

Vendor / catalog

TaskScheduleReg?What it doesBatchfile
vendor-catalog-sync0 2 * * *✅ vouchersSyncs vendor catalogs; skips if already synced todayvendor_catalog_task.go:33
vendor-catalog-cleanup0 3 * * 0✅ vouchersDeletes catalog snapshots > 30 daysvendor_catalog_task.go:181
vendor-catalog-retry* * * * * ⚠️✅ vouchersRetries failed catalog snapshots10/runvendor_catalog_retry_task.go:43
vendor-balance-sync*/15 * * * * ⚠️✅ vouchers‖topups‖esimFetches + updates vendor wallet balances5 concurrent, 30s/vendorvendor_balance_sync_task.go:41

Shopify

TaskScheduleReg?What it doesBatchfile
fetch-shopify-orders* * * * *✅ shopifyPulls new Shopify orders (10-min lookback) → voucher orders60s timeoutfetch_shopify_orders_task.go:44
process-pending-shopify-orders* * * * *✅ shopifyFulfils pending Shopify orders; refund-cancels on non-recoverable failure50, 60s/orderprocess_pending_shopify_orders_task.go:44
detect-shopify-cancellations*/5 * * * *✅ shopifyReflects Shopify cancellations (delivered vouchers NOT revoked)24h fallbackdetect_shopify_cancellations_task.go:40

Payouts

TaskScheduleReg?What it doesBatchfile
payout-webhook-delivery*/1 * * * *✅ payoutsSends pending payout webhooks50payout_webhook_task.go:40
scheduled-payout-processor*/1 * * * *✅ payoutsProcesses due scheduled payouts (FOR UPDATE SKIP LOCKED)20, 30sscheduled_payout_processor_task.go:41
payout-processor*/1 * * * *NOT REGISTEREDWould process queued payouts → provider10, 30spayout_processor_task.go:41
payout-status-sync*/5 * * * *NOT REGISTEREDWould sync non-terminal payout statuses with providers20, 15spayout_processor_task.go:175

Webhooks

TaskScheduleReg?What it doesBatchfile
webhook-delivery*/1 * * * *✅ alwaysSends all pending webhook deliveries (orders, wallets, …)50webhook_task.go:40

Inventory / prefetch / import-export

TaskScheduleReg?What it doesBatchfile
inventory-pump0 */6 * * *✅ vouchersPumps voucher inventory into the Valkey pools1000inventory_pump_task.go:32
prefetch-processor*/2 * * * *✅ vouchersProcesses pending prefetch jobs; exponential-backoff retry10, 60s/jobprefetch_processor_task.go:41
import-processor* * * * *✅ vouchersProcesses product/inventory import jobsimport_processor_task.go:48
import-cleanup0 0 * * *✅ vouchersCleans old import jobsimport_processor_task.go:125
export-processor* * * * *✅ alwaysProcesses export jobs10export_tasks.go:35
export-cleanup0 2 * * *✅ alwaysCleans old exports50export_tasks.go:102

G2A / maintenance

TaskScheduleReg?What it doesBatchfile
g2a-reservation-expiry@every 5m✅ vouchersFlips RESERVED reservations past TTL to EXPIRED (frees no stock)single UPDATEg2a_reservation_expiry_task.go:23
job-execution-cleanup0 3 * * *✅ alwaysSummarizes then purges old job_executions (30-day retention)1000/deletejob_execution_cleanup_task.go:43

26 tasks total: 24 registered, 2 not. Import/export files (export_tasks.go, import_processor_task.go) each define two tasks.

Flagged issues

Implemented but not registered — dead crons

PayoutProcessorTask and PayoutStatusSyncTask constructors exist in scheduler/payout_processor_task.go but are never referenced in main.go (the payouts block registers only the webhook + scheduled-payout tasks). Their jobs never run on a schedule → payout state reconciliation and provider submission simply don't happen. This is the mechanism behind the payout pipeline being dead and the payout state machine.

Stale schedule comments — trust the cron string

TaskComment saysActually runs
vendor-catalog-retryevery 5 minutesevery minute (* * * * *)
vendor-balance-syncevery 3 hoursevery 15 minutes (*/15 * * * *)
pending-order-retryMax 5 retry attemptsMaxRetryCount 10
  • No task-layer auto-retry or paging alert — most HandleFailure overrides only log (placeholder "could add alerting"). Retry/backoff lives inside the jobs (retry_after, MaxRetryCount, prefetch SchedulePrefetchRetry). Alerts are a separate query over job_executions in the admin API.

Key files

  • Scheduler core: scheduler/scheduler.go, scheduler/tracked_task.go; startup main.go:284,628
  • Tasks: scheduler/*_task.go; jobs: jobs/*_job.go
  • Registration: main.go:170-242
  • Execution logging: jobs/execution_log_manager.go, services/execution_log_service.go; admin http/handler/admin_job_executions.go

On this page