OctoWiki

Top-up & eSIM Flows

The two single-unit order resources alongside the voucher engine — recharges and esim_orders. Their sync/hybrid-sync flows, the recreate-budget recovery model, the delivered=PROCESSING nudge, retry crons, and where they diverge from vouchers.

Top-up (mobile recharge) and eSIM are single-unit, non-fan-out resources living alongside the voucher engine. Unlike vouchers (processOrder fans one order into N order_items across vendors), each creates exactly one row (recharges / esim_orders) and talks to exactly one vendor.

Read Order Lifecycle first for the voucher engine; this page documents where top-up/eSIM parallel it and where they diverge. The money mechanics are in Wallet & Ledger; the vendor adapters in Vendor Adapters.

Top-up (mobile recharge)

  • Entry: Handler.CreateTopupOrder (topup.go:421) → services.CreateTopupOrder (topup_order.go:91). Fully synchronous — the vendor call runs inline in the request (contrast eSIM).
  • input_data validation (validateTopupInputFields, topup.go:1104): rejects unknown/extra keys, enforces required + 500-char cap, per-type checks (phone E.164-normalized, number, select against options), optional ValidationRegex with a ReDoS guard (skips patterns >200 chars). ⚠️ It mutates inputData in place to store the normalized phone.
  • Create tx (topup_order.go:191): debit wallet (or claim voucher if prepaid) → CreateRecharge (Type=TOPUP, Status=PENDING, Currency=walletCurrency — must match Amount currency or FX refund routing breaks) → store recharge_input_values. Commit.
  • Vendor submission (inline, 45s): CreateOrder; classify the response — a vendor error with no structured status stays PENDING so the cron polls; a 2xx terminal decline propagates failure fields. Persisted in a second transaction.
  • recharges state machine (enum.go:319): PENDING / DELIVERED / FAILED / CANCELLED / RECHARGED. Terminal = {DELIVERED, FAILED, CANCELLED}. No sub-status column — progress is Status + StatusText + failure fields + RetryCount/RetryAfter.

Dead status values

RECHARGED is defined and validated but never written anywhere in production. And refreshTopupOrderStatusFromVendor guards on a "PROCESSING" string literal (topup.go:711) that is not a RechargeStatus member and never persisted for recharges — a dead branch / copy artifact.

topup_unit (topup_unit.go) is a pure display helper — resolves the in-game currency unit ("Diamonds", "UC") by precedence: admin override → vendor par_value_currency → regex inference from the name, guarding against showing a bare ISO money code as a gaming unit.

eSIM

  • Entry: Handler.CreateEsimOrder (esim.go:28) → services.CreateEsimOrder (esim_order.go:121). Hybrid-sync: quantity forced to 1; the handler passes context.WithoutCancel(ctx) so the dispatch goroutine outlives the request, applies its own esimHandlerDeadline = 60s, and always returns HTTP 200 — clients branch on status, not HTTP code.
  • Phase 1 (sync tx): mirrors top-up, plus denormalizes the product snapshot and sets SubStatus=INITIAL, ActivationStatus=NOT_INSTALLED, RetryAfter = now+90s (so the retry cron can't race the goroutine). ⚠️ ClientDiscount is already a percent here — top-up passes a 0–1 fraction ×100. Easy-to-miss divergence.
  • Phase 2 (detached goroutine): DispatchEsimOrder (esim_order.go:400) — the reusable engine shared by the initial goroutine, the retry cron, and admin retry.
    • First-dispatch fast-path (retry==0 && no vendor_order_id) skips recovery → straight to tryRecreateEsimOrder.
    • Two-phase poll-before-recreate: poll by vendor_order_id, then by merchant_ref (where ErrOrderNotFound is definitive → safe to recreate), then recreate with EsimRecreateBudget = 5 (retry_count bumped before PurchaseEsim so an accept-then-404 loop can't run forever). ErrDuplicateMerchantRef (DT One 1007001) → fetch existing and adopt.
  • persistDispatchOutcome (esim_order.go:732): row-lock (LockEsimOrderForUpdate) + terminal short-circuit — the load-bearing guard against double-refund and status regression. Activation code + ICCID are encrypted before storage. Auto-refund on FAILED runs inside the locked tx — if the credit fails, the whole transition rolls back, so there's never a FAILED order without a refund.

The delivered=PROCESSING airtight rule

The Octopus eSIM webhook adapter maps esim.delivered → EsimStatusProcessing (not terminal). The delivered webhook can't carry the activation code, so treating it as terminal DELIVERED would strand the order without its deliverable. Instead it's a "poll now" nudge — the order stays PENDING and the cron's GetOrderStatus (which does return the code) completes it as DELIVERED/VENDOR_CODE_FETCHED. Pinned by a test asserting "delivered webhook must NOT be terminal." DT One folds SUBMITTED/CONFIRMED → PROCESSING similarly. See Octopus eSIM adapter.

  • Lifecycle: RefreshEsimInstallationStatus (install-poll cron) sets ACTIVE + fires esim.installed/esim.activated once per genuine transition; EsimExpiryJob flips ACTIVE→EXPIRED + fires esim.depleted.
  • State machine (esim_order.go): status PENDING/DELIVERED/FAILED/CANCELLED; sub-status INITIAL / VENDOR_ORDER_PENDING / VENDOR_ORDER_CREATED / VENDOR_CODE_FETCHED / VENDOR_ORDER_FAILED (⚠️ REFUNDED/NOT_REFUNDED/COMPLETED are defined but never assigned); activation status NOT_INSTALLED/ACTIVE/EXPIRED. Delivery artifact is the encrypted activation_code (LPA/QR) + ICCID, reveal-once via MarkEsimOrderRevealed.

Retry crons

Top-up and eSIM each have their own dedicated retry cron — the voucher pending-order-retry is not used by either.

CronScheduleJobDrives
topup-order-retry* * * * *TopupOrderRetryJobProcessTopupRechargeRetry per pending recharge
esim-order-retry* * * * *EsimOrderRetryJobDispatchEsimOrder per pending order
esim-installation-poll*/5 * * * *EsimInstallationPollJobinstall/activate refresh
esim-expiry0 2 * * *EsimExpiryJobACTIVE→EXPIRED + esim.depleted

Backoff: top-up 1<<retryCount min capped at 60; eSIM esimRetryBackoff = 1m/5m/30m/2h/6h.

Budget-vs-cron-max mismatch differs between the two

Top-up cron MaxRetryCount=10 is higher than the recreate budget of 5 — a budget-exhausted, parked recharge (retry_count 5–10) is still swept and re-parked with long backoff. eSIM cron max == budget == 5, so a parked eSIM order at retry_count 5 falls out of the eligibility query entirely (retry_count < 5 is false) and requires admin reset. Different operational behavior for "stuck" orders.

Idempotency

Identical contract on both: optional client client_reference, server-minted KSUID reference_number (internal — vendor merchant_order_id + refund routing). Pre-check + unique partial index on (client_id, client_reference)hard 400 ErrDuplicateClientRef, no silent replay. Vendor-side dedup reuses reference_number as merchant_order_id; ErrDuplicateMerchantRef (SEAGM 20135 / DT One 1007001) is caught and converted to fetch-existing.

Concurrency hazard (top-up webhook)

The top-up webhook path is not concurrency-safe

ProcessDirectTopUpWebhook (vendor_webhook_service.go:282) writes status with no row lock, no terminal short-circuit, and no auto-refund on FAILED — unlike both the eSIM webhook and the top-up cron path (applyTopupVendorStatus), which are hardened. Consequences:

  • A webhook racing the retry cron can overwrite a terminal status (status regression).
  • A webhook-driven FAILED leaves the recharge un-refunded.

The eSIM webhook (ProcessEsimWebhook) is the reference: fully locked, terminal short-circuit, auto-refund inside the lock. Bring the top-up webhook up to parity.

Webhook auth also differs: top-up uses HMAC signature (VerifyWebhookSignature) with a vendor_webhook_logs idempotency key; eSIM uses a URL path token (constant-time, since DT One callbacks have no signature) and relies on the terminal short-circuit inside the lock (its webhook-log insert sets no idempotency key).

Voucher vs top-up vs eSIM

AspectVoucherTop-upeSIM
Fan-outYes (N order_items)No (single unit)No (Quantity→1)
Tableorders + order_itemsrechargesesim_orders
Sync modelengine-drivensynchronous inlinehybrid-sync (goroutine + 60s deadline, always 200)
Sub-statusyesnoneyes + activation status
Delivery artifactvoucher code(s)recharge applied to MSISDNencrypted activation_code (QR) + ICCID, reveal-once
Vendor lineper-item, multi-vendorsingle, direct_topup factorysingle, esim factory
Recoveryprefetch/inventory + per-item retryrecreate-budget 5, 2-phaserecreate-budget 5, 2-phase
Terminal-write lockenginecron yes; webhook NOdispatch and webhook lock + short-circuit
Auto-refund on FAILEDenginecron path only (not webhook)dispatch + webhook, inside lock
Prepaid burnn/aRedeemResourceTopupRedeemResourceEsim
NotificationsOrderNotifiertopup.delivered/failed/cancelledesim.delivered/failed/cancelled + lifecycle installed/activated/depleted

Findings summary

SeverityFindingWhere
HighTop-up webhook: no row-lock / terminal short-circuit / auto-refund → status regression + un-refunded failuresvendor_webhook_service.go:282
LowRechargeStatusRecharged dead (never written)enum.go:319
Low"PROCESSING" literal guard for recharges is a dead branchtopup.go:711
LoweSIM sub-statuses REFUNDED/NOT_REFUNDED/COMPLETED defined but never assignedesim_order.go:45
LowvalidateTopupInputFields mutates caller's inputData map in placetopup.go:1161
LowTwo-transaction window in top-up create (PENDING commit, then vendor status in tx2) — crash-safe by designtopup_order.go:386
LoweSIM esimHandlerDeadline (60s) < esimDispatchTimeout (90s) relies on WithoutCancel wiringesim.go:24, esim_order.go:33

Key files

  • Services: services/{topup_order,esim_order}.go, services/vendor_webhook_service.go, services/{recharge_notifier,esim_notifier}.go
  • Handlers: http/handler/{topup,topup_unit,esim,esim_webhook,direct_topup_webhook}.go
  • Models: database/models/{recharge,esim_order,enum}.go
  • Jobs/scheduler: jobs/{topup_order_retry,esim_order_retry,esim_installation_poll,esim_expiry}_job.go + matching scheduler/*_task.go
  • Vendors: services/external_vendors/{direct_topup,esim}/{factory,contract}.go

On this page