Inventory Flow
How pre-loaded voucher codes move DB → Valkey pools → orders. The exact pool key formats, the ZSET FEFO (first-expiry) allocation, the two-phase reserved/general algorithm, prefetch, import, tagging, and the ALLOCATED leak.
Voucher inventory is pre-loaded gift-card codes. They flow DB inventories → Valkey sorted-set pools → orders, allocated soonest-expiry-first. This is the mechanism behind order fulfilment and ties directly to the inventory leak on cancel/reset.
It's FEFO, not FIFO
Allocation is First-Expiry-First-Out — each pool is a Valkey ZSET scored on expires_at (unix), and allocation reads ascending score, so the soonest-expiring voucher is consumed first. It is not insertion-order FIFO and not ksuid-ordered (ties break on Valkey member lexical order). Because the pump also inserts sorted by expires_at ASC, FEFO and FIFO coincide when all items share an expiry.
The inventories table & lifecycle
Model database/models/inventory.go. Secrets (code, pin, claim_url, voucher_reference_number, checksum) are AES-256-GCM ciphertext; plaintext SHA3-256 hashes (*_hashed) sit alongside for lookup/dedup. Linkage: client_id (reserved-to-client tag), order_item_id (allocation link), vendor_id, import_id, pre_fetched, expires_at, deleted_at.
Status enum (enum.go:277): AVAILABLE, ALLOCATED, DELIVERED, COMPLETED, BLOCKED, REVOKED.
Only 2 of the 6 statuses are ever written
AVAILABLE (import/prefetch insert) and ALLOCATED (allocation) are the only statuses any code writes. DELIVERED/COMPLETED/BLOCKED/REVOKED are defined but never assigned — and the generic UpdateInventory status setter (repo/inventory.go:145) has zero callers. There is no lifecycle past ALLOCATED; delivery state lives on the order item, not the inventory row.
The pump: DB → Valkey pools
PumpInventoryToValkey (services/inventory.go:66) rebuilds the pools from the DB every 6h (cron inventory-pump, 0 */6 * * *, ClearCache=true — clear-then-repump). It is a periodically-rebuilt projection, not an incremental sync.
Per AVAILABLE row (expires_at >= now+30d, vendor_id NOT NULL — vendorless stock never enters cache):
ZADD poolKey score=expires_at member="inv-{id}"— the pool is a sorted set keyed on expiry.HSet "inventory:inv-{id}"— metadata hash (vendor/product/denom/expiry/original_id + client_id if reserved).
Exact pool key formats
| Pool | Key format | When |
|---|---|---|
| Reserved | inventory:reserved:{clientID}:{productID}:{denom%.2f}:{vendorID} | row has client_id > 0 |
| General | pool:{productID}:{denom%.2f}:{vendorID} | untagged |
| Metadata | inventory:inv-{id} | every row |
(generateValkeyPoolKey, inventory.go:319; denomination always %.2f.) The ValkeyCache prepends the global keyPrefix, so on-wire keys are {prefix}pool:... — this is how sandbox is isolated on the shared Valkey.
Re-pump triggers beyond the cron: after tag/untag (refreshInventoryCache), prefetch completion, and import completion.
Two-phase allocation (the FEFO claim)
AllocateInventoryFromCache (services/inventory.go:731) runs two phases via allocateFromPools:
- Phase 1 (reserved) — only if the order has a
ClientID: draws from that client's reserved pool first. - Phase 2 (general) — always: falls back to the shared pool.
Each phase iterates the client's allowed vendor IDs (blacklist-filtered), and for each pool: ZRANGEBYSCORE(now, now+365d, LIMIT remaining) grabs the soonest-expiring unexpired candidates, then per candidate ZREM to atomically claim it — the winner gets removed==1, race losers get 0 and skip to the next.
Atomicity is ZREM-return-value only
There is no Lua script, no WATCH/MULTI, no pipeline. ZRANGEBYSCORE (read) and ZREM (claim) are separate round-trips, so two allocators can read the same candidate — only one's ZREM returns 1. Correctness depends on (a) ZREM atomicity and (b) the DB row being flipped to ALLOCATED before the next pump (which only pumps AVAILABLE). The order-item→inventory mapping is positional by index (create_voucher_order.go:1751).
Multi-vendor re-vendoring: an item costed against vendor A can be filled from vendor B's pool if A is depleted (all allowed-vendor pools are iterated). The DB row keeps its original vendor_id — only order_item_id+status are written — so re-vendoring is invisible in inventories.vendor_id but auditable via the InwardLog rows, which record the true fulfilling vendor. See Order Lifecycle → inventory allocation.
DB write to ALLOCATED
After the pop, UpdateOrderItemsWithInventory fetches the encrypted secrets, copies them onto the order items, writes InwardLog audit rows, and runs BulkUpdateInventoryOrderItemID — a single UPDATE inventories SET order_item_id=…, status='ALLOCATED' — which is the DB write marking the popped rows allocated.
Prefetch
Prefetch proactively buys vouchers from a vendor API and loads them into inventory + Valkey so future orders serve from cache. InitiatePrefetch (prefetch_service.go:122) creates a PrefetchJob (batch 100, max 5 concurrent, 3 retries); SelectBestVendor scores vendor_products by discount and live-checks availability; the processor orders vouchers, stores them as pre_fetched=true AVAILABLE rows, then pumps them (reserved-pool if a ClientID is set). Admin: create/kill/list under /admin/prefetch/*, driven by the prefetch-processor cron (every 2 min). See Cron Catalog.
Import
Bulk-load pre-purchased codes from Excel (services/inventory_import.go). Template columns include Tagged Client Name (blank = general pool; set = reserved). The pipeline stages through a per-import temp table: insert → enrich FKs by name → validate (required fields, expiry, duplicate codes via checksum) → encrypt + hash → BulkInsertInventoryFromTempInventoryImportTable (INSERT … 'AVAILABLE' … reference_code='IMP-{uuid}-{row}', expires_at defaults to now+1yr if blank).
Crypto: EncryptVoucherData = AES-256-GCM with key SHA-256(APP_KEY) and a random prepended nonce; hashes are SHA3-256; checksum = SHA3-256("code|pin|link|voucherRef") used for dedup. On delivery, GenerateVouchers decrypts (tolerant of legacy plaintext). See Secrets & Config.
Tags
Tagging reserves AVAILABLE stock to a client by setting client_id (never touches status):
TagInventoryToClient— selectsAVAILABLE AND client_id IS NULLfor (product, denom),ORDER BY expires_at ASC LIMIT qty(reserves soonest-expiring), setsclient_id.UntagInventoryFromClient— reverse,ORDER BY expires_at DESC LIMIT qty, setsclient_id=NULL.
A tagged row moves from the general pool to the client's reserved pool on the next (targeted clear+repump) pump. Purpose: carve out stock so a client's orders draw from dedicated inventory (Phase 1) before the shared pool.
Availability queries
The canonical count (GetAvailableInventoryCount, repo/inventory.go:1461) is status='AVAILABLE' AND client_id IS NULL AND deleted_at IS NULL (minus blacklisted vendors) — surfaced in the admin inventory dashboard.
Availability counts read the DB, not the pool. Pool occupancy (ZCARD per pool, GetValkeyPoolStats) is a separate metric that diverges from DB "available" between pumps. Don't treat the dashboard count as pool-backed sellability.
Consistency, staleness & the leak
The ALLOCATED → AVAILABLE leak (confirmed)
No code path ever releases an ALLOCATED row. The only status writers set AVAILABLE (create/import) or ALLOCATED (allocate) — nothing writes AVAILABLE back. Admin cancel/refund/reset update the order, items, and wallet but never touch inventories (admin_ui.go:2668). So a cancelled order's allocated codes are double-lost: the DB row is stuck ALLOCATED forever, and the pool member was already ZREM'd, so the pump (AVAILABLE-only) never re-adds it. A fix needs UPDATE inventories SET status='AVAILABLE', order_item_id=NULL WHERE order_item_id IN (…) plus a re-pump/ZADD — neither exists. See Admin Order Actions.
Other consistency hazards:
| Hazard | Detail |
|---|---|
| >1yr stock stranded | Pump admits any expires_at >= now+30d (no upper bound), but allocation's ZRANGEBYSCORE max = now+365d. Vouchers expiring more than a year out are pumped but never selectable. |
| No DB fallback on cache miss | Empty/missing pool → allocation returns nil → treated as insufficient inventory. DB-available stock is invisible to allocation if the pump hasn't run. |
| Partial cache clear | The pump job's clearProductCache only clears a hardcoded denomination list [10,25,50,100,250,500,1000] — pools for other denominations aren't cleared before repump (mostly self-healing since ZADD updates score, but retired-denomination pools are never purged). |
| Staleness window | Up to 6h between pumps; newly tagged/imported stock isn't in the pool until a targeted re-pump fires. |
| Double-allocation guard | Rests solely on ZREM atomicity + flip-to-ALLOCATED before next pump; no DB row lock or unique constraint on order_item_id. |
Cheat sheet
| Thing | Value |
|---|---|
| Reserved key | inventory:reserved:{client}:{product}:{denom%.2f}:{vendor} |
| General key | pool:{product}:{denom%.2f}:{vendor} |
| Member | inv-{id} |
| Structure | Sorted Set (ZSET), score = expires_at unix |
| Order basis | earliest-expiry (FEFO); no LPOP, no ksuid |
| Claim | ZRANGEBYSCORE(now, now+365d) then per-member ZREM |
| Atomicity | ZREM return value only — no Lua/MULTI/WATCH |
| Pump filter | AVAILABLE, expires_at >= now+30d, vendor_id NOT NULL |
| Statuses written | only AVAILABLE + ALLOCATED |
| Leak | cancel/refund never release ALLOCATED |
| Encryption | AES-256-GCM; SHA3-256 hashes + checksum |
Key files
- Service:
services/inventory.go(pump:66, allocate:731, key gen:319),services/inventory_import.go,services/prefetch_service.go - Repo:
database/repo/inventory.go,database/repo/inventory_import.go - Cache:
cache/cache_valkey.go(ZSET ops) - Fulfilment:
http/handler/create_voucher_order.go:1648 - Admin/cron/jobs:
http/handler/admin_inventory.go,scheduler/inventory_pump_task.go,jobs/{inventory_pump_job,prefetch_processor_job,prefetch_state_machine}.go
State Machines
Every order/resource state machine — voucher orders (order-level + item checkpoints + voucher links), recharge (mobile & gaming), eSIM (status/sub-status/activation), and payouts — with enum tables, transition tables, diagrams, and the dead-state catalog.
Wallet & Ledger
The client money layer every order and payout debits — wallets, the single-entry ledger, the atomic debit stored procedure, the refund paths, charges/FX, and prepaid voucher-burning. Includes precision and double-refund hazards.