The $10M currency-conversion gap
A rounding-and-timing gap in cross-border dispute settlement quietly lost money on a subset of cases. Finding it meant reverse-engineering a 95-state dispute lifecycle; fixing it meant a change that spanned 30+ services and four organizations.
Context
When you dispute a card charge across a border, money doesn't just move once. It moves through a chargeback lifecycle — provisional credit, first chargeback, merchant response, second chargeback, pre-arbitration, representment — and at several of those steps the bank debits or credits internal ledgers. When the transaction settled in one currency but the customer was billed in another, a foreign-transaction fee and a currency conversion had to be applied at exactly the right steps.
The platform had been built for a market that doesn't charge a foreign-transaction fee, so that logic simply didn't exist. Ported to a market that does, the gap meant the bank was eating conversion losses on a slice of every cross-border dispute — money that was nobody's job to notice.
How it surfaced
It started as a small bug I picked up as a fresh grad. Fixing it revealed the bug was a symptom: the same missing concept was absent across 10–20 services. Prior attempts to tackle the root cause had been shelved as too hard — the knowledge was spread across applications and external dependencies (a card network, an external mainframe settlement partner, the fraud/dispute case-status system, the accounting ledgers, and the ledger-execution engine), and no single person held the whole map.
Step 1 — recover the state machine
You can't fix a lifecycle you can't see. There were 95 distinct case states and transitions an agent could walk a dispute through, and the "spec" was really just behavior spread across services and millions of historical cases (2–3M in one market alone). So I wrote a program that crawled the case data, inferred every state-to-state relationship, and auto-generated a finite-state machine of the actual business logic — turning tribal knowledge into a diagram everyone could argue about.
Step 2 — design the fix, get four teams to yes
The change had to propagate FX-fee and conversion fields through the entire lifecycle and correct how the ledger-execution engine moved money at each step. That touched 30+ services owned by different teams. With no formal mandate, I effectively wore the product-owner hat: defining intent, laying out the pros and cons of each path when there was no obvious answer, and getting explicit buy-in from four organizations before writing the migration.
- Accounting / ledgers — reconcile to the cent; they had been eating write-offs manually.
- The card network — align on representment rules so a fix didn't create a dispute-risk hole.
- Case-status team — propagate the new fields through their state model.
- Ledger-execution team — the service that actually debits/credits customer ledgers.
Requirements that shaped it
- Apply FX + conversion at the correct lifecycle steps only
- Handle every one of the 95 states, including in-flight cases
- Rebill customers to the exact amount they paid
- Correctness: money must reconcile — this is a ledger
- Auditability: every adjustment traceable (later, SR&ED-grade)
- Backward-compatible: no breakage for open disputes
Testing
A ledger bug doesn't get a second chance in production, so I was obsessive about coverage — 90+ end-to-end scenarios across the state machine, run in dev and again in production verification, with stakeholders kept in the loop at every gate. The release went out clean.
Results
It earned company-wide recognition and a promotion to Senior Software Engineer. The research narrative also qualified for a $300K+ SR&ED tax incentive.
What I'd do differently
- Ask for help earlier. I took on too much alone at the start. The state-machine extraction and the stakeholder alignment were two full-time jobs; splitting them sooner would have shaved months.
- Make the FSM a shared service, not a one-off script. The generated state machine was gold — it should have lived as a queryable source of truth other teams could build on, not a tool I ran.
- Continuous reconciliation monitoring. The whole class of bug is invisible without a job that watches for money that doesn't reconcile. That alarm should exist by default.
Scaling it 10× / 100×
At 10× volume the FSM-driven decisioning holds — it's stateless per case and parallelizes across the dispute stream. The pressure points are the external partners (the card network and settlement mainframe have their own throughput and batch windows) and the ledger-execution path, which must stay strongly consistent. At 100× I'd move conversion decisioning behind an event log for replay and audit, batch ledger adjustments idempotently, and treat the reconciliation monitor as a first-class SLO rather than an afterthought.
Written to be public-safe: internal project names, service codenames, colleagues, and partners are generalized to their technical essence. Nothing confidential here — just the engineering.