Payment system architecture: the parts that matter after launch

The shape of a payments architecture

Think of four layers, each with one job:

1. Orders or intents – what the customer is trying to pay for, in your domain language: an invoice, a cart, a booking.
2. Payments – your record of each attempt to move money, with its own state machine and a reference to the processor’s object.
3. Processor adapter – the only code that talks to Stripe or any other gateway. It sends idempotency keys, maps errors, and translates processor objects into your payment states.
4. Ledger – immutable, double-entry records of money moving between accounts. Balances, revenue and payouts are derived from here.

Two flows feed these layers: synchronous API calls you make, and asynchronous webhooks the processor sends. Reconciliation sits beside all of it, checking that your ledger, the processor’s records and the bank agree.

The most common mistake is collapsing these layers into a single “paid” boolean on the order table. It works until the first partial refund.

Model payments as a state machine

A payment isn’t paid or unpaid. It moves through states such as created, requires action, processing, succeeded, failed, partially refunded, refunded and disputed. Write the allowed transitions down and enforce them in code, so a late “failed” event can’t overwrite a “succeeded” one.

Store the processor’s identifier (a PaymentIntent ID, for example) on your payment record with a unique constraint. That single constraint prevents a whole family of duplicate-record bugs.

Idempotency keys on every write

Networks fail between your request and the processor’s response. Without protection, a retry after a timeout creates a second charge. That’s how double charges happen in otherwise correct code.

Stripe accepts an Idempotency-Key header on POST requests. If a request with the same key arrives again, Stripe returns the stored result of the first one instead of performing the operation twice. Reusing a key with different parameters returns an error, and Stripe may prune keys once they’re at least 24 hours old, so idempotency keys protect retries, not long-term deduplication.

The pattern that works:

1. Create your payment record first, with a key derived from it (a UUID stored on the row works well).
2. Commit that to your database before calling the processor.
3. Call the processor with that key in the Idempotency-Key header.
4. On a timeout or connection error, retry with the same key. Retry 409 and 429 responses with the same key after a backoff. Don’t retry other 4xx responses; record the failure. Treat a 500 as unknown: Stripe replays the stored 500 for that key, so re-fetch the object or wait for the webhook, and never switch to a new key until you know the outcome.
5. Make your own endpoints idempotent too, so a double-clicked Pay button or a client retry hits the same record.

Webhooks as the source of truth for outcomes

Many payment outcomes are only known asynchronously: 3D Secure completes later, bank debits take days, disputes arrive weeks after the charge. Your system has to learn these from webhooks.

Stripe signs every webhook with your endpoint secret; verify the Stripe-Signature header against the raw request body before trusting anything. Delivery is at least once, so the same event can arrive several times, and there’s no ordering guarantee, so a charge.refunded event may arrive before the charge.succeeded one you expected first.

The approach that holds up in production is to store the event, return a 2xx immediately, process from a queue, deduplicate on event ID, and in the handler re-fetch the current object from the API rather than acting on the event payload alone. Then apply the state transition your state machine allows. I cover this end to end in the Stripe webhooks guide, and the broader pattern in event-driven architecture.

A double-entry ledger, not a balance column

As soon as money is held, split or paid out, you need a ledger. A balance column updated in place loses history and can’t be audited. A double-entry ledger records every movement as entries that debit one account and credit another, with each transaction’s entries summing to zero.

Rules I don’t compromise on:

• Entries are append-only. Corrections are new entries that reverse the old ones.
• Amounts are integers in the currency’s minor unit, with the currency stored on every entry. Never floats.
• Every entry references its cause: the payment, refund, payout or dispute that created it.
• Balances are derived by summing entries, and cached only where performance demands it.
• Fees are their own entries, not netted invisibly into the payment amount.

My guide to payment reconciliation goes deeper into ledger database design and how it underpins reconciliation.

Retries, timeouts and failure handling

Failure handling is where payment architecture earns its keep. A few rules that cover most cases:

• Separate unknown from failed. A timeout means you don’t know what happened. Retry with the same idempotency key, or wait for the webhook; don’t tell the customer it failed.
• Retry with exponential backoff and jitter, and cap the attempts. After the cap, move the job to a dead-letter queue that a human reviews.
• Don’t retry declines automatically. A card decline is an answer, not an error.
• Run a sweeper job that finds payments stuck in an intermediate state and re-fetches them from the processor.
• Alert on rates, not individual failures: success rate drops, webhook backlog growth, reconciliation mismatches.

Reconciliation closes the loop

Even with all of the above, something will drift: a webhook endpoint misconfigured for an afternoon, a refund issued from the dashboard, a fee you didn’t model. Daily reconciliation compares your ledger with the processor’s balance transactions and payouts, and payouts with bank deposits, and flags every difference. If you only build one safety net, build this one.

Stripe integration choices that affect architecture

A few Stripe integration decisions shape the rest of the system:

• Hosted Checkout vs Elements. Checkout keeps card data off your servers and handles payment methods and authentication flows for you. Elements gives more control over the UI. Either keeps you out of handling raw card numbers.
• PaymentIntents everywhere. Build around PaymentIntents and SetupIntents rather than the older Charges API; they model authentication steps properly.
• Billing vs your own subscription logic. Use Stripe Billing unless your pricing genuinely doesn’t fit. See SaaS billing.
• Connect for platforms. If you pay out to third parties, Connect changes your ledger, liabilities and onboarding. My marketplace payments guide covers that.

Build order for a new payment system

1. Payment records with a state machine and unique processor IDs.
2. Processor adapter with idempotency keys.
3. Webhook endpoint with signature verification, event storage and a queue.
4. Ledger, even a simple one, from day one.
5. Daily reconciliation job with alerts.
6. Refunds and disputes as first-class flows, not dashboard operations.

If you’d like help building or reviewing this, it’s the core of my payment integration services. I learned most of it building the payment platform at Yola and payment integrations for BLU Loyalty, then again at Indiemaker on Stripe.

Related

• Payment integration services that hold up under real customers
• Stripe webhook handling that survives production
• Payment reconciliation for SaaS and marketplaces
• Marketplace payments: how to design them on Stripe Connect
• Fintech consulting for software that moves money
• Usage based billing and subscriptions, designed to stay correct