Idempotency: The API Property Most Backend Devs Forget Until It's Too Late

The Double-Charge That Wasn't a Bug

A customer contacts support: they were charged twice for the same order. You investigate. The order was created once. The payment API was called once from your backend. But the payment provider shows two successful charges.

What happened: your backend called the payment provider, the provider processed the payment, but the network response was lost before it reached your server. Your service timed out and retried. The provider received a second request — a new request, from its perspective, with no way to know it was a retry — and processed it again. Two charges, one order, no bug in your code, and a very angry customer.

This is the canonical idempotency failure. The solution is not "don't retry" — retries are essential for reliable systems. The solution is idempotency keys.

What Idempotency Means

An operation is idempotent if executing it multiple times produces the same result as executing it once. GET, HEAD, PUT, and DELETE are idempotent by HTTP specification. POST is not — each POST to /payments is intended to create a new payment.

An idempotency key is a client-generated identifier that the server uses to recognize duplicate requests. On the first request with a given key, the server processes the operation and records the result associated with that key. On subsequent requests with the same key, the server returns the previously recorded result without executing the operation again.

POST /payments
Idempotency-Key: order-8472-attempt-1
Content-Type: application/json

{ "amount": 2999, "currency": "USD", "source": "card_xxx" }

If the response to this request is lost and the client retries with the same Idempotency-Key, the server returns the original response — the payment was charged once.

Implementing Idempotency Keys on the Server

The implementation requires a persistent store keyed on the idempotency key with a TTL long enough to cover your retry window:

public PaymentResult processPayment(PaymentRequest request, String idempotencyKey) {
    // Check for existing result
    Optional<PaymentResult> cached = idempotencyStore.get(idempotencyKey);
    if (cached.isPresent()) {
        return cached.get(); // Return previous result, do not reprocess
    }

    // Acquire a lock to prevent concurrent duplicate processing
    // (two retries arriving simultaneously)
    try (Lock lock = lockService.acquire("payment:" + idempotencyKey)) {
        // Double-check after acquiring lock
        cached = idempotencyStore.get(idempotencyKey);
        if (cached.isPresent()) {
            return cached.get();
        }

        PaymentResult result = paymentGateway.charge(request);
        idempotencyStore.set(idempotencyKey, result, Duration.ofHours(24));
        return result;
    }
}

The distributed lock between the first check and the processing step is necessary to handle concurrent duplicate requests — two retries arriving at the same time, both missing the cache on first check.

Generating Idempotency Keys on the Client

Idempotency keys must be generated by the client and must uniquely identify the operation intent — not just the request. A random UUID per request attempt is wrong: each retry generates a new key, defeating the purpose.

The key should be generated once for the logical operation and reused across all retry attempts:

// Generate the key when the user initiates the action
String idempotencyKey = UUID.randomUUID().toString();
// Store this key with the pending order, retry with the same key
orderService.initiatePayment(order, idempotencyKey);

If the user navigates away and comes back, you retrieve the stored key and continue with it. If you generate a new key on page reload, you risk a new charge.

Which Operations Need Idempotency Keys

Any operation that:

• Has real-world side effects that should not repeat (charges, transfers, sends)
• Can be retried by either your code or an upstream client
• Would cause user-visible harm if executed twice

Payments and financial transactions are the clearest case. Also applies to: sending notifications (duplicate emails or SMS), provisioning resources (don't create two databases for one request), creating records where uniqueness is expected by the user.

Read operations (GET) are inherently idempotent — no key needed. Pure data updates where "apply this state" is idempotent by nature (PUT semantics) don't require keys either, though they need thoughtful design to truly be idempotent.

Stripe as the Reference Implementation

Stripe's idempotency key implementation is the industry reference. Keys are accepted on all POST requests. They're stored for 24 hours. Concurrent requests with the same key are serialized. The stored response includes the HTTP status code, so if the first request resulted in an error, retrying returns the same error rather than reprocessing.

If you're building a payments or financial API, read Stripe's idempotency documentation before designing your own. The edge cases they've handled are non-obvious.

The Practical Takeaway

Audit every POST endpoint in your API that creates or triggers a real-world operation. For each one: does the client retry on timeout? If yes, is the server idempotent? If not, add idempotency key support before the first production retry causes a duplicate. For payment operations, this is not optional.