Idempotency in REST APIs: Prevent Duplicate Requests

Imagine a user clicks the Place Order button twice because the page freezes. Or a payment request times out, and the client retries it.
Without proper API design, you could end up with two orders, duplicate payments, or the same action being processed multiple times.
This is where Idempotency comes in. It helps you design APIs that safely handle retries without repeating unintended side effects.

---

🔹 What Is Idempotency?​

Idempotency means that executing the same request multiple times produces the same final result as executing it once.
For example, if a customer submits an order twice using the same idempotency key, the server should create only one order and return the original response for subsequent requests.
The goal isn't to prevent requests from being sent more than once. It's to prevent the same operation from being applied more than once.

---

🔹 Safe vs. Idempotent HTTP Methods​

These two concepts are related, but they are not the same.

HTTP Method	Safe	Idempotent	Behavior
GET	Yes	Yes	Retrieves data
HEAD	Yes	Yes	Retrieves headers
POST	No	No (by default)	Creates or processes data
PUT	No	Yes	Replaces a resource
DELETE	No	Yes	Deletes a resource
PATCH	No	Not necessarily	Partially updates a resource

Safe methods don't change the server's intended application state.
Idempotent methods may change the state, but repeating the same operation should have the same intended effect as performing it once.
For example:

• GET: Fetching a product 100 times doesn't change the product.
• PUT: Updating a user's profile with the same data repeatedly produces the same final state.
• DELETE: Deleting an order once or multiple times should leave it deleted.

Notice that an idempotent request doesn't necessarily return the same HTTP response every time. A repeated DELETE request, for example, might return a different status code while keeping the same final state.

---

🔹 Why Is Idempotency Important?​

Networks aren't always reliable. Requests can fail or get delayed for several reasons:

• Network timeouts.
• Temporary server failures.
• Automatic client retries.
• Users clicking buttons multiple times.
• Mobile connections dropping during a request.

Consider this scenario:

1. A customer submits a payment.
2. The payment provider processes it successfully.
3. The response gets lost because of a network timeout.
4. The client assumes the payment failed and retries.
5. The server processes the payment again.

Potential result
The customer gets charged twice.
That's why a good API should be designed for retries, not just successful requests.

---

🔹 How to Make an API Idempotent​

There are two common approaches, and you can use them together depending on your application.

1. Idempotency-Key in HTTP Headers​

This is a common approach for operations that create side effects, such as:

• Processing payments.
• Creating orders.
• Sending emails.
• Booking reservations.

The client generates a unique key for each logical operation and sends it with the request.
Example:

POST /api/orders HTTP/1.1
Content-Type: application/json
Idempotency-Key: 8f14e45f-ea2b-41fc-9a5b-7c9e6672a001

The server checks this key before processing the request.
A typical workflow looks like this:

1. New key: Store the key and mark the request as processing.
2. Completed request: Return the previously saved response without executing the operation again.
3. Request still processing: Return a suitable response, such as HTTP 409 Conflict, or wait for the original request to finish.
4. Failed request: Apply a defined retry policy based on whether the operation actually happened.

The server should also associate the key with the request payload or its fingerprint. Reusing the same key with different request data should be rejected.
Important implementation details:

• Store keys in a database or another reliable shared storage system.
• Use atomic operations or database transactions to prevent concurrent requests from processing the same key.
• Store the original response so retries can receive consistent results.
• Define an expiration policy for old keys.
• Handle failures carefully, especially when an external payment provider is involved.

This approach works well when an operation has side effects that must not happen twice.

2. Unique Constraints on Business IDs​

Sometimes you already have a unique identifier for an operation.
For example, a payment provider may give you a transaction_id or payment_intent_id.
You can store that identifier in your database with a UNIQUE constraint.
Example:

CREATE TABLE payments (
id BIGINT PRIMARY KEY,
transaction_id VARCHAR(255) NOT NULL UNIQUE,
amount DECIMAL(10, 2) NOT NULL
);

Now, if the same payment transaction is submitted twice, the database rejects the second insert because the transaction ID already exists.
Your application can catch the duplicate-key error and return the existing payment result instead of creating another payment.
Advantages:

• Simple to implement.
• Enforced directly by the database.
• Protects against concurrent duplicate inserts.
• Useful for payment transactions and external event processing.

Limitations:

• Requires a reliable business identifier.
• You must handle duplicate-key errors correctly.
• It doesn't automatically prevent every type of duplicate side effect.

---

🔹 Which Approach Should You Use?​

Situation	Recommended Approach
Creating orders	Idempotency-Key
Processing payments	Idempotency-Key + Unique Constraint
Handling payment provider webhooks	Unique event ID
Sending emails	Idempotency-Key or unique job ID
Preventing duplicate records	Database UNIQUE constraint

For critical operations, combining application-level idempotency with database constraints provides stronger protection.

---

🔹 Common Idempotency Mistakes​

Watch out for these implementation mistakes:

• Generating a new key on every retry: A retry must reuse the original key.
• Checking before inserting without atomicity: Two concurrent requests might both pass the check.
• Ignoring request payload differences: The same key shouldn't be accepted for unrelated operations.
• Not handling processing states: A second request may arrive while the first is still running.
• Deleting keys too early: A delayed retry could execute the operation again.
• Assuming database protection covers external services: A database transaction alone cannot guarantee exactly-once execution across an external payment provider.

---

🔹 Final Thoughts​

Idempotency isn't just an HTTP concept. It's an important API design principle.
You can't always prevent clients from sending the same request multiple times, especially when network failures and retries are involved.
What you can do is design your system so that repeating a request doesn't repeat its side effects.
Remember:
Design for retries, not just successful requests.
That's how you avoid duplicate orders, duplicate payments, and many hard-to-debug production issues.