# Retry: Try a Failed Request Again Safely, With Waits, Limits, and Jitter

> Retries hide brief glitches from users, but without limits, growing waits, and safe operations, they can make an outage worse.

**In short:** A retry tries a failed request again to hide brief glitches, and it needs limits, growing waits, jitter, and safe operations or it can make an outage worse.

From "System design", a free course by Venkat R. Murty: https://designervenkat.online/learn/system-design/retry

## What is a retry?

Many failures in computer systems are short-lived. A server may be busy for a moment, a network connection may drop once, or a service may answer slowly for a few seconds. A retry is when a program sends the same request again after a failure, hoping the second try works. This lesson shows how to retry well, and how retries can go wrong.

The program that sends a request is called the client. A phone app, a web browser, or another service can all be clients. In a distributed system, which is a system made of several machines that talk over a network, many failures are brief. A client that waits a little and tries again often gets the answer it wanted.

Done well, retries hide brief glitches, so the user never notices them. Done badly, retries send extra traffic to a service that is already struggling, and the outage gets worse. The rest of this lesson explains the rules that keep retries on the good side.

- **Request:** A message one program sends to another, asking it to do something or return something.
- **Client:** The program that sends a request, such as a phone app.
- **Transient failure:** A failure that is short-lived and likely to pass if you try again after a short wait.
- **Permanent failure:** A failure that repeats no matter how many times you try, such as a request with bad input.
- **HTTP status code:** A three-digit number a web server sends back to say how a request went.
- **Timeout:** The longest time a client waits for an answer before it gives up on that try.
- **Backoff:** Waiting longer between each retry than the one before.
- **Jitter:** A random amount of extra time added to a wait, so many clients do not act at the same moment.
- **Retry storm:** A flood of repeated requests from many clients that overloads a service trying to recover.
- **Idempotent:** Describes an operation that ends with the same result no matter how many times it runs.
- **Idempotency key:** A unique label a client attaches to one operation and reuses on every retry, so the server knows it is the same operation.

## Which failures should you retry?

A retry only helps when the next try might succeed. So the first job is to sort each failure into one of two groups. Microsoft's guidance on transient faults says to retry only when a fault is likely to be transient, and it notes there is no point retrying an operation that is invalid or that failed for a fatal reason.

- **Retry these:** Timeouts, HTTP 503 responses (service unavailable, often because it is busy), and dropped connections. These usually clear up within seconds, so a short wait and another try can work.
- **Do not retry these:** HTTP 400 responses (bad request), 401 and 403 responses (the caller is not allowed in), and requests with bad input, such as a card number with too few digits. Sending the same bad data again only produces the same error.
- **Check unclear cases:** Some errors do not say what they mean. Read the documentation for that service. Only retry when you are confident the cause is temporary, and when the service has said a retry is safe.

> **Tip:** A simple starting rule: retry timeouts and 503 responses, and fail fast on 4xx errors. The exception is a few 4xx codes that the service documents as temporary, such as 429 (too many requests).

### Why cap the number of attempts?

A retry needs an end. If a client keeps trying forever, it holds on to memory, threads, and connections while it waits. AWS notes that when a client waits a long time, it keeps those limited resources busy, and a server can run out of them. So set a fixed maximum, often a small number of retries, and then return a clear error to the user.

Count the total time too. Microsoft advises that the timeout for each try, plus the waits between tries, should fit inside the total time the operation is allowed to take. A retry plan that adds up to longer than the user will wait is a bad plan, even if each piece looks small.

**Quick check:** A checkout call returns HTTP 400 because the card number has too few digits. Another call returns HTTP 503 from the same payment service a moment later. What should the app do?

A. Retry both errors with backoff, because every error might clear up
B. Retry the 503 with capped backoff and jitter, and show the 400 message at once
C. Retry the 400 with a longer wait, since the user may have typed the number wrong and will fix it later
D. Stop all payment calls for the whole app until a person checks the logs

**Answer:** B. A 503 is a temporary problem, so a capped retry can help. A 400 means the input is bad, and sending the same input again only repeats the error. The user needs to fix the card number.

## What are backoff and jitter?

Retrying right away, again and again, makes the problem worse. Backoff means the client waits longer after each failed try. The common form is exponential backoff, where the wait grows by a fixed multiple each time. The formula below uses a base wait, the number of the retry n starting at 1, and a cap, which is the longest wait the client will ever use.

$$
w_n = \min(\text{cap}, \text{base} \times 2^{n-1})
$$

In words: the wait before retry number n is the smaller of the cap and the base multiplied by two raised to the power n minus one. With a base of 1 second and no cap, the waits are 1, 2, 4, and 8 seconds. The cap stops the waits from growing into minutes. AWS points out that a capped backoff still has a problem. If every client waits the same capped time, they all keep retrying at the same rate.

### Why does jitter matter?

Picture 10,000 apps that all fail at the same moment and all wait exactly 2 seconds. They all send again together, and the service gets a huge spike just as it tries to recover. This is how a retry storm starts. Jitter adds randomness to each wait, so the retries spread out over time. AWS describes jitter as a random amount of time that spreads out the arrival of requests.

Many guides treat full jitter as the safe default. With full jitter, the wait is a random value between zero and the backoff value for that retry. The client picks the value itself, so the clients never have to agree on a schedule.

> **Tip:** Always add jitter when many clients may fail together, which is the usual case for a shared service. Backoff without jitter still lets large groups of clients line up their retries.

### Try the backoff numbers

The sample below computes the capped waits, and then applies a jitter fraction. In a real client, the fraction would be a new random number between 0 and 1 for each retry. Here it is fixed, so the output is the same every time you run it.

```python
BASE = 1.0
CAP = 8.0

def wait_for(retry):
    # retry counts from 1; the wait doubles each time, up to the cap
    return min(CAP, BASE * 2 ** (retry - 1))

def jittered(retry, fraction):
    # fraction stands in for a random number from 0 to 1
    return wait_for(retry) * fraction

for retry in range(1, 6):
    print(retry, wait_for(retry))
print(jittered(3, 0.5))

```

Output:

```text
1 1.0
2 2.0
3 4.0
4 8.0
5 8.0
2.0

```

The first five lines show the waits before retries 1 to 5. The wait stops at 8 seconds because of the cap. The last line applies a jitter fraction of 0.5 to the wait for retry 3, which is 4 seconds, so the jittered wait is 2 seconds.

## Make retries safe with idempotency

Retries are risky for actions that change something. A timeout does not tell the client whether the server finished the work before the connection failed. If the client sends a payment again, the customer may pay twice. AWS warns that a timeout or failure does not mean the side effects, such as a saved order or a charge, did not happen.

The fix is to design the operation to be idempotent. Reading a product price is naturally idempotent, because reading it many times changes nothing. Creating a new order is not, unless the system is built to notice a repeat. When an operation is idempotent, a retry is safe, because repeating it does not add a second charge or a second order.

### Idempotency keys

The common tool is the idempotency key. The client creates one unique label for each logical operation, such as one tap on the Pay button, and sends that same label with every attempt. The server saves the key with the result. When the same key arrives again, the server returns the saved result and does not repeat the work.

The client must make the key once and reuse it. If it makes a new key for each retry, the server sees a new operation every time, and the duplicate problem returns. The sample below is a small in-memory stand-in for a payment service. It has no network and charges no real money, but it follows the same rule.

```python
class PaymentService:
    def __init__(self):
        self.saved = {}          # key -> receipt
        self.charge_count = 0    # how many times the card was really charged

    def charge(self, key, amount):
        if key in self.saved:
            return self.saved[key]   # a retry: replay the saved receipt
        self.charge_count += 1
        receipt = f"charged {amount} (receipt {self.charge_count})"
        self.saved[key] = receipt
        return receipt

service = PaymentService()
key = "checkout-42-pay-tap-1"
print(service.charge(key, 25))
print(service.charge(key, 25))
print(service.charge(key, 25))
print("card charges:", service.charge_count)

```

Output:

```text
charged 25 (receipt 1)
charged 25 (receipt 1)
charged 25 (receipt 1)
card charges: 1

```

The first call does the work. The second and third calls find the saved key and return the same receipt. The count of card charges stays at one, even though the client sent three requests.

**Quick check:** A developer's retry code creates a new idempotency key on every attempt. The first payment request times out, but the charge actually went through. What happens when the code retries?

A. The server returns the saved receipt, so the customer is charged once
B. The server treats the retry as a new purchase, so the customer may be charged twice
C. The server rejects every retry with a 400 bad request error
D. The jitter delay undoes the first charge automatically

**Answer:** B. The key identifies the operation. A new key makes the server think the retry is a different purchase, so the saved result is never found. The client must reuse one key for all attempts of the same operation.

## Worked example: a checkout that retries

Here is a checkout screen in a shopping app. It calls a payment service, and the service returns 503 for a short time. The steps below show how the app uses backoff, jitter, a cap, and one idempotency key together.

1. **Create one key:** When the customer taps Pay, the app creates one unique idempotency key for this purchase and keeps it with the pending order. Every attempt will use this same key.
2. **First try fails:** The payment service returns 503. The app waits about 1 second, plus a random jitter amount, before it tries again.
3. **Retry 1 with the same key:** The app sends the same request with the same key. It gets 503 again, so it waits about 2 seconds, plus a random jitter amount.
4. **Retry 2 with the same key:** Another 503. The app waits about 4 seconds, plus a random jitter amount, and then makes its third retry.
5. **Retry 3 succeeds or ends:** If this retry succeeds, the server finishes the one charge, and the app shows the receipt. If it still fails, the app stops, does not retry again, and shows a clear message that the payment is not confirmed, with a button the customer can press later.

Notice what the key does. Suppose the first request reached the server and charged the card, but the reply was lost. The retries send the same key, so the server returns the original result. The customer is charged once, and the app does not need to guess what happened.

> **Tip:** Keep the whole retry inside the wait the customer will accept. Tune the numbers with your team, and measure how long the screen actually waits.

## When retries keep failing

Retries help when failures are rare or brief. When the service stays down, a retry policy with a cap only limits the damage; it does not fix the service. AWS notes that retries multiply across layers. If five layers each retry three times against a failing database, the load can grow many times over. Cyberaro's guide adds that retry budgets, which limit how many retries a client may send in total, are a useful control.

When the same service keeps failing, the next step is usually a circuit breaker. A circuit breaker stops calls to a failing service after enough failures, so the clients fail fast and the service gets room to recover. That pattern has its own lesson, so here it is only a pointer. For now, remember the three rules: retry only transient failures, cap the attempts, and make the operation idempotent before you retry it.

### Check your understanding

Try the idea on a new case. A mobile app's request to a service times out. Ten thousand users' apps then retry every second with no limit, while the service is already overloaded.

**Quick check:** What goes wrong in this case, and which two changes to the retry policy would you make?

A. Nothing goes wrong; a steady retry every second is the fastest way to recover
B. The retries add load to an overloaded service and create a retry storm. Cap the attempts, and use exponential backoff with jitter
C. The service recovers faster when clients stop retrying, so add a longer timeout to the screen only
D. Retries are always safe, so the only change needed is to add more users' devices

**Answer:** B. A fixed one-second retry with no limit keeps sending traffic while the service is struggling. The fix is a cap on attempts, so retries stop, and growing waits with jitter, so the clients spread out instead of arriving together.

## Sources

- [AWS Builder Center: Timeouts, retries, and backoff with jitter](https://builder.aws.com/content/3EumjoZascWd1oZiEgL8ORlv3qE/timeouts-retries-and-backoff-with-jitter)
- [Microsoft Learn: Recommendations for handling transient faults (Azure Well-Architected Framework)](https://learn.microsoft.com/en-us/azure/well-architected/design-guides/handle-transient-faults)
- [DEV Community: The Idempotency Pattern: Making Retries Safe in Distributed Systems](https://dev.to/ankitdevcode/the-idempotency-pattern-making-retries-safe-in-distributed-systems-hn8)
- [Robertson blog: Retry Strategies, Retry Storms, and Idempotency](https://www.myrobertson.com/blog/retry-strategies-and-idempotency)
- [Cyberaro: Retry Storms in Distributed Systems: How Retry Logic Amplifies Production Failures](https://cyberaro.com/article/retry-storms-in-distributed-systems-why-resilience-code-amplifies-failure)
- [The Daily Byte: The Retry with Backoff Pattern: How to Try Again Without Making Things Worse](https://thedailybyte.substack.com/p/the-retry-with-backoff-pattern-how)
