Back Pressure: How an Overloaded System Tells Callers to Slow Down
When work arrives faster than it can be handled, the overloaded part must push back, or the whole system can fall over.
In short
Back pressure is when an overloaded part of a system pushes back on callers to slow down, because work is arriving faster than it can be handled.
12 min readFree
What you will learn
Picture a cafe where the counter keeps taking orders while the kitchen falls further behind. Tickets pile up, the wait grows, and at some point the kitchen cannot move at all. Software systems face the same problem. This lesson explains back pressure. It is the way an overloaded part of a system signals the parts that send it work to slow down, wait, or stop. You will learn why an endless waiting line is dangerous, the main ways to apply back pressure, and why the signal must reach all the way back to where the work starts.
Read these key words first. They appear again in the lesson.
Key words
- Back pressure
- A signal from an overloaded part of a system that tells the sender to slow down, wait, or stop.
- Producer
- A part of the system that creates work and sends it onward, such as a web browser or a mobile app.
- Consumer
- A part that receives work and handles it, such as a worker program or a database.
- Queue
- A line where work waits until the consumer is ready to handle it.
- Bounded queue
- A queue with a fixed maximum size. When it is full, it cannot take more work.
- Upstream
- The parts of a system that send work toward a given part. They sit earlier in the chain.
- Latency
- The time one request takes, from the moment it starts until the answer comes back.
- Load shedding
- Deliberately refusing or dropping some work so the work that is accepted can finish.
- HTTP status code
- A three-digit number a web server sends back to say how it handled a request.
- Retry-After
- A response header that tells the caller how long to wait before trying again.
What is the problem when work arrives faster than it leaves?
Work in a system usually moves in one direction. A producer makes a request, and a consumer handles it. If the producer sends work faster than the consumer can finish it, the extra work has to wait somewhere. That place is a queue. A short queue is useful, because it soaks up a brief spike in traffic. But if the gap between arrivals and finishing lasts, the queue keeps growing.
A growing queue causes three harms. First, every waiting item takes up memory. Second, each new item waits longer than the one before it, so latency rises. Third, a waiting item can become useless. The person who sent it may have given up, yet the consumer still spends its time on that work. In the end memory runs out and the program crashes, and every item in the line is lost with it.
Why does a growing queue hide the problem?
A queue looks like free insurance. While it has room, every request is accepted, so the producer seems to be doing fine. The damage builds up out of sight. A queue with no size limit never warns anyone. It keeps accepting work until memory is full, and by then the crash comes suddenly. Engineers call this kind of slow damage a hidden failure, because nothing looks wrong until the moment it breaks.
What does back pressure mean?
Back pressure is the signal that stops the pile-up. An overloaded consumer pushes resistance back toward the producer, and the producer then slows down, pauses, or drops some work instead of adding more to the line. The name comes from fluid moving through a pipe. When the far end is blocked, the push is felt all the way back up the pipe.
The idea works only when three things are true. The queue has a limit. The consumer can tell when it is full or busy. And the producer listens to that message and changes what it does. If any one of these is missing, the system goes back to piling up work.
Tip
Back pressure is not the same as a queue with a size limit. A limit caps memory, but on its own it does not tell the producer anything. The signal has to reach the sender for the design to work.
What are four ways to apply back pressure?
Engineers use several tools to push back. Each one trades something away. The four most common are bounded queues, slowing or blocking the producer, rejecting requests with a status code, and load shedding. Most real systems combine two or more of them.
What are bounded queues?
A bounded queue has a fixed maximum size. Once it is full, the system must make a choice at that moment: wait, refuse, or drop. This is useful because it forces a decision instead of letting the line grow forever. A bounded queue is only a memory cap by itself, though. The sender may not know it was refused, so it may keep sending. For back pressure to work, the refusal has to be reported to the sender.
Blocking or slowing the producer
Inside one program, a full queue can simply block the thread that tries to add to it. That thread waits until there is room, so the producer slows down by itself. This works well inside one machine. Across machines it does not, because the sender is a different program and there is no thread to pause. The cost is also real. If the caller is a web request handler, each waiting caller holds a thread. When many callers wait, the server runs out of threads while new requests keep arriving.
Rejecting requests with 429 or 503
A server can answer a request with a refusal instead of making it wait. The two common codes are 429 and 503. The HTTP standard defines 429 as the answer for a user who has sent too many requests in a given amount of time. The 503 code means the service is unavailable, which fits a server that is busy. The server can add a Retry-After header saying how many seconds to wait, or give a point in time. Retry-After only helps when the caller obeys it.
Tip
Always send Retry-After with a 429 or 503. Without it, many clients retry at once, which makes the overload worse.
Load shedding
Load shedding means the system refuses some work on purpose, once it reaches a limit it can handle. The work it accepts then finishes quickly. Some designs go further and drop lower-priority work first, so the most important requests keep their speed. Load shedding acts inside the overloaded service itself, while rate limiting and throttling are the controls at the entry point. Back pressure handles pressure inside a system, and load shedding handles the boundary where outside traffic enters.
Trade-offs at a glance
Bounded queue
Protects memory and forces a decision when full. Costs: a full queue still needs a rule for what happens next, and refused work must be reported.
Blocking the producer
Slows the sender without extra code, inside one program. Costs: waiting callers hold threads, and it does not work across machines.
Rejecting with 429 or 503
Tells the sender to back off, and keeps the server responsive. Costs: callers must handle the refusal, and some users see errors.
Load shedding
Keeps accepted work fast and protects the core. Costs: some work is lost or delayed, so the choice of what to drop must be made carefully.
A small code example
The sample below uses a bounded queue of size three. A producer tries to add six photo jobs. The first three fit. The rest are refused with a 503 message, so the producer knows to wait. The program uses only the Python standard library.
import queue
jobs = queue.Queue(maxsize=3)
accepted = 0
rejected = 0
for i in range(1, 7):
try:
jobs.put_nowait(f'photo-{i}')
accepted += 1
except queue.Full:
rejected += 1
print(f'photo-{i}: queue full, answer 503')
print('accepted:', accepted)
print('rejected:', rejected)
print('waiting:', jobs.qsize())Output
photo-4: queue full, answer 503 photo-5: queue full, answer 503 photo-6: queue full, answer 503 accepted: 3 rejected: 3 waiting: 3
The refusal is the important part. Without the limit, all six jobs would sit in memory. With the limit, the program stays small, and the refused jobs can be told to come back later.
Worked example: a ticket-sales waiting room
Imagine a ticket-sales site that opens sales for a popular concert. Thousands of visitors arrive in the same minute. Each buyer passes through a checkout service, which talks to a payment database. The payment database can only handle so much at once. If every visitor went straight into checkout, the payment database would slow down, and buyers already partway through would time out. This example shows how back pressure protects them.
- 1
Find the checkout limit
Before sale day, the team load-tests the checkout service. From those tests they choose how many buyers can be in checkout at the same time, so the payment database stays responsive.
- 2
Put the crowd in a waiting room
Visitors who arrive when checkout is full join a waiting room. The page shows that they are in line and tells them the sale is busy. The waiting room is itself a queue, so it also needs a size limit.
- 3
Admit buyers as space opens
When a buyer finishes or leaves, the next person in the waiting room is let in. The number of active checkouts never goes past the limit.
- 4
Answer direct overflow with try-again
If a request reaches checkout while it is full, the service answers with 503 and a Retry-After header. The page shows a 'try again shortly' message instead of a slow, failing screen.
- 5
Protect the buyers already inside
Because active checkouts are capped, the payment database handles only the buyers who are already in progress. Those buyers finish their purchases, and the site stays up.
Notice that the site never accepted more work than checkout could handle. It did not hide the overload. It moved the pressure out to the visitors, who saw clear messages and waited, instead of pushing the stress onto the payment database.
Where back pressure fits in a design
A back-pressure signal has to travel all the way to the source of the work. If the signal stops at the first service, the services before it keep sending work, and the overload just moves up the chain. A common way to make this happen is for each service to pass the refusal upstream, so the browser or app that made the first request is the one that finally waits.
- 1
The database is busy
The payment database is the consumer at the bottom of the chain. It has no free capacity.
- 2
The checkout service stops taking work
Checkout sees its own slots are full and refuses new requests with a 503 and Retry-After.
- 3
The front end passes the refusal on
The web front end receives the 503 from checkout and does not hide it. It returns the same signal to the browser.
- 4
The client waits
The browser shows a message and waits the stated time before trying again. The signal has now reached the source of the load.
The producer must respond
A signal does nothing unless the sender acts on it. A producer that ignores Retry-After and sends the same request again at once makes the overload worse. Many small attempts from many clients can turn a small problem into a large one. The producer should wait, slow its rate, and send fewer requests while it is being refused. Retry rules for this are a topic of their own, so here it is enough to know that the sender has to cooperate.
Here is a short scenario to test this idea.
Quick check
A phone app syncs notes to a server. The server answers with 429 and Retry-After: 30. What should the app do?
Checks for a back-pressure design
Before you ship a service, check the places where work can pile up. Each queue needs a size limit. Each limit needs a rule for what happens when it is full. Each refusal needs a status code and a Retry-After value that the caller can read. Finally, someone must watch the rate of refusals, because a rising count is an early warning.
Tip
Do not treat a long queue as good service. If a user has waited long enough to give up, finishing their work later wastes the consumer's time. A short, clear refusal is often kinder.
Quick check
An image-upload service puts every photo into an unbounded queue for a slow thumbnail worker. During a traffic spike the queue grows until the server runs out of memory and crashes. What back-pressure change would you make, and what would users see?
Recap
The problem
When work arrives faster than it is handled, queues grow, memory fills, latency rises, and the system can crash.
The signal
Back pressure is the message from an overloaded consumer that makes the producer slow down, wait, or stop.
The tools
Bounded queues, blocking or slowing the sender, rejecting with 429 or 503 plus Retry-After, and load shedding. Each has a cost.
The chain
The signal must travel back to the original source, and each service in between should pass it along.
The response
The producer must wait and slow down when it is refused. Aggressive retries turn a small overload into a large one.
Sources
- Matthews Wong, Backpressure in Distributed Systems: Stop a Fast Producer
- RFC 6585, Additional HTTP Status Codes (section 4, 429 Too Many Requests)
- RFC 6585 copy at the HTTP Working Group site
- Redisson glossary, What Is Backpressure in Distributed Systems?
- Software Architecture wiki, Backpressure
- DEV Community, Backpressure: 5 Mechanisms and the 3 That Break Under Load
- Matthew Robertson, Backpressure in Distributed Systems: Stability, Correctness, and Graceful Degradation
- Software Patterns Lexicon, Backpressure Handling in Python
- restfulapi.net, HTTP Status 429 (Too Many Requests)
- The Code Forge, Backpressure: Stop Your System From Drowning in Requests
- Birjob, Rate Limiting, Circuit Breakers, and Backpressure
- Birjob, Rate Limiting, Throttling, and Backpressure: A Developer's Guide
- Level Up, Rate Limiting and Backpressure: Protecting Systems from Themselves
Last updated on 17 November 2026
Related lessons
- Machine Learning FoundationsBack propagation: who is to blame?
- System designNo Caching: Why Repeated Fetches of the Same Data Slow Systems Down
- System designPerformance Antipatterns: Common Design Mistakes That Slow Systems Under Load
- System designChatty I/O: Why Many Small Requests Slow Systems, and How to Batch Them
Learn with me
Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.