Chatty I/O: Why Many Small Requests Slow Systems, and How to Batch Them
Too many small network or storage requests can slow a task far more than the data itself does. Learn to spot the pattern and fix it with fewer, larger calls.
In short
Chatty I/O means too many small network or storage requests, which can slow a task more than the data itself, and the fix is fewer, larger calls.
13 min readFree
What You Will Learn
This lesson covers Chatty I/O, a common performance problem. It appears when a program sends a large number of small requests to finish one job. A request is a message that asks another machine, a file, or a database for something. Each one is an I/O operation, short for input and output. You will learn why many small requests slow a system, how to spot the pattern, how to fix it, and where the fix can go too far.
Key words
- I/O (input and output)
- Any time a program reads from or writes to something outside itself, such as a network connection, a file, or a database.
- Request
- A message one program sends to another and waits on for a reply.
- Round trip
- One request sent out plus one reply sent back.
- Latency
- The time a request spends waiting before its answer arrives.
- Antipattern
- A design habit that seems reasonable at first but often causes trouble later.
- Payload
- The data carried inside one request or one response.
- Batching
- Grouping many small items or requests into one larger request.
- N+1 problem
- One request for a list, then one more request for each item in that list. With N items, that is N+1 requests.
- Endpoint
- A web address on a server that handles one kind of request, such as one that returns a list of posts.
Why Are Many Small Calls Expensive?
Network calls and other I/O work are slow compared with work a program does in its own memory. The program that asks must stop and wait for the answer. When a task needs dozens of answers, most of the time goes to waiting, not to computing. A system does not usually slow down because one request is huge. It slows down because many ordinary requests stack up one after another.
The key idea is that every request has a fixed cost. You pay that cost even when the request asks for one tiny piece of data. So the total time grows with the number of requests, not only with the amount of data moved.
What Is the Fixed Cost of Every Request?
Network delay
Each request travels to the other machine and the reply travels back. That trip costs time even when the reply is only a few bytes.
Connection and protocol work
A program often borrows a connection, prepares the request, and reads the reply. Each of these steps takes time, and they repeat for every call.
Server handling
The remote server must accept each request, look up the answer, and send it back. A server that handles many tiny requests spends much of its effort on setup rather than on useful work.
A helpful way to think about it is that each extra request pays an overhead that one combined request would avoid. Even when every single lookup is fast, the overhead adds up as the number of lookups grows.
What Is the N+1 Pattern?
The classic form of Chatty I/O is the N+1 pattern. The program first asks for a list of N items. Then it loops over the list and asks again, once for each item, to get related data such as an author. The first request gives you one call, and the loop adds N more. Some object-mapping libraries hide this loop, so the developer never sees the many requests being sent.
requests_made = 0
def get_posts():
global requests_made
requests_made += 1
return [{'id': n, 'title': 'Post ' + str(n)} for n in range(1, 31)]
def get_author(post_id):
global requests_made
requests_made += 1
return 'author' + str(post_id)
def get_authors(post_ids):
global requests_made
requests_made += 1
return {pid: 'author' + str(pid) for pid in post_ids}
posts = get_posts()
authors = [get_author(p['id']) for p in posts]
print('chatty requests:', requests_made)
requests_made = 0
posts = get_posts()
authors = get_authors([p['id'] for p in posts])
print('batched requests:', requests_made)
Output
chatty requests: 31 batched requests: 2
The sample fakes a server by counting each request it receives. The first version asks for the 30 posts, then asks for each author one by one. That is 31 requests. The second version asks for the posts, then asks once for all 30 authors. That is 2 requests. The data is the same in both versions. Only the number of trips changed.
This is also why the pattern hides so well. Each lookup is fast on its own, and the code reads cleanly. The slowdown only shows up when the list gets long or when many users load the page at once.
What Are Chatty Calls Between Services?
The same problem happens between separate services, not only between a program and its database. A front end may call a user service, then a photo service, then a post service, each for one small field. A design that needs constant back-and-forth between two services can also be a sign that those services should be combined or better organized.
How Do You Spot Chatty I/O?
Chatty I/O is easier to find than it sounds. You do not need special tools to begin. Start with the user action that feels slow, then count what the system does to finish it. The usual symptoms are high latency, which means long waits, and low throughput, which means little work finished over time.
One action, many requests
A single click, page load, or save triggers dozens or even hundreds of requests. That count is the first clue.
High request count, tiny payloads
The number of requests is high, but each one carries very little data. Many small replies are a warning sign, because the data itself is rarely the cost.
Low CPU, high wait time
The servers and the app are mostly idle, yet the page is slow. The program spends its time waiting on I/O rather than computing.
Repeated near-identical calls
Traces or logs show the same call again and again inside a loop, each with a different ID. That loop is usually the place to fix.
Application performance management tools, often called APM packages, can record request counts and timings for you. Their numbers help you find the pattern without guessing. Deeper monitoring is covered in a separate lesson.
Tip
Count the requests for one user action before you change any code. If one click makes 90 requests, you already know where to look, even before you read the code.
Quick check
A product page loads in 4 seconds. The servers are almost idle, and the request log shows the same 'get price' call repeated 60 times in a row with different product IDs. What is the most likely cause?
How to Fix It
The fix has one core idea: send fewer, larger requests. You still need the same data. You just ask for it in fewer trips. Work through the steps below in order, and check your numbers after each change.
- 1
Measure first
Count the requests for one user action and note which loop or code path makes them. Write the number down so you can compare later.
- 2
Batch the calls
Collect the IDs you need into one list and send them together. A batch lookup asks for many items at once.
- 3
Fetch related data together
A join is a database query that reads two related tables in one go. Use joins, or a query that returns exactly what the screen needs, so related rows arrive together.
- 4
Cache where it fits
A cache is a saved copy of data kept close by so repeat reads skip the slow trip. It helps most with data that is read often and changes rarely. Caching has its own lessons, so use it as a later step.
- 5
Measure again
Compare the new request count with the old one, and check that the screen still shows the same result. A good fix lowers the count and keeps the answer the same.
Batch and bulk endpoints
Microsoft's example shows a web API that offered one GET call per property of a user, such as name, gender, and birth date. Clients then made three calls for every user. The fix was one endpoint that returns the whole user. Each reply is larger, but each client makes fewer calls. This packaging idea is the core of the fix.
GraphQL servers can hit the same problem. GraphQL is a query style where the server runs a small function for each field, so a simple server can end up making one database lookup per item in a list.
Joins and batched lookups
There are two common database fixes for the N+1 pattern. The first is a join, which reads the parent rows and their child rows together. The second is a batched lookup. It fetches all the children for a set of parent IDs in one query. Both turn a long loop of requests into a fixed, small number of requests.
Worked Example: A Social Feed
Here is a realistic case. A social app shows a feed of posts. The feed screen loads 30 posts. For each post, the app then makes three separate API calls: one for the author, one for the like count, and one for the comments. That is about 90 calls for the posts, plus the first call for the list. Each call is small, but together they make the screen slow.
Before the fix
The app works like a loop. It asks the server for the feed, then goes through the 30 posts. For each post it sends three requests and waits for each reply before the next one. The total is one call for the list plus 30 times three, which is 91 requests for a single screen. The server may be lightly loaded, yet the screen still takes a long time, because the waiting adds up.
After the fix
- 1
Design one batched endpoint
Add an endpoint, such as one for the feed, that returns the 30 posts together with each author's name, the like count, and the first few comments.
- 2
Build the response on the server
The server joins the post, author, and count data, or runs grouped lookups, and packs the result into one reply. The server may still run several queries inside, but the app only waits for one answer.
- 3
Render from one reply
The app reads the single response and draws the feed. Requests for one feed load drop from 91 to 1.
- 4
Check the trade-off
Confirm that the response holds only what the feed shows. If comments are long, return a short preview and load the rest when the user opens a post.
Tip
When you redesign an endpoint, ask which fields the screen really shows. Build the response from that list, not from every field the data store has.
Trade-offs: When Fixing Goes Too Far
Batching is a trade, not a free win. Fewer calls mean each call carries more data, and that can cause new problems. The right answer depends on how the app is really used.
Huge payloads
A single reply with everything in it can be slow to build, slow to send, and slow for the app to read. Batching should reduce waiting, not move the waiting into one very large response.
Fetching data nobody uses
If clients only need a user's name, a bundle with every field wastes time and bandwidth. This failure is called extraneous fetching, and it is covered in its own lesson.
One slow part holds up everything
A combined response is only as fast as its slowest piece. If one part of the batch is slow, the whole screen waits for it.
Join side effects
A join can repeat parent rows when a child list is large. The result can be bigger than you expect, so check the shape of the data after you join.
A good rule is to split data by how often it is used. Return that part in the common request, and fetch rarely used parts separately. The goal is a small, fixed number of requests, not one request at any cost. As one write-up puts it, the aim is a bounded query count.
Tip
Set a target before you batch. For example, aim for a fixed number of requests per screen, and track the size of each reply too. Stop when both numbers look reasonable.
Quick check
A mobile app shows 40 products in a list. A developer fixed a chatty screen by making one endpoint that returns every field of every product, including long descriptions and every review. The list now loads slowly, and the app uses a lot of data. What went wrong?
Check Your Understanding
Use the idea you just learned to answer this scenario. Think about the number of trips the page makes, not only how fast each one is.
Quick check
An order-history page loads 50 orders, then calls the server once per order to get its line items. The page feels slow, but the server is barely busy. What is the problem, and what one change would fix it?
Key Takeaways
Chatty I/O happens when one task needs many small requests. Each request has a fixed cost, so the waiting grows with the number of requests. The classic form is the N+1 pattern: one request for a list, then one per item. You can spot it by counting requests per user action, looking for high counts with tiny payloads, and finding idle servers with long waits. The fix is to batch, join, or bundle related data into fewer requests. Keep the trade-off in mind: a bundle that is too large or fetches unused data can make things slow in a new way. Aim for a small, fixed number of requests, and always check that the screen still shows the right result.
Sources
- Microsoft Azure Architecture Center: Chatty I/O antipattern
- Microsoft Azure Architecture Center: Microservices architecture style
- Shopify Engineering: Solving the N+1 Problem for GraphQL through Batching
- DEV Community: Killing N+1 Queries: From Per-Row Lookups to Flat, Batched Reads
- OpenReplay blog: Understanding the N+1 Query Problem
- NHIMG glossary: What Is N+1 Problem? Definition & Examples
- Vocalhyre: What is the N+1 query problem and how do you fix it?
Last updated on 18 November 2026
Related lessons
- System designThrottling: Slow or Refuse Excess Requests to Keep Systems Healthy
- System designBack Pressure: How an Overloaded System Tells Callers to Slow Down
- System designAsynchronism: Accept the Request Now, Do the Slow Work Later
- System designPerformance Antipatterns: Common Design Mistakes That Slow Systems Under Load
Learn with me
Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.