Load Balancing Algorithms: How a Balancer Picks the Right Server
The rules a load balancer uses to choose a server for each request, and how to pick the right rule for your workload.
In short
Load balancing algorithms are the rules a load balancer uses to choose a server for each request, such as round robin or least connections.
10 min readFree
What you will learn
A popular website can get thousands of visitors at the same time. One computer rarely handles all of them, so teams run many copies of the same app. A load balancer stands in front of those copies and decides, for each new request, which copy should answer. The fixed method it uses to make that decision is called a load balancing algorithm. By the end of this lesson you will know the main methods, what each one looks at, and how to choose among them.
Key words
- Load balancer
- A program or device that receives incoming requests and passes each one to a server in a group.
- Server pool
- The group of servers that can all answer the same kind of request.
- Algorithm
- A fixed set of steps a computer follows to reach a decision.
- Request
- One message from a user's device asking a server to do something, such as show a page.
- Active connection
- A link between a user and a server that is still open and in use.
- Static algorithm
- A rule that makes its choice without checking how busy each server is right now.
- Dynamic algorithm
- A rule that checks live data about each server before it chooses.
- Session stickiness
- Sending the same user to the same server again and again.
- Health check
- A regular test the balancer runs to see whether a server still responds properly.
Why does the rule matter?
If every request went to one server, that server would slow down or crash while the others sat idle. The load balancer spreads the work across the pool so that no single server carries too much. Wikipedia describes load balancing as sharing a set of tasks across a set of resources so that the overall work gets done more efficiently, and it warns against letting some units become overloaded while others are left idle.
The algorithm is the part that makes the spreading happen. Two balancers with the same servers can give very different results if one uses a fixed rotation and the other checks live load. The rest of this lesson explains the two big families of rules, then shows how to match a rule to your app.
What are static algorithms?
Static algorithms make their choice from facts known ahead of time, such as server limits and expected traffic. The balancer never asks a server how busy it is. Static algorithms are simple balancing strategies that depend only on the static parameters of the system defined beforehand. Their big advantage is that they are easy to set up. They are usually simpler to configure and work well when backend servers have predictable workloads and similar characteristics.
What is round robin?
Round robin hands out requests in a fixed cycle. The first request goes to server A, the second to server B, the third to server C, and then the cycle starts again at A. The balancer keeps only a counter, so it needs no data from the servers.
servers = ['A', 'B', 'C']
for i in range(6):
print(i + 1, servers[i % len(servers)])Output
1 A 2 B 3 C 4 A 5 B 6 C
Round robin works well when the servers are alike and every request costs about the same. Its weak point is that it counts requests, not work. It does not consider current server workload. If one server happens to receive several heavy requests in a row, it can fall behind while the others keep pace.
What is weighted round robin?
Weighted round robin is the same cycle with a twist. Each server gets a weight, and a server with a higher weight receives more turns. Weighted round-robin assigns each server a weight proportional to its capacity. So a server with weight 2 gets two turns for every one turn given to a server with weight 1. The catch is that a person sets these weights by hand, so they must be changed when the hardware changes.
IP hash and consistent hashing
IP hash turns the client's IP address into a number and maps that number to a server. The same address always produces the same number, so the same client usually lands on the same server. This is called server affinity. This is one basic way to get session stickiness without extra shared storage. The downside is balance: a few busy addresses can overload one server, and adding or removing a server can move many clients to a new place.
Consistent hashing is a smarter version of the same idea. You do not need to know the ring math to use the idea: it keeps most users where they were when the pool changes.
What are dynamic algorithms?
Dynamic algorithms look at data about each server before they choose. The data can be the number of open connections, the recent response times, or the load on the machine. Dynamic algorithms adapt their distribution decisions based on real-time server metrics like CPU usage, memory consumption, and active connections. The extra accuracy has a cost. The balancer has to collect the data, and that adds work and network traffic.
Least connections
Least connections counts the open links to each server and sends the next request to the server with the smallest count. Servers that already have no room for more sessions are skipped. This rule reacts to what is happening now. A server stuck with a few long jobs will show a high count, so fewer new requests go to it.
The rule is strong when requests take very different amounts of time. Least connections adapts to variable request cost. Round robin would keep sending equal shares to a server that is still busy with a slow job. Least connections notices the backlog and routes around it.
Least response time and weighted variants
Least response time watches how quickly each server has been answering and favors the fastest one. This rule suits latency-sensitive workloads. Response-time-aware algorithms can help optimize latency-sensitive applications.
Weighted variants add capacity to the live data. Weighted least connections uses both the open-link count and the weight set for each server. It considers both the current number of active connections and a pre-assigned weight for each server when making load balancing decisions. A big server with a higher weight can therefore take more open links before a smaller one is chosen.
Worked example: checkout requests on a shopping site
Imagine a shopping site with several identical web servers behind one load balancer. Most requests are quick, such as opening a product page. Checkout requests are heavier because they talk to payment and stock systems. The team sets the balancer to least connections. Here is how it handles a busy moment, using a made-up example with no real site data.
- 1
Read the counts
The balancer keeps a running count of open links for every server. Suppose one server has the fewest open links. That server is the least busy one right now.
- 2
A new request arrives
A shopper opens a product page. The balancer compares the counts and sends the request to the server with the fewest open links.
- 3
A heavy checkout holds a server
Another server is busy with several long checkout requests. Its count stays high, so new requests skip it while those checkouts run.
- 4
The count falls as work finishes
When a checkout completes, its link closes and the count drops. The server becomes a candidate again and can take new requests.
- 5
A failed server is removed
If one server fails its health check, the balancer stops sending it traffic until it passes the check again.
Tip
A rule can only pick the least busy server. If every server is busy, the balancer still picks one. Watch the overall load, because the algorithm cannot create capacity that is not there.
How to choose a rule
No single method is best everywhere. The appropriate choice depends on multiple factors such as system configuration settings, requirements, and characteristics of incoming requests. Use the steps below as a starting point, then check the results in your own system.
- 1
Are the servers identical?
If they are alike, round robin or least connections are both fine. If they differ in size, move to a weighted version so the bigger machines take more work.
- 2
Do request times vary?
If every request takes about the same time, round robin is simpler and good enough. If some requests run long, least connections usually does better.
- 3
Must a user stay on one server?
If the server keeps session data in its own memory, use IP hash or sticky sessions. If session data lives in shared storage, you can choose freely.
- 4
Add health checks last
Whatever rule you pick, pair it with health checks so a dead server stops getting traffic.
Match the rule to the workload
Similar, short requests
Round robin is simple and fits well. Every server gets a fair turn, and no live data is needed.
Long or uneven requests
Least connections spreads the load better, because open-link counts show which servers are still busy with heavy work.
Mixed server sizes
A weighted method lets stronger machines take a larger share, so small machines are not overwhelmed.
Need for session stickiness
IP hash or sticky sessions keep a user on one server, which matters when that server holds the user's session data.
Pair the rule with health checks
A fast rule is useless if it keeps sending users to a dead server. If the reply fails, the balancer marks that server as unhealthy. Load balancing algorithms must account for server failures and automatically reroute traffic to healthy servers. When the server passes again, it rejoins the pool.
Tip
Make the health check test what users actually need. A server that answers a simple ping may still fail on the checkout page. A check that loads a real page gives a truer signal.
Quiz: test your understanding
Quick check
Your app runs on three servers. One is twice as powerful as each of the other two. Some requests take a few milliseconds, and others take several seconds. Would plain round robin work well, and what should you pick instead?
Quick check
An online store keeps each shopper's cart in the memory of the server that handled the first request. Shoppers lose their carts when a later request reaches a different server. Which change deals with this most directly using the rules in this lesson?
See the rules in code
The next sample shows the core of least connections. The dictionary holds the open-link count for each server. The code picks the server with the smallest count.
active = {'A': 5, 'B': 1, 'C': 3}
best = min(active, key=active.get)
print('next request goes to', best)Output
next request goes to B
Real balancers add more detail, such as weights, health state, and timeouts. The core idea stays the same: read the live numbers, then choose.
Summary
Load balancing algorithms are the rules that pick a server for each request. Static rules such as round robin and weighted round robin use fixed settings. Hash-based rules such as IP hash keep a client on one server. Dynamic rules such as least connections and least response time use live data. Choose by looking at server sizes, request cost, and the need for stickiness, then add health checks so failed servers stop receiving traffic.
Sources
- Load balancing (computing) - Wikipedia
- Load Balancing Algorithms - GeeksforGeeks
- Load Balancing Algorithms: Round Robin, Least Conn, IP Hash - Matthew Swong
- Load Balancing Algorithms: Round Robin to Consistent Hash - Techplained
- Session 2.3: Load Balancing Algorithms - Hibranwar
- Round Robin Load Balancing Explained - Free Load Balancer
- Load Balancing Algorithms - Grokking System Design Fundamentals (Design Gurus)
- The Algorithms Behind Load Balancing - Singh Sanjay
- Understanding Load Balancing Algorithms and Strategies - OneUptime
- System Design: Load Balancer - Towards Data Science
- Top Load Balancing Algorithms: Choosing the Right Strategy - DZone
- Load Balancers Explained in 7 Minutes - Hayk Simonyan
Last updated on 17 November 2026
Related lessons
- System designLayer 7 Load Balancing: Routing Each Request by Its Content
- System designLayer 4 Load Balancing: Spreading Connections by Address and Port Alone
- System designLoad Balancers: Sharing Requests Across Servers and Skipping Broken Ones
- System designQueue-Based Load Leveling: Smooth Sudden Traffic Bursts With a Queue
Learn with me
Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.