esc
  • WorkflowsPages
  • Case studiesPages
  • Machine Learning FoundationsLearn
  • AI EngineeringLearn
  • AI Guardrails & Red TeamingLearn
  • UI/UXLearn
  • Design SystemsLearn
  • Production System DesignLearn
  • Book a callContact
  • Download ResumeContact
Let’s talk

I’ll build the agent for it.

Load Balancers: Sharing Requests Across Servers and Skipping Broken Ones

A load balancer stands in front of several servers, sends each request to a healthy one, and keeps the site running when a server breaks.

In short

A load balancer stands in front of several servers, sends each request to a healthy one, and keeps the site running when a server breaks.

12 min readFree

What You Will Learn

A website that runs on one server has a hard limit. When too many people visit at once, that server slows down or stops answering. The usual fix is a load balancer. In this lesson you will learn what a load balancer is, why teams use one, how it decides where each request goes, and why the balancer itself needs a backup. You will also walk through a real AWS setup, so you can see the idea working on a screen.

Key words

Load balancer
A tool that stands in front of several servers and passes each incoming request to one of them.
Request
A message from a browser or app that asks a server for something, such as a web page.
Client
The program that sends requests, such as a web browser on a phone or laptop.
Backend server
A server behind the load balancer that does the real work of answering requests.
Server pool
The group of backend servers that a load balancer shares traffic across.
Health check
A small, regular test that the balancer runs to see whether a server still answers correctly.
Single point of failure
One part whose breakdown stops the whole system, even when everything else still works.
Active-passive
A setup where one machine handles all the traffic while a standby machine waits to take over.
EC2 instance
A virtual server that Amazon Web Services (AWS) rents out over the internet.

Why Is One Server Not Enough?

Every server has limits. It has a fixed amount of processor power, memory, and network speed. Its capacity, meaning how much work it can handle in a given time, is finite. When requests arrive faster than it can handle them, they wait in a line, and users see slow pages or errors. A bigger machine helps for a while, but it still has a ceiling. If that one machine stops, the whole site stops with it.

A load balancer tackles two problems together. It spreads the work across many servers, so each one handles a manageable share. It also keeps the site reachable when one server breaks, because the other servers are still ready to answer.

  • More capacity

    Several servers together can handle more requests than one. When traffic grows, you add a server to the pool instead of replacing the one you have.

  • Higher availability

    Availability means how often a system is up and answering. If one server fails, the rest keep serving users, and many visitors never notice the failure.

  • One address for users

    Visitors type one address and reach the balancer. They do not need to know how many servers sit behind it or which one answers their request.

Tip

Adding servers alone does not solve the problem. Visitors still need one entry point that decides where each request goes. That entry point is the load balancer.

How Does a Load Balancer Work?

In a well-built system, clients do not talk straight to the backend servers. Every request goes to the balancer first. The balancer looks at its server pool, leaves out any server it knows is unhealthy, and passes the request to one of the rest. The chosen server does the work and sends its answer back. The balancer then returns that answer to the client. The client sees one service, even though several machines are involved.

  1. 1

    The client sends a request

    A browser asks for the shop's home page. It sends the request to the balancer's address, not to any particular server.

  2. 2

    The balancer picks a healthy server

    It checks the server pool and chooses a server that passed its most recent health checks. Servers that failed are skipped.

  3. 3

    The balancer forwards the request

    The request is passed on to the chosen server.

  4. 4

    The server does the work

    The backend server builds the answer, such as the page or the data the visitor asked for.

  5. 5

    The balancer returns the answer

    The balancer sends the result back to the client, which never learns which server did the job.

The balancer does not do the work itself. Its job is to decide where each request should go and to notice when a destination has stopped working. That second job depends on health checks, which are explained next.

What Are Health Checks?

A balancer that keeps sending traffic to a dead server makes users wait and then fail. To prevent this, the balancer runs health checks. In AWS's Application Load Balancer, the balancer sends a test request to each registered server at a set interval. The default interval is 30 seconds. The server must answer correctly, or the check counts as failed. You can point the test at a specific path, such as a simple health page that the application serves.

One failed check does not remove a server right away. AWS waits for a set number of failures in a row before it marks a target unhealthy. The default is two. Going the other way, a server that is out of service must pass a set number of checks in a row before it comes back, and the default there is five. These thresholds stop the balancer from switching a server on and off because of one slow reply.

  • Active checks

    The balancer sends its own test requests on a timer, whether or not real users are visiting.

  • Passive checks

    The balancer watches the replies to real user requests. If a server keeps returning errors, the balancer can notice without sending any extra tests.

  • Thresholds

    The number of failures in a row before removal, and the number of passes in a row before return. Both settings protect against brief hiccups being treated as real outages.

Tip

A health check should prove that the application can really answer, not just that the machine is switched on. A check that asks for a small page from the application is more useful than a check that only tests whether the port is open.

Worked Example: An AWS Load Balancer

Here is how a small website could be set up in the AWS console, which is the web page where you manage AWS services. The site runs on three EC2 instances. The goal is that visitors reach one address, the three servers share the work, and a broken server is dropped automatically. Follow the steps in order. Menu names are the ones the AWS console uses.

  1. 1

    Create a target group

    In the navigation pane, under Load Balancing, choose Target Groups. A target group is the list of servers the balancer may send requests to. Give it a name, and choose the protocol and port your servers use, such as HTTP on port 80.

  2. 2

    Register the three instances

    Add the three EC2 instances to the target group. Each one is now a target, meaning a server the balancer is allowed to use.

  3. 3

    Set the health check

    Open the Group details tab of the target group and find the Health check settings section. Enter the path the balancer should test, then save. For a first try, keep the default timing and thresholds.

  4. 4

    Create the load balancer

    Create an Application Load Balancer with a listener on port 80. The listener is the entry point that accepts connections from clients. Set its default action to forward traffic to your target group.

  5. 5

    Confirm the targets are healthy

    Open the targets view of the target group. All three should show as healthy once they pass their first checks. Until a target passes, the balancer does not send it requests.

  6. 6

    Test a failure

    Stop one instance on purpose. After it fails two checks in a row, which takes roughly one minute with the default 30-second gap, its status changes to unhealthy. From then on, the balancer sends requests only to the other two. When you start the instance again and it passes five checks in a row, it returns to service.

Notice what the site owner does not have to do. The owner never changes the website address when a server dies. Visitors keep using the same address, and the balancer handles the change behind the scenes. AWS also runs its balancer across several Availability Zones, so the balancer does not depend on a single location.

What If the Load Balancer Fails Too?

The balancer solves the problem of a failing server, but it adds a new risk. Every request now passes through it. If that one balancer stops, all traffic stops, even when every backend server is healthy. A part whose failure stops the whole system is a single point of failure. A balancer must be protected in the same way as any other key part of the system.

The usual fix is to run more than one balancer, so a backup can take over. Managed services such as AWS's balancer already spread their work over several places. If you run your own balancer software, you must set up the pair or group yourself. There are two common patterns, described below.

  • Active-passive

    One balancer carries all the traffic and a second one waits on standby. It is simpler to set up, but the standby sits idle until a failure happens.

  • Active-active

    Several balancers accept traffic at the same time. If one fails, the others keep serving, each with a larger share. Hardware is used better, but the setup needs more care.

  • Managed service

    A cloud provider runs the balancer on several machines for you. You still choose the settings, but the provider handles much of the redundancy.

Tip

When you design a system, ask what happens if the balancer itself stops. If the honest answer is that the whole site goes down, add a second balancer before launch.

What Is an Active-passive Pair?

In an active-passive pair, the active balancer owns a shared address that users connect to. The standby balancer keeps watching it, usually by listening for regular signals from the active one. When those signals stop, the standby claims the shared address and starts handling requests. Users may see a short pause during the switch. A key design risk is making sure only one balancer is active at a time, so the two never fight over the same address.

Active-Active Group

In an active-active group, every balancer accepts traffic at once. Users may reach any of them, and if one stops, the rest keep working. Each survivor then carries a larger share, so the group must have enough spare capacity to absorb a loss. Some visitors may also need their requests kept on the same server, which adds setup work.

Try It: Skipping a Broken Server

The program below is not a real balancer. It shows the core rule in a few lines: send each request only to servers that are marked healthy. Three servers exist, but web-3 has failed its health checks, so it never receives a request. The data is fixed, so the output is always the same.

Python
servers = {'web-1': True, 'web-2': True, 'web-3': False}
healthy = [name for name, ok in servers.items() if ok]
print('healthy servers:', healthy)
for request_id in range(1, 5):
    target = healthy[(request_id - 1) % len(healthy)]
    print('request', request_id, '->', target)

Output

healthy servers: ['web-1', 'web-2']
request 1 -> web-1
request 2 -> web-2
request 3 -> web-1
request 4 -> web-2

Change web-3 to True and run the program again. It joins the rotation at once. Real balancers do the same kind of update, but they learn each server's status from health checks rather than from a fixed list.

Check Your Understanding

Quick check

A shop's website runs on one server and crashes every Friday sale when thousands of people visit at once. The owner adds two more identical servers. What else must be added so visitors are shared across all three, and what should it do if one server crashes?

Quick check

A team runs two balancers in an active-passive pair. The active balancer loses power. The standby stops receiving the regular signals it expects from the active one. What should happen next?

Quick check

A balancer uses a default setting that requires two failed health checks in a row before it removes a server. One of a server's checks fails because of a brief network hiccup, and the next check passes. What does the balancer do?

Key Takeaways

A load balancer sits in front of a server pool, accepts every request, and forwards each one to a healthy server. It lets a system handle more traffic by adding servers, and it keeps users served when one server fails. Health checks tell the balancer which servers are alive. Because the balancer can itself become a single point of failure, teams run it as an active-passive pair or as a group of balancers. In the next lessons you can explore how the balancer chooses servers, and how it works alongside DNS and content delivery networks.

Sources

How is this lesson?

Last updated on 17 November 2026

Learn with me

Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.