esc
  • WorkflowsPages
  • Case studiesPages
  • Machine Learning FoundationsLearn
  • AI EngineeringLearn
  • AI Guardrails & Red TeamingLearn
  • UI/UXLearn
  • Design SystemsLearn
  • Production System DesignLearn
  • Book a callContact
  • Download ResumeContact
Let’s talk

I’ll build the agent for it.

Horizontal Scaling: Adding More Machines to Share the Load

Learn how adding machines differs from making one machine bigger, what must be true for it to work, and what it costs.

In short

Horizontal scaling means handling more load by adding more machines that share the work, instead of making one machine bigger.

11 min readFree

What you will learn

Suppose a website becomes slow because too many people use it at once. The team has two main choices. They can buy a stronger computer, or they can add more computers and split the work between them. The second choice is called horizontal scaling. In this lesson you will learn what it means, what must be true before it works, and what it costs. By the end you should be able to explain why a ticket-sale site might add servers for one weekend and remove them the next week.

Key words

Horizontal scaling
Handling more load by adding more machines that share the work. It is also called scaling out.
Scaling out
Another name for horizontal scaling. You add machines to the group.
Vertical scaling
Handling more load by making one machine stronger, with a faster processor, more memory, or more storage. It is also called scaling up.
Server
A computer, or a program running on one, that answers requests from users or other programs.
Load balancer
A piece of equipment or software that receives requests and passes each one to one of several servers. This lesson only names it. Other lessons explain it in depth.
Stateless
A server keeps no memory of a user between requests. Each request carries or fetches what the server needs.
Session
The saved record that tells a website who is logged in and what they are doing.
Single point of failure
One part whose failure stops the whole system.
Fault tolerance
The ability of a system to keep working when one of its parts breaks.

What are the two ways to add capacity?

Capacity means how much work a system can handle at one time. When a system reaches its capacity, users wait longer or see errors. Engineers usually have two basic ways to add capacity. Each way has a name, and you will see both names often.

  • Scale up (vertical)

    You upgrade the one machine you already have. That can mean a faster processor, more memory, or larger storage. The machine stays the same single box, only stronger.

  • Scale out (horizontal)

    You add more machines and spread the work across them. Each machine handles a share of the requests, and together they do more than one could alone.

  • Which one is simpler

    Scaling up is usually simpler at first. The application often needs no code changes, and there is only one machine to watch. Scaling out takes more planning, but it can keep growing when the first option stops.

Both options are real choices. Many teams start by scaling up, because it is quick. They move to scaling out when the single machine can no longer keep up. The next sections explain why.

Where does scaling up run out?

Scaling up has a hard ceiling. The largest machine you can buy is still one machine, so there is only so much more power you can add. Vertical scaling is simple, but it stops at the largest hardware that exists. Cost also climbs quickly. Vertical scaling may result in high costs because the hardware is expensive. Scaling out can often keep growing by adding more units of ordinary size.

  • Hardware limits

    A single machine has a maximum size. Once you reach it, the only move left is to add more machines.

  • Cost

    Very large machines are expensive per unit of power. A group of smaller machines can often cost less for the same total work.

  • Single point of failure

    One big machine is still one machine. If it fails, the whole service can stop. Upgrading it can also mean downtime while you work on it.

What makes horizontal scaling work?

Adding machines does not help by itself. Three things must be true, or the extra machines cause new problems. The first is that each application server must be able to handle any request. The second is that something must send traffic to the right machines. The third is that data and user sessions must stay consistent across the group.

  • Stateless application servers

    Each web or application server should not keep private memory about a user. Then any server can answer any request, and you can add or remove servers freely.

  • Something that spreads traffic

    A load balancer sits in front of the group and gives each request to one server. The details of how it chooses are covered in other lessons. For now, know that it is the usual front door for a scaled-out system.

  • Shared data and sessions

    Information that many servers need must live somewhere all of them can reach. Sessions are a common example. Keeping that data consistent across many machines can be challenging.

Why do stateless servers matter?

A stateful server keeps user details in its own memory. If user Alice logs in on server A, server A remembers her. When her next request lands on server B, server B may not know who she is. This is why sticky sessions, which pin a user to one server, are often used as a quick fix. They tie the user to a machine, so the group cannot treat servers as interchangeable.

A stateless design removes that tie. The application stores shared details in a central place, such as a database or a fast shared cache. Each server reads what it needs from there. Then the load balancer can send any request to any server. Servers become replaceable, so you can remove one during a quiet hour without losing a user's place.

How do you keep sessions and data consistent?

Moving sessions to a shared store solves the logout problem, but it adds a new part to run. The shared store must be fast, must be available, and must be kept up to date. A common choice is a fast in-memory store that all servers can reach. Another choice is to store a signed token in the user's browser, so the server can check it without looking up a shared record. Each choice trades simplicity for different risks, and the team must pick one on purpose.

Worked example: a ticket-sale surge

Imagine a ticket-sale website, such as the one used for concerts. Most days, a few web servers can handle the visitors. When a popular concert goes on sale, a huge crowd arrives in minutes. This example is a made-up illustration of the idea. It is not a description of how any real company runs its systems. Here is how the team could use horizontal scaling.

  1. 1

    Start with a normal group

    The site runs on a small number of identical web servers. They sit behind one address, and a load balancer shares the requests among them.

  2. 2

    Watch for the surge

    The team sees the request count rise sharply as the sale opens. Response times climb, which is the early sign that the current group is at its limit.

  3. 3

    Add identical servers

    The team starts more copies of the same web application. Because the servers are identical and stateless, each new one can take requests right away. The users still use one address, so nothing changes for them.

  4. 4

    Keep sessions in shared storage

    A buyer's login and cart live in a shared store. When that buyer's next click reaches a new server, the server reads the cart from the store and continues the purchase.

  5. 5

    Remove servers when the rush ends

    When traffic falls, the team stops the extra servers. Nothing important is lost, because the sessions and orders were never stored only on those machines. Some teams automate this step, adding servers when load rises and removing them when it falls.

The key idea is that the group can grow and shrink. The capacity follows the demand, and the users see one steady address throughout.

Benefits and costs

Horizontal scaling is powerful, but it is not free. Weigh both sides before you choose it. The following cards list the main benefits and costs.

  • Higher capacity

    You can keep adding machines as demand grows. The limit is set by your design and shared parts, not by the size of one box. Horizontal scaling is often described as offering much more room to grow than scaling up.

  • Better fault tolerance

    If one machine fails, the others still serve users. This does not happen automatically, though. Scaling adds capacity, but it does not by itself guarantee high availability.

  • Operational complexity

    You now run many machines, not one. You need to deploy changes to all of them, watch them, and handle partial failures. The team must be ready for that work.

  • Network overhead

    Machines talk to each other and to shared stores over the network. Each extra hop adds some delay and load. More machines usually means more messages to send.

  • Design needs

    The app must work so that any server can handle any request. That means no private memory for user state, and shared data kept in a place all servers can reach. Old code may need changes before it can scale out.

Tip

Before you add servers, find the real bottleneck. Ask whether the limit is the processor, memory, the network, or a shared database. Adding web servers does not help if every server waits on the same slow database. A bigger single machine may also be the simpler fix for a small, steady load.

See the session problem in code

The code below is a small model, not a real web server. It shows why a session kept in one server's memory breaks when a second server is added. The first program keeps each session inside its own server. The second program uses one shared store that every server reads.

Python
# Each server keeps sessions in its own memory
servers = {"A": {}, "B": {}}
servers["A"]["alice"] = "logged in"

# Alice's next request lands on server B
print("Server B sees:", servers["B"].get("alice", "logged out"))

Output

Server B sees: logged out

Now the same idea with one shared store. Each server reads from the same place, so the result is the same no matter which server takes the request.

Python
# All servers read sessions from one shared store
shared_store = {}
shared_store["alice"] = "logged in"

for name in ["A", "B"]:
    print("Server", name, "sees:", shared_store.get("alice", "logged out"))

Output

Server A sees: logged in
Server B sees: logged in

The second program is the stateless pattern in miniature. The servers hold no private copy of Alice's session. The shared store holds it, so adding a server does not change the answer.

Check your understanding

Quick check

Your photo-sharing app runs on one server that is at its limit. Users stay logged in because their session is stored in that server's memory. You add a second server, and some users start getting logged out at random. Why does this happen, and what change would let horizontal scaling work?

Quick check

A video app runs on five identical servers. After a big event, traffic drops, and the team wants to remove two servers. Uploaded videos must not be lost. Where should the uploaded files be kept so that removing a server is safe?

Key takeaways

Horizontal scaling handles more load by adding machines that share the work. Scaling up makes one machine stronger, but it has a hard limit, a rising cost, and a single point of failure. Scaling out needs stateless application servers, a way to spread requests across them, and a careful plan for shared data and sessions. The gain is more capacity, and fault tolerance improves only when redundancy and failure handling are configured properly. The price is more machines to run, more network traffic, and an app design that lets any server handle any request. Start with the real bottleneck, then choose the simplest option that will keep working as demand grows.

Sources

How is this lesson?

Last updated on 17 November 2026

Learn with me

Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.