esc
  • WorkflowsPages
  • Case studiesPages
  • Machine Learning FoundationsLearn
  • AI EngineeringLearn
  • AI Guardrails & Red TeamingLearn
  • UI/UXLearn
  • Design SystemsLearn
  • Production System DesignLearn
  • Book a callContact
  • Download ResumeContact
Let’s talk

I’ll build the agent for it.

Availability Patterns: Fail-Over and Replication to Keep Systems Running

Learn how systems stay reachable when one server fails, using fail-over and replication, and how to choose a pattern by the downtime a product can accept.

In short

Availability patterns, fail-over and replication, keep a system reachable when one server fails, chosen by how much downtime a product can accept.

10 min readFree

What you will learn

Some systems stop working when one machine breaks. Users see errors, and orders or messages can be lost. This lesson explains the standard ways to keep a system running and reachable when a part fails. In this course these ways are called availability patterns. There are two main families. Fail-over switches the work to a backup part. Replication keeps copies of data or services on more than one machine, so one failure does not take everything down.

By the end you should be able to explain active-passive and active-active fail-over, describe master-slave and master-master replication at a high level, and choose a pattern based on how much downtime a product can accept. Here are the key words used in this lesson.

Key words

Availability
How much of the time a system is up and reachable when people need it.
Uptime
The time a system is running and answering requests correctly.
Downtime
The time a system is down or cannot be reached.
Single point of failure
One part whose failure stops the whole system.
Redundancy
Keeping an extra part ready, so the system survives when the main part fails.
Fail-over
Moving the work from a broken part to a healthy backup part.
Heartbeat
A short, regular 'I am alive' signal sent from one machine to its partner.
Replication
Keeping copies of the same data or service on several machines.

What does availability mean?

Availability measures how much of the time a system can be used. A system that answers requests is available. A system that is down, or that users cannot reach, is not. Teams talk about uptime, which is the running time, and downtime, which is the time the system is not usable. A simple way to see it: a fast website that is offline is not useful to anyone.

The biggest risk for availability is a single point of failure. Imagine a food ordering site where the whole order system runs on one server. If that server loses power, nobody can order, however well the code is written. Availability patterns exist to remove these single points, so that one broken part does not stop the service.

Redundancy is the main tool for this. It means keeping an extra part that can do the job when the main part fails. The extra part might be a standby server, a second copy of a database, or a second machine in another building. The idea is simple: if one part dies, another keeps going. The price is that you now own, power, and maintain more than one part.

What is fail-over?

Fail-over is the act of moving work from a failed component to a healthy one. Before it can switch, the system has to notice that something is wrong. A common method is the heartbeat. The active machine sends a short signal at a regular interval, and its partner watches for these signals. When they stop for long enough, the partner decides the active machine has failed and takes over.

Fail-over comes in two main forms. The difference is whether the backup sits idle or works alongside the main part.

What is active-passive fail-over?

In an active-passive setup, one server is active and handles all the traffic. A second server, called the passive or standby server, stays ready but does no regular work. When heartbeats from the active server stop, the standby takes over. In one common approach, the standby takes over the active server's network address, so users keep reaching the same address without changing anything.

The switch is not instant. The system must first detect the failure, then promote the standby and send traffic to it. Users may see a short pause or a few failed requests during this switch-over. Also, if the standby's copy of the data is behind the active server's copy, recent changes may be lost when it takes over.

What is active-active fail-over?

In an active-active setup, both servers handle requests at the same time. A traffic splitter, often called a load balancer, shares the requests between them. If one server fails, the other keeps serving and takes over the failed server's work. Because neither machine sits idle, the system can often handle more load. The cost is a more complex design, especially when both servers can change the same data.

What are the trade-offs of fail-over?

  • Idle standby cost

    An active-passive standby is often a full machine that does little until a failure. You pay for it all the same.

  • Possible downtime

    The switch-over takes time, so users may feel a short outage while the system finds the failure and moves the work.

  • Possible data loss

    If the backup is behind the main part, changes it never received can disappear when it takes over.

  • More parts, more complexity

    Every extra machine needs setup, monitoring, and testing, and the switch itself must be tested too.

Replication: keeping copies

Replication keeps copies of data, or copies of a service, on several machines. If one copy is lost, another still has the information. Fail-over handles the switching side of availability, while replication handles the data side. The two often work together: copies are kept in several places, and when the main copy fails, one of the others takes its place. Two common forms are master-slave and master-master.

Master-slave replication

In master-slave replication, one machine is the master. It accepts all writes, which are the changes to the data. Other machines, called slaves or replicas, copy the master's changes. They usually serve reads, which are requests that only look at the data. If the master fails, one replica can be promoted to become the new master. Some teams use the words primary and replica for the same roles.

Master-master replication

In master-master replication, which is also called multi-master, more than one machine accepts writes. Each one passes its changes to the others. If one master fails, the others keep taking writes, so the system stays up without a promotion step. The cost is conflict. Two masters may change the same record at nearly the same time, so the system needs rules that decide which change wins.

How replication connects to consistency

Consistency means that readers see the same, correct data. Replication makes this harder, because copies can briefly disagree. A master can send its changes to replicas after it confirms a write, which is fast but can lose the newest changes if the master crashes. It can also wait for replicas to confirm first, which protects the data but makes each write slower. This is the link between availability and consistency: the faster and more available a design is, the more it may accept short disagreement between copies.

A single master makes it easier to keep one agreed order of changes, but it is less flexible. Multi-master systems are usually loosely consistent, which means copies may differ for a while. You do not need to solve this yet. Later lessons on consistency go deeper.

Redundancy costs time and money

Every availability pattern adds parts, and every part adds cost, setup work, and new ways to fail. A standby that sits idle still needs power and care. A switch-over is one more thing to test. So the real question is not which pattern is best. The real question is how much downtime this product can tolerate, and what it costs to get closer to zero.

Choosing a pattern by downtime tolerance

  • Tolerates short outages

    An internal tool used during office hours may accept a few minutes offline. A simple active-passive setup is often enough.

  • Cannot pause for long

    A checkout that earns money every minute may need active-active parts, so that one failure does not stop all traffic.

  • Cannot lose recent data

    Synchronous replication waits for copies before confirming a write. This prevents loss, but every write becomes slower.

Tip

Before you pick a pattern, ask two questions. How long can users wait when a part fails? How much recent data can the business afford to lose? The answers point you to the simplest pattern that meets the need.

Worked example: a food delivery order database

This example is an illustration of the pattern, not a report on any one company's real setup. A food delivery app stores its orders in a database. The main database is the primary, and a standby replica keeps a copy of it. Here is what happens when the primary crashes in the middle of the dinner rush.

  1. 1

    Normal work

    The primary accepts every new order and every status change. The replica receives each change and keeps its copy up to date.

  2. 2

    Failure

    The primary crashes. The app can no longer write new orders to it.

  3. 3

    Detection

    The heartbeat between the two machines stops. After missing signals for a set time, the system decides the primary is down.

  4. 4

    Promotion

    The standby replica is promoted to primary, so its copy becomes the one the app writes to.

  5. 5

    Redirect

    The app connects to the new primary. Customers may see a brief pause or a retry message.

  6. 6

    Recovery

    Customers place orders again. If the replica had not yet received the last few changes, the team checks those orders for gaps.

Watching a heartbeat in code

The short program below imitates the detection step. Each value in the list is one tick of time, where True means the active server answered and False means it did not. After three misses in a row, the program swaps the roles. Real systems use more careful checks, but the core idea is the same.

Python
missed_limit = 3
heartbeats = [True, True, False, False, False]
active = 'server-A'
standby = 'server-B'
missed = 0
for tick, alive in enumerate(heartbeats, start=1):
    if alive:
        missed = 0
    else:
        missed += 1
    if missed == missed_limit:
        active, standby = standby, active
        print(f'tick {tick}: failover, {active} is now active')
print('final active:', active)

Output

tick 5: failover, server-B is now active
final active: server-B

Check yourself

Quick check

A payment service runs on one server. A second server keeps a copy of its settings and does no work until the first server stops answering heartbeats. Which pattern is this, and what is one downside if the second server's data is slightly behind when it takes over?

Quick check

A news site runs three identical web servers that all answer readers at the same time. A traffic splitter sends each request to a server that is still up. One server fails. Which pattern is in use, and what is its main cost?

Key takeaways

  • Remove single points

    Availability patterns add redundancy so that one failed part does not stop the whole service.

  • Pick by downtime

    Fail-over and replication each have costs. Choose the simplest pattern that meets the downtime and data-loss limits of the product.

  • Expect trade-offs

    Faster recovery and less data loss usually mean more complexity, more cost, or slower writes.

Sources

How is this lesson?

Last updated on 17 November 2026

Learn with me

Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.