Publisher/Subscriber: Announce Events Once and Let Many Services Listen
A sender publishes an event to a topic, a broker copies it to each subscriber, and no service has to call another one directly.
In short
In publisher/subscriber messaging, a sender publishes an event to a topic and a broker copies it to each subscriber, so no service calls another directly.
14 min readFree
What You Will Learn
This lesson explains the Publisher/Subscriber pattern, usually shortened to pub/sub. In this pattern, one part of a system announces that something happened, and every other part that cares about that news gets its own copy. You will learn the roles each part plays, the benefits and costs, the main design choices, and how a real checkout screen uses the idea. You will also see a small program that copies the idea in a few lines.
Key words
- Publisher
- The part of a system that sends a message announcing an event. It does not need to know who will read the message.
- Subscriber
- A part of a system that asks to receive certain messages and then handles them.
- Message
- A small packet of data sent from one part of a system to another.
- Event
- A message that says something already happened, such as an order being placed.
- Broker
- The middle service that receives messages, keeps them, and passes copies on to subscribers.
- Topic
- A named channel for one kind of event, such as OrderPlaced.
- Subscription
- One subscriber's own line on a topic. It holds that subscriber's copies of the messages.
- Fan-out
- Sending one message out to many receivers, each getting its own copy.
- Idempotent
- Safe to repeat. Handling the same message twice leaves the same final result as handling it once.
- Dead-letter queue
- A holding place for messages that keep failing, so a person or tool can inspect them later.
Why Do Direct Calls Cause Trouble?
Many early systems let one service call the next one directly. Imagine an Order service that calls an Email service, then an Inventory service, then an Analytics service. The Order code must know each address, wait for each reply, and decide what to do when one of them fails. If the team later adds a Fraud Check service, someone has to edit the Order code again. Every new receiver makes the sender harder to change and harder to run.
Pub/sub offers a different shape. The sender announces the event once and moves on. The receivers attach themselves to the announcement. The sender's code stays the same when receivers come and go, and a slow or broken receiver does not hold up the sender.
What Are the Core Roles and Flow?
The pattern has three roles. Each one has a clear job, and none of them calls another directly. The publisher sends. The broker holds and copies. The subscriber receives and acts.
Publisher
Packs an event into a message and sends it to a topic. It does not know how many subscribers exist, or whether any exist at all.
Broker and topic
The broker is the messaging system. It stores each message on the topic and makes a copy for every subscription that is interested in that topic.
Subscriber and subscription
A subscriber owns a subscription on the topic. It reads its own copies, handles them, and tells the broker when it has finished with each one.
- 1
Publish
The publisher sends one message to the topic and returns to its own work.
- 2
Store
The broker keeps the message so that it is not lost if a subscriber is offline.
- 3
Copy
The broker makes one copy for each subscription on that topic.
- 4
Handle
Each subscriber takes its copy and does its own job at its own speed.
- 5
Confirm
The subscriber confirms to the broker that it finished. Until then, the broker can keep the copy and try again.
Two things stay true in every step. The publisher never calls a subscriber, and a subscriber never calls the publisher. Both only talk to the broker.
Some tools use slightly different words. Azure Event Grid, for example, describes its delivery as push-based, and Azure Service Bus uses topics with subscriptions. The idea stays the same in each one.
Pub/sub is not the same as a plain message queue. A queue usually sends each message to one consumer. A pub/sub topic sends every message to every subscription, so each receiver sees all of the events it signed up for.
Note the difference with another pattern. In competing consumers, several workers share one queue and each message is handled by just one of them. In pub/sub, every subscriber gets every message. A later lesson covers the shared-queue pattern in detail.
Fan-out is the name for that one-to-many delivery. The word describes a single message spreading out to many receivers, the way a fan spreads air. Fan-out is the main feature that makes pub/sub useful.
A new subscriber does not need the publisher's permission to join. It creates its own subscription on the topic, and from then on it receives copies of the new events.
What Are the Benefits and Trade-offs?
Pub/sub is popular because it solves real problems. It also brings costs that you must plan for. A good designer lists both before choosing it.
Loose coupling
The publisher and the subscribers are loosely coupled, which means each one can change without the other changing too. The broker sits between them as a buffer, so either side can be updated alone.
Easy to add subscribers
A new service can start receiving events by adding its own subscription. The publisher code does not need a change, and no other subscriber is affected.
Independent scaling and faults
Each subscriber can be scaled on its own. If one subscriber fails, the publisher and the other subscribers keep working, and the broker holds the messages until the failed one recovers.
Now the costs. The first is eventual consistency. Eventual consistency means that copies of the data catch up after a short delay rather than at the same instant. An email may go out a little after the order is saved, and the stock count may lag for a moment. Users and designers must accept that the system is briefly out of step.
Possible duplicates
Brokers often deliver at least once. That means a message can arrive twice if the subscriber crashed before confirming it. Subscribers must be prepared for repeats.
Possible out-of-order delivery
By default, subscribers may receive messages in a different order from the one in which they were sent. A design must not depend on arrival order unless the broker offers ordering for that case.
Need for idempotent handlers
A handler is the code that does the work for one message. Because repeats are possible, the handler should be idempotent, so a second copy causes no extra harm.
Tip
Expect the trade-offs before you adopt the pattern. If a step must finish before the user sees a result, such as a payment being approved, a plain call or a different design may fit better.
What Are the Key Design Choices?
Once you decide to use pub/sub, three choices shape how it behaves. Each one trades simplicity for control, so choose them on purpose.
The first choice is how subscribers pick their messages. The second is whether messages are pushed to subscribers or pulled by them. The third is what happens when a handler keeps failing.
Next, the three choices in turn.
Topics Versus Filtered Subscriptions
Subscribers usually want only part of what a publisher sends. There are two common ways to give them only what they need. The first is to use separate topics, one for each kind of event. Each subscriber then listens to the topics it needs. The second is a filtered subscription. Here, one topic carries many events, and the broker checks each message's content or labels, then sends a copy only to the subscriptions whose rules match.
Broad topics are simple to run, but every subscriber must throw away the messages it does not want. Narrow topics cut that wasted work, but you then have more topics to name and manage. Some brokers also support wildcard subscriptions, where one rule matches a family of topic names. Pick the granularity that keeps both the publisher and the subscribers simple.
Push Versus Pull Delivery
Push delivery
The broker sends each message to the subscriber as soon as it can. The subscriber does not need to ask. Azure Event Grid is an example of this push style for event notifications.
Pull delivery
The message waits in the subscription until the subscriber asks for it. The subscriber decides when it is ready, which helps when it must work at a steady pace.
Push gives fast delivery with little effort from the subscriber. Pull gives the subscriber control over its own pace, and the broker holds the backlog until the subscriber is ready. Pull suits high-throughput work and long-running workers.
Retries and Dead-Letter Queues
Sometimes a handler fails. The data may be bad, or a service it needs may be down for a while. Pub/sub systems deal with this by retrying, and by setting aside messages that cannot be handled at all.
- 1
Handler fails
The subscriber does not confirm the message, or it reports an error, so the broker knows the work was not finished.
- 2
Retry later
The broker tries again after a wait. Many systems make each wait longer than the last, which is called exponential backoff.
- 3
Move to a dead-letter queue
After a set number of failed tries, the broker moves the message to a dead-letter queue. That keeps one bad message from blocking the subscription forever.
- 4
Inspect and replay
A team looks at the stored message, fixes the cause, and then sends the message back for another try.
Dead-letter queues matter most when a message is malformed, or when it needs a resource that is not there. Keep the failed messages somewhere safe for analysis, and set an alert on the count so that someone notices a growing pile. Azure Service Bus, for example, supports this step as a built-in feature.
Worked Example: Placing an Order
Picture the checkout screen of an online shop. The customer fills in the cart, presses Place Order, and expects a quick confirmation. Behind that button, several things must happen. The shop must send an email, reserve stock, and record the sale for reports. The Order service should not wait for all three, and it should not know about them.
- 1
Customer presses Place Order
The checkout screen sends the order to the Order service. The customer sees a confirmation as soon as the order is saved.
- 2
Order service publishes one event
The Order service sends one OrderPlaced event to the OrderPlaced topic. The event holds an order ID and the order details. The Order service does not list the receivers anywhere.
- 3
Broker copies the event
The broker makes one copy for each subscription on the topic. Here the subscriptions belong to the Email, Inventory, and Analytics services.
- 4
Email service reads its copy
The Email service sends the order confirmation to the customer, then confirms the message to the broker.
- 5
Inventory service reads its copy
The Inventory service reduces the stock for the ordered items, then confirms the message.
- 6
Analytics service reads its copy
The Analytics service adds the sale to its reports, then confirms the message.
- 7
Fraud Check joins later
Next month, a Fraud Check team creates its own subscription on the same topic. The Order service stays unchanged.
On Azure, the topic and subscriptions would be built with Azure Service Bus, where each consumer reads from its own subscription on the topic. On AWS, a team would typically use an SNS topic with one SQS queue per subscriber. The shape is the same in both cases: one publish, then one copy for each subscriber.
Tip
Notice what the checkout screen does not wait for. The customer sees the order accepted before the email is sent or the stock is updated. That is the main reason designers reach for pub/sub here.
Try the Idea in Code
The program below is a small model of a broker. It stores subscribers by topic name, and when a message is published, it calls each subscriber's handler. The Inventory handler remembers which order IDs it has handled, so a repeated event does nothing new. The Email handler does not check, so you can see the repeat it receives.
class Broker:
def __init__(self):
self.subscriptions = {}
def subscribe(self, topic, name, handler):
self.subscriptions.setdefault(topic, []).append((name, handler))
def publish(self, topic, message):
for name, handler in self.subscriptions.get(topic, []):
handler(message)
log = []
def email_handler(message):
log.append("email sends receipt for " + message["id"])
def make_inventory_handler():
done = set()
def handle(message):
if message["id"] in done:
log.append("inventory skips duplicate " + message["id"])
return
done.add(message["id"])
log.append("inventory reserves items for " + message["id"])
return handle
broker = Broker()
broker.subscribe("OrderPlaced", "email", email_handler)
broker.subscribe("OrderPlaced", "inventory", make_inventory_handler())
order = {"id": "order-1001"}
broker.publish("OrderPlaced", order)
broker.publish("OrderPlaced", order)
for line in log:
print(line)
Output
email sends receipt for order-1001 inventory reserves items for order-1001 email sends receipt for order-1001 inventory skips duplicate order-1001
Read the output carefully. The publisher made two publish calls, and each subscriber received both messages. Only the Inventory handler protected itself against the repeat. That is the lesson of the idempotent handler, and a real broker needs it too.
Quick check
A Shipping service needs only orders from one country, but all orders go to one OrderPlaced topic. Which design lets Shipping get only what it needs without changing the Order service?
How Do You Write a Safe Subscriber?
A safe subscriber assumes that any message may arrive more than once. It also cannot assume the order of arrival. The main tool is the message ID. Each event carries a unique ID, and the subscriber uses that ID to notice repeats.
- 1
Read the ID
Take the unique ID from the message before doing any work.
- 2
Check the record
Look up whether this ID was already handled. If it was, skip the work and confirm the message.
- 3
Do the work and record the ID
If the ID is new, do the job and save the ID in the same step where possible, so the two cannot drift apart.
- 4
Confirm to the broker
Tell the broker the message is finished, so it is not sent again.
Tip
Save the processed ID in the same database transaction as the work, when your storage allows it. If the work succeeds but the ID is not saved, a retry could repeat the work.
Common Mistakes
Assuming exactly-once delivery
Many brokers default to at-least-once delivery. Design every handler as if a repeat can happen.
Using pub/sub for a reply
Pub/sub channels run one way. If the publisher needs an answer, a separate reply channel or a direct call fits better.
Ignoring the failure path
Without a dead-letter queue, one broken message can keep failing and crowd out the work behind it.
Check Your Understanding
Read the scenario, then pick the best answer. Think about the roles and the rules you have just learned.
Quick check
The Email service keeps failing on one malformed OrderPlaced message. Inventory and Analytics handle every event without trouble. What should the broker do next?
Quick check
Your Order service calls Email, Inventory, and Analytics directly. The team wants to add Fraud Check next month without changing the Order service. The broker may deliver the same OrderPlaced event twice. What must the Fraud Check service do?
Key Takeaways
A publisher announces an event to a topic. A broker copies that event into each subscription. Subscribers handle their copies on their own, and no service has to call another one directly. This gives loose coupling, easy addition of new subscribers, and independent scaling. The cost is eventual consistency, possible duplicates, possible out-of-order delivery, and the need for idempotent handlers. Choose between topics and filtered subscriptions, pick push or pull, and plan a dead-letter queue for the messages that keep failing.
Sources
- Publisher-Subscriber pattern, Azure Architecture Center (Microsoft Learn)
- Asynchronous Messaging Options, Azure Architecture Center (Microsoft Learn)
- Competing Consumers pattern, Azure Architecture Center (Microsoft Learn)
- What is Pub/Sub? An Interactive Guide to Messaging, Encore blog
- Publish-Subscribe (Pub/Sub), System Design course, AlgoMaster
- What Is Google Cloud Pub/Sub? Topics, Pull vs Push and DLQs, Encore
- Message Queues vs Pub/Sub, Encore
- Event-driven architecture with Google Pub/Sub, ansezz blog
- Pub/Sub Pattern for .NET, GitHub guide (Azure Service Bus topics and subscriptions)
Last updated on 18 November 2026
Related lessons
- System designEvent-Driven: Announce What Happened and Let Other Parts React
- System designEvent Sourcing: Store Every Change as an Event and Rebuild State From It
- System designCompeting Consumers: Many Workers, One Queue, Each Message Handled Once
- System designChoreography: Services React to Events and Pass the Work Along
Learn with me
Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.