esc
  • WorkflowsPages
  • Case studiesPages
  • Machine Learning FoundationsLearn
  • AI EngineeringLearn
  • AI Guardrails & Red TeamingLearn
  • UI/UXLearn
  • Design SystemsLearn
  • Production System DesignLearn
  • Book a callContact
  • Download ResumeContact
Let’s talk

I’ll build the agent for it.

Gateway Offloading: Move Shared Jobs to One Gateway, Not Every Service

Learn how one gateway in front of many services can handle TLS, login checks, rate limits, and logging, and what risks that central point brings.

In short

Gateway offloading moves shared jobs like TLS, login checks, rate limits, and logging to one gateway instead of repeating them in every service.

13 min readFree

What You Will Learn

Many systems are split into several small services. Each service does one job, such as orders, payments, or profiles. Some jobs, however, every service needs: checking who the user is, locking down the connection with encryption, slowing down abusive callers, and writing logs. Gateway Offloading is a design pattern that moves these shared jobs out of each service and into one gateway that sits in front of all of them.

By the end of this lesson, you will be able to explain the problem this pattern solves, walk through the steps a request takes through a gateway, name the jobs that usually move there, and describe the risks you must plan for. You will also see one example from an online shop's mobile app.

Key words

Gateway
A server that sits at the edge of a system. Outside callers reach the gateway first, and the gateway passes allowed requests on to the right service.
Backend service
A service behind the gateway that does real work, such as showing an order list. Outside users usually cannot reach it directly.
Cross-cutting concern
A job that many parts of a system need, such as security or logging. It cuts across every service instead of belonging to just one.
TLS termination
Ending the encrypted connection (TLS, the lock icon in a browser) at one point, so the work of decrypting is done there.
Authentication
Checking that a caller really is who they claim to be, usually by checking a login token.
Token
A small piece of data, issued after login, that proves the user has signed in. The caller sends it with each request.
Rate limiting
Capping how many requests one caller can make in a set time, so one user or bot cannot overload the system.
Offloading
Moving work from one place to another. Here, it means moving shared jobs from each service to the gateway.
Single point of failure
One part whose failure stops the whole system. If the gateway goes down and nothing else can take its place, every request fails.

What Is the Problem of the Same Job Copied Into Every Service?

Imagine a company with twelve microservices. If each service handles its own encryption setup, login checks, rate limits, and logs, then the same work is repeated twelve times. That repetition is the core problem. Microsoft's guidance says that shared features need configuring, managing, and maintaining, and that a shared feature shipped with every application deployment adds admin work and raises the chance of errors. Any update to a shared feature must then reach every service that uses it.

Security work is a common example. Token validation, encryption, and certificate handling can require specialist skills. Without a gateway, a team may need to install and renew a client-facing certificate on every instance of every service, and track when each one expires.

Using a shared piece of code does not remove the problem. The same bug can sit inside twelve separate copies, and each team must find it, fix it, test it, and release it on its own schedule.

  • Duplicated work

    Each service builds and tests the same login check, encryption setup, and throttle. Effort is spent again and again, and skills are spread thin.

  • Configuration drift

    Over time the copies slowly differ. One service may allow old encryption versions, while another checks tokens in a slightly different way. Nobody notices until something breaks.

  • One missed update

    If one team forgets to patch its copy, that service stays weak. Attackers look for the weakest door, so one missed update can become a security gap for the whole system.

What Is the Solution of One Gateway at the Edge?

Gateway Offloading places a gateway at the edge of the system. Callers send every request to the gateway. The gateway performs the shared jobs first, and only then forwards the request to a backend service. Because those jobs now live in one place, the backend services can be simpler. They can focus on business logic, which is the code that follows the shop's own rules, such as how an order total is calculated.

Azure's guidance states the goal plainly: centralizing shared concerns in the gateway simplifies application development and avoids duplicating them across services. The same guidance says that the gateway can handle client-facing certificate management, authentication, TLS termination, monitoring, protocol translation, and throttling on behalf of the services.

Offloading also has a useful side effect. Because one team now owns the shared rules, changes can be made once and checked in one place. Microsoft's operations guidance notes that managing the offloaded feature from one point is easier than managing it on many nodes.

Tip

Offload a job only when many services share it. If only one service needs a feature, keeping it in that service is often simpler.

The next two sub-topics show how a single request moves through the gateway and which jobs usually belong there.

The order of the steps matters. A request should be checked before it reaches a service, and the gateway should decide whether the caller is allowed before doing expensive work.

How Does One Request Move Through the Gateway?

  1. 1

    The caller connects over encryption

    The app opens an HTTPS connection to the gateway's public address.

  2. 2

    The gateway ends the encryption

    The gateway unlocks the encrypted message using its certificate. This is TLS termination, and it is done once at the edge instead of in every service.

  3. 3

    The gateway checks the token

    The gateway reads the login token and confirms it is valid. If the token is missing or wrong, the request stops here with a refusal, and no service ever sees it.

  4. 4

    The gateway applies the limits

    The gateway counts the caller's recent requests. If the caller is over the allowed amount, it refuses the request for now.

  5. 5

    The gateway logs and forwards

    The gateway writes a record of the request and sends the allowed request to the right backend service, often over a fresh internal connection.

  6. 6

    The service does its own work

    The backend runs the business logic, such as looking up a user's orders, and returns the result. The gateway then passes the answer back to the caller.

Which Jobs Usually Move to the Gateway?

  • TLS termination

    The gateway handles the encryption with the caller, so services do not each need certificates and encryption settings.

  • Authentication

    The gateway checks login tokens, so services can trust that a request has already passed a check.

  • Rate limiting

    The gateway enforces request limits per caller. Amazon's gateway, for example, supports a steady rate plus a burst allowance for each method in REST APIs.

  • Caching

    The gateway can keep a copy of a common answer and return it without calling the service again. This saves work for the services.

  • Compression

    The gateway can shrink responses before sending them, so no service needs to implement that step itself.

  • Request logging

    The gateway records each request in one format. Even if a service is badly instrumented, the gateway still gives a basic view of traffic.

Worked Example: Viewing Order History in a Shop App

Consider an online shop with a mobile app. A customer taps 'view order history'. The shop uses a cloud API gateway, such as Azure API Management or AWS API Gateway, in front of its services. The shop configures the gateway to do three things before the request reaches the orders service.

  1. 1

    Secure connection

    The app calls the shop's public HTTPS address. The gateway ends the encryption there. The orders service receives the call without any TLS code of its own.

  2. 2

    Login check

    The app sends the user's token. The gateway checks it. If the token has expired, the app gets a refusal and is asked to sign in again. The orders service is never called.

  3. 3

    Per-user limit

    The gateway counts this user's recent calls. If the user is tapping the button far too fast, the gateway refuses the extra calls, so one user cannot flood the orders service.

  4. 4

    Forward the request

    The gateway sends the valid request to the orders service with the user's identity attached. The orders service looks up the list in its own database and returns it.

The orders service contains no login code, no encryption setup, and no throttle. Its only job is to answer the question 'what orders belong to this customer?' The sketch below shows the same idea in a few lines. It is a teaching model, not a real gateway.

Python
TOKENS = {"token-abc": "user-1"}

def gateway(token, path):
    user = TOKENS.get(token)
    if user is None:
        return "401 unauthorized"
    return "200 forwarded " + path + " for " + user

print(gateway("token-abc", "/orders/history"))
print(gateway("bad-token", "/orders/history"))

Output

200 forwarded /orders/history for user-1
401 unauthorized

Tip

In the sketch, the bad token is stopped before any order code runs. That is the main benefit: the check is written once and every request passes through it.

What Trade-offs Must You Plan for?

Gateway Offloading makes the system simpler in some ways and riskier in others. Microsoft's guidance warns that the gateway adds a network hop to every request, and that each offloaded job adds processing time. The gateway also becomes a high-value target, because one successful attack on it can reach every service behind it. A good design accepts these costs on purpose and plans how to handle them.

There are three main risks: the gateway can fail, it can become slow under load, and the traffic behind it may not be protected. A fourth risk is misuse, when teams put business rules into the gateway. The next sub-topics cover each one.

  • Single point of failure

    If one gateway fails, every service behind it becomes unreachable.

  • Bottleneck

    A gateway that is too small slows down every service behind it at peak times.

  • Unprotected inner traffic

    Traffic from the gateway to services can be unencrypted if no one sets it up correctly.

Keeping the Gateway Up and Fast

Microsoft advises running several gateway instances so that one failure does not stop everything. It also suggests draining connections before restarting an instance, so that active sessions are not dropped. Capacity planning matters just as much. The gateway must handle peak bursts, not only average traffic, because a small gateway that was cheap to run will slow every service behind it. Redundancy and scaling are part of the design from day one, not an extra step added later.

Protecting Traffic Behind the Gateway

Ending encryption at the gateway does not mean the inside is safe by default. Microsoft's guidance says that when the gateway terminates TLS, it must reestablish TLS to the backend, and that unencrypted HTTP should not be used between them. Treat every network as untrusted, even inside the company. Another gap is bypass: if a service's own address is open to the internet, a caller can skip the gateway and avoid the login check and the limits. Services should accept requests only from the gateway path. Forwarded details such as the client's IP address can also be lost unless the gateway passes them on explicitly.

Keeping Business Logic Out of the Gateway

The gateway should handle shared jobs only. Microsoft's guidance is direct: never offload business logic to the gateway. A rule such as 'a gold member gets 10 percent off' belongs in the pricing service, because it changes with the business, and the people who own that rule need to change it. If the gateway holds such rules, it becomes a large, complex program that many teams must touch, and it stops being a simple shared front door.

Tip

A useful test: if the rule would still be needed when the company changes its website or app, it may be a shared job. If the rule only matters to one product's sales, it belongs in that service.

Check Your Understanding

Quick check

A company has 12 microservices. Each one has its own copy of code that checks login tokens. A bug is found, and the 12 teams would each need to patch their copy. Which change best reduces this problem, and what new risk should the company plan for?

Quick check

A shop's gateway already checks tokens and limits calls. A developer proposes moving the rule 'orders over 100 dollars get free delivery' into the gateway, so the orders service stays small. What is the best response?

Quick check

After a gateway is added, an engineer finds that the orders service still has a public address that anyone can call. What is the main risk, and what should the team do?

Key Takeaways

Gateway Offloading moves shared jobs such as TLS termination, authentication, rate limiting, caching, compression, and request logging from many services into one gateway at the edge. This removes repeated work, reduces drift between services, and lets backend services focus on business logic. The price is a central point that must be made redundant, scaled for peak load, and protected, along with the traffic behind it. Keep the gateway for shared jobs only, and never let business rules slip into it. Routing, aggregation, and other gateway designs are covered in their own lessons.

Sources

How is this lesson?

Last updated on 18 November 2026

Learn with me

Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.