# Returning Results: How Slow Background Work Reports Back to the User

> When work finishes after the request has already ended, the system still has to deliver the outcome. Here are the common ways to do it.

**In short:** When slow work finishes after the request has ended, the system still has to deliver the outcome, usually by polling a status address or by pushing a message such as a webhook.

From "System design", a free course by Venkat R. Murty: https://designervenkat.online/learn/system-design/returning-results

## What you will learn

Some work takes too long for a user to wait on. Exporting a large report or processing an uploaded video can take minutes. This lesson explains how a system returns the outcome of that kind of work after the original request has already ended. You will learn the common delivery options, how to track a job's state, how to report errors, and how to keep results available for later. The lesson ends with a worked example and a short quiz.

- **Returning results:** How a system gives the final outcome of a piece of work back to the person or program that asked for it.
- **Request:** A message that a client, such as a web page or phone app, sends to a server to ask for something.
- **Background work:** Work the server does after it has already replied, so the user does not have to wait for it to finish.
- **Job ID:** A unique code the server gives to one piece of work, so the client can ask about it later.
- **Acknowledgment:** A quick reply that says the server got the request. It does not say the work is finished.
- **Polling:** The client asks the server again and again whether the work is done yet.
- **Push:** The server sends the outcome to the client without waiting to be asked.
- **Job state:** The stage a job is in right now, such as queued, running, succeeded, or failed.

## Why does slow work need its own reply?

A normal request follows a simple pattern. The client asks, the server does the work, and the server answers in the same exchange. That pattern breaks when the work is slow. The Azure guidance on this problem says that for long-running background work, you cannot wait for the work to finish before you send a response. So the server splits the job into two parts. First it replies at once with an acknowledgment and a job ID. Then it finishes the slow part in the background and delivers the outcome later.

> **Tip:** An 'accepted' reply is not a success message. It only means the server has taken the job on. The real result arrives later, so the client must be built to check for it.

The quick reply has a standard shape. Its steps are below. Each one keeps the user waiting for as little time as possible.

1. **Check the request:** The server first checks that the request is valid. If it is broken, the server rejects it right away with an error, instead of starting work that will fail.
2. **Create a job record:** The server saves a record with a unique job ID and a starting state, usually queued. The record is what the client will check later.
3. **Hand the work off:** The server passes the work to a place where it waits to be picked up by a worker. How that waiting place and the workers function is taught in the Background Jobs lesson, so it is not repeated here.
4. **Reply with 202:** The server answers with HTTP status code 202. That code means the request was accepted for processing, but the processing has not finished. The reply includes the job ID and where to check progress.

The 202 status is the key signal. A client that reads a 202 knows to come back for the answer. It should not treat the reply as the final result.

## What are the three ways to deliver the outcome?

- **Polling a status endpoint:** The client sends a GET request to a status address every so often. The server replies with the current state. When the state is final, the client fetches the result. This is simple and works from almost any client, but many of the checks may return nothing new.
- **Server push:** The server sends the outcome without being asked. Common forms are a webhook, where the server calls an address the client registered, a WebSocket, which is a connection that stays open for messages in both directions, an email, or a phone push notification. Push feels fast, but it needs more setup on both sides.
- **A processing state on screen:** The user interface shows a label such as Processing and updates it when the job completes. This is a choice about what the user sees. The screen usually gets its updates through polling or through a push.

These options are not rivals in every case. A product can show a processing label on screen while it polls in the background. It can also use push for some events and polling as a backup.

### How does polling work in more detail?

Polling can be made gentler on the server. The Azure guidance suggests the first reply can include a hint about how long the client should wait before its next check. A client should also stop asking once the job reaches a final state. Common final states are succeeded, failed, and canceled. The cost to watch is volume. Every active job means a steady stream of checks, and a long list of jobs can add a lot of load.

> **Tip:** Polling is often the best first version. It needs no public address on the client side.

### How does push work in more detail?

A webhook is usually meant for server-to-server messages. The receiving server needs a public address that the sender can reach, and it must handle signature checks and retries. Browsers cannot receive webhooks directly. A WebSocket can reach a browser, but it needs connection handling, rules for retrying, and a plan for sessions that have gone stale. Webhook delivery is usually at least once. That means the same message can arrive more than once, so the receiver should treat repeats safely.

## How do you choose a pattern?

No single pattern wins everywhere. Polling is easy in a browser or phone app and does not need a public address. Push is a better match when the result must arrive within seconds, or when the receiver is another server. Many mature systems use both. Push is the main channel, and polling acts as a fallback that catches anything a missed push left behind.

**Quick check:** A user clicks 'Export report' in a web app. The report takes about 5 minutes to build. What should the app do right away?

A. Keep the spinner on screen and hold the connection open until the file is ready.
B. Reply with 202, a job ID, and a status address, then show Processing until the job finishes.
C. Reply with 400 Bad Request, because slow exports are not allowed.
D. Send an empty success reply now, then email the file later without any job record.

**Answer:** B. A 202 reply with a job ID lets the page stay usable while the work runs. The user can check the status later. Holding the connection open for five minutes is the problem this pattern solves.

## Tracking job states and errors

A job should always have a clear state. The Azure guidance lists a status field with values such as Pending, Running, Succeeded, Failed, and Canceled. The same names should be used everywhere in the system, so the client always knows what each one means. The four states below cover most of the story.

- **Queued:** The job is saved and waiting for a worker to pick it up. Nothing has started yet.
- **Running:** A worker has picked up the job and is doing the work. The status reply can add a progress value if one is known.
- **Succeeded:** The work is done and the result is ready to fetch. This is a final state, so polling can stop.
- **Failed:** The work stopped because of an error. This is also a final state. The reply should carry a reason the user can understand.

Each status check reads the state that is stored at that moment. The sample below is a toy version of this idea. It has no real waiting, so the output is always the same. Each poll reads the state and then moves the job one step forward, until it reaches the last state.

```python
states = ["queued", "running", "succeeded"]
job = {"id": "report-42", "state": 0}

def poll(job):
    current = states[job["state"]]
    if job["state"] < len(states) - 1:
        job["state"] += 1
    return {"id": job["id"], "status": current}

for attempt in range(1, 5):
    print(attempt, poll(job))

```

Output:

```text
1 {'id': 'report-42', 'status': 'queued'}
2 {'id': 'report-42', 'status': 'running'}
3 {'id': 'report-42', 'status': 'succeeded'}
4 {'id': 'report-42', 'status': 'succeeded'}

```

### Reporting errors and retries

When something goes wrong, the user needs plain words, not a code. A message such as 'The file has no data rows' helps far more than 'Error 500'. Only when the retries run out should the job move to failed. The failed reply should then offer a next step, such as fixing the input and starting the job again.

> **Tip:** Keep the error message and the retry count in the job record. Support staff and the user can then see what happened without guessing.

## Keeping results for later

A result should not disappear the moment one screen shows it. A user may close the page, lose a phone signal, or come back the next day. The finished file, or a link to it, should be stored with the job record. Many systems remove old results after a set time. The product should tell the user how long a result is kept.

> **Tip:** Give each result its own stable address. Then the same link still works after a page reload or a new login.

## Worked example: a video upload

The course example is a video upload. The steps below follow the flow, using public material about how video sites report progress. Where a public page does not say how a notice is sent, this lesson says so.

1. **The file transfer ends:** The viewer's browser sends the video file. A public troubleshooting guide notes that a progress bar reaching 100 percent only means the file transfer is complete, not that the video is ready.
2. **The site confirms and shows Processing:** Instead of making the viewer wait while the video is encoded, the site confirms the upload and shows a Processing label.
3. **The job moves through stages:** Video sites show stage labels while a video processes, such as which quality is ready so far, rather than an error.
4. **A first version becomes playable:** So the viewer may be able to watch before the job is fully complete.
5. **The uploader is told:** In this example, the uploader gets a notification once encoding finishes. A public open-source app shows the same design idea: it displays Uploaded and then Processing, then Ready or a failure message, and it sends a server-sent notification when processing ends.

Put together, the video example uses every idea in this lesson. The quick reply comes first, the state moves through stages, the page shows the state to the user, and the finished video stays available after processing.

## Quick check

**Quick check:** A staff member watches a live dashboard in a web browser. New support tickets must appear within one second. Which delivery method fits best?

A. Poll the status address once every ten minutes.
B. Use a WebSocket so the server can push new tickets to the open page.
C. Register a webhook so the browser receives a call from the server.
D. Send an email for each new ticket and let the staff member read it.

**Answer:** B. A WebSocket keeps a connection open, so the server can push updates to the page right away. Polling every ten minutes is far too slow, and browsers cannot receive webhooks directly. Email is not a live view.

## Key takeaways

When work runs in the background, the system first replies with an acknowledgment and a job ID. It then returns the outcome by polling a status address, by pushing a message, or by updating a processing label on screen. Track each job with clear states and plain error messages. Store finished results so people can fetch them again later. Pick the simplest pattern that meets the need, and add push or a fallback only when the need is real.

## Sources

- [Asynchronous Request-Reply pattern, Azure Architecture Center (Microsoft Learn)](https://learn.microsoft.com/en-us/azure/architecture/patterns/asynchronous-request-reply)
- [Async APIs: The 202 Accepted + Polling Pattern for Long-Running Operations (DEV Community)](https://dev.to/apikumo/async-apis-the-202-accepted-polling-pattern-for-long-running-operations-5e37)
- [HTTP 202 Accepted (protocols.page)](https://protocols.page/protocols/http/status/202/)
- [Async Request-Reply Pattern in Microservices (Akin Gundogdu)](https://www.akingundogdu.com/blogs/async-request-reply-pattern-in-microservices/)
- [Polling vs. Long Polling vs. SSE vs. Webhooks: The Practical Tradeoffs That Matter (HackerNoon)](https://hackernoon.com/polling-vs-long-polling-vs-sse-vs-webhooks-the-practical-tradeoffs-that-matter)
- [Webhooks vs. Polling APIs: Which Architecture Should You Choose? (instawebhook)](https://instawebhook.com/blog/webhooks-vs-polling-apis-which-architecture-should-you-choose)
- [Webhook vs Polling vs WebSocket: When to Use Each (DEV Community)](https://dev.to/henry_hang/webhook-vs-polling-vs-websocket-when-to-use-each-4obh)
- [Polling, WebSockets, or webhooks for product updates (oleg.is)](https://oleg.is/blog/polling-websockets-webhooks)
- [The Asynchronous Request-Reply Pattern: Building Responsive Distributed Systems (Decoding Digital Anomalies)](https://neo01.com/2020/04/asynchronous-request-reply-pattern/)
- [Handling Long-Running Generation Tasks (apxml)](https://apxml.com/courses/deploying-diffusion-models-scale/chapter-4-building-scalable-inference-apis/handling-long-running-tasks)
- [YouTubeVideoUploadState enum, Google Ads API reference](https://developers.google.com/google-ads/api/reference/rpc/v23/YouTubeVideoUploadStateEnum.YouTubeVideoUploadState)
- [YouTube Video Is Stuck Processing HD: What Up to SD Means (AethoVPN blog)](https://www.aethovpn.com/en/blog/entertainment/youtube-video-stuck-processing-hd)
- [11 Ways to Fix 'Still Processing' Video Upload Issue on YouTube (MacMyths)](https://macmyths.com/11-ways-to-fix-still-processing-video-upload-issue-on-youtube/)
- [Upload videos, IBM Video developer docs](https://developers.video.ibm.com/channel-api-video-management/upload-videos-new)
- [Pull request #606: upload complete, processing status and a server-sent ready notification (AlUmmahNow on GitHub)](https://github.com/Al-Ummah-Now/AlUmmahNow/pull/606)
