# Schedule Driven: Running Background Work at Fixed Times

> Learn how jobs that start on a clock work, when to pick them, and how to avoid the common mistakes.

**In short:** A schedule-driven job is background work that starts because the clock says so, run by a scheduler that watches the time and a worker that does the job.

From "System design", a free course by Venkat R. Murty: https://designervenkat.online/learn/system-design/schedule-driven

## What you will learn

Some work in a software product should not wait for someone to press a button. Reports must be built, old data must be cleared, and backups must be taken, all on a steady rhythm. A schedule-driven job is a background job that starts because the clock says so, for example every night at 2 a.m. or every 5 minutes. By the end of this lesson you will know how these jobs run, when to choose them over jobs that react to events, and which mistakes to guard against.

- **Schedule-driven job:** Work that starts because a timer or rule says it is time, not because a user clicked something.
- **Background job:** Work that runs behind the scenes, so the user does not sit and wait for it to finish.
- **Scheduler:** The part that watches the clock and starts a job at the times you set. Cron is a well-known example.
- **Worker:** The program that does the real work once the scheduler starts it.
- **Batch job:** A job that handles many records in one run, such as all of yesterday's orders.
- **Idempotent:** Safe to run more than once. Running it again gives the same result and creates no duplicates.
- **Retry:** Trying a failed step again, often after a short wait.
- **Alert:** A message that tells a person something needs attention.
- **Off-peak hours:** Quiet times, such as the middle of the night, when few people use the system.

## How does a schedule-driven job work?

A schedule-driven job has three parts: a rule that says when it runs, a scheduler that watches the clock, and a worker that does the work. The steps below show how these parts hand off to each other.

1. **Write the rule:** Someone sets a time pattern, such as 2 a.m. every day. Cron, a common scheduler on Unix-like systems, writes these rules in a short text form. Cron schedules use five fields: minute, hour, day of the month, month, and day of the week.
2. **The scheduler waits:** The scheduler does nothing but watch the clock until the next time in the rule arrives. A scheduled job is known as a cron job when cron is the scheduler.
3. **The scheduler starts a worker:** When the time comes, the scheduler starts the job. The job can run on the same machine or on a separate worker.
4. **The worker does the job:** The worker reads the data it needs, does the work, and saves the results to a database or file. The job does not run inside the page a user is looking at.
5. **Results are stored for later:** The user sees the outcome later, for example as a report waiting in the app. Whether and how the job reports back is a separate topic.

A simple crontab line shows the idea. This line starts a summary script at 7:00 every day. The five fields come first, then the command.

```
# minute hour day-of-month month day-of-week  command
0 7 * * * /usr/bin/python3 /opt/shop/send_sales_summary.py
```

This simple pattern is only part of the story. The scheduler is not the same as the worker, and the rule is not the same as the job's logic. Keeping those roles separate makes the system easier to test and to fix.

### What is typical scheduled work?

Scheduled work often has a recurring pattern that people can predict. Examples include nightly data updates, monthly sales reports, periodic health checks, and backups. Azure's guidance lists routine work like daily reports, data retention cleanup, and data consistency checks as typical fits for a timer.

### Why do users never wait?

A user should never have to wait for a scheduled job. Many applications need background work that runs apart from the user interface. The job runs in its own process, so the page the user is on stays fast. The user gets a finished result later, instead of waiting on a spinner while the work runs.

## When should you choose a schedule-driven job?

The key question is what decides the moment the work should happen. If the clock decides, use a schedule. If a specific thing that happened decides, use a trigger that reacts to that thing. The three cards below show the signs that point to a schedule.

- **Recurring and predictable:** The work repeats on a known pattern, such as every morning or on the first day of each month. A cron job is a good fit for time-triggered, predictable, bounded tasks.
- **Batch style:** The work handles many records together, like building a monthly report or cleaning up old rows. Batch work often suits a scheduler, which starts one run that covers the whole set.
- **A small delay is fine:** A cron schedule runs at fixed times, so there is always some gap between when something happens and when the job handles it. If that gap is acceptable, a schedule is usually simpler and cheaper.

### Schedule or event?

Jobs that must react right away belong on the other side. A payment receipt sent right after a customer pays should not wait for the next scheduled run. Those jobs are triggered by an event, such as a message arriving or a user action. The Event-Driven lesson covers that style in detail, so here we only compare the two.

> **Tip:** Ask two questions. Does the clock decide when this should run? Can the result wait until the next scheduled run without harm? If both answers are yes, a schedule-driven job is a strong choice.

**Quick check:** An online store wants to delete expired shopping carts and email a sales summary to managers every morning at 7 a.m. Should it use a schedule-driven job or run the work on each user action, and why?

A. Run both on each user action, because users always see the freshest data that way.
B. Use a schedule-driven job for both, because both run at a fixed time each day and nobody needs to wait for the result.
C. Use a schedule-driven job for the cart cleanup, but run the sales email on each user action.
D. Run both on user actions, because a scheduler can never start work on time.

**Answer:** B. Both tasks happen at a fixed daily time, repeat, and are batch-style. Nobody is waiting on the result, so a schedule-driven job fits both. Running them on user actions would repeat heavy work on every click.

## Worked example: a bank's monthly statement

Here is how a bank's mobile app might use a schedule-driven job to make your monthly statement PDF. The bank's own internal design is not published in the sources used for this lesson, so treat this as an illustration of the idea. The job runs at 1 a.m. on the first day of each month, so the statement is already waiting when you open the app.

1. **The rule is set:** The bank sets a rule for 1 a.m. on the first day of every month. Nobody has to start the job by hand.
2. **The scheduler starts a worker:** At 1 a.m. the scheduler starts a statement worker. Your app is not involved at this point.
3. **The worker gathers the month's activity:** The worker reads last month's transactions for each account and totals them.
4. **The worker builds the PDF:** The worker writes each statement as a PDF and saves it where the app can fetch it.
5. **You open the app later:** When you open the app that morning, the statement is already there. You never waited on a spinner while it was built.

> **Tip:** Night hours are a smart choice here. Few customers are online at 1 a.m., so the heavy PDF work is less likely to slow the app for anyone else.

### Checking the design

Now picture a failure. If the same statement job ran twice, a customer might get two copies, or worse, two charges. The design needs a rule that a statement for one account and one month is made only once, even if the job starts again. Keep that rule in mind for the pitfalls below.

## What are the common pitfalls and safeguards?

Scheduled jobs look simple, but three problems show up again and again. Jobs can overlap or run twice, jobs can fail without anyone noticing, and heavy jobs can slow the whole system at busy times. Each problem has a known fix.

- **Overlap and double runs:** A job can run twice, or two runs can overlap, if the previous run is still busy when the next one starts. Schedulers can also start extra copies when more than one scheduler is running.
- **Silent failure:** A job that never starts, or that stops halfway, can fail with no error on screen. Nobody notices until someone sees stale or missing data.
- **Heavy work at peak time:** A large job that runs during the busiest hours can slow down the pages and services that customers use.

### Make jobs idempotent

The first safeguard is to make each job safe to repeat. Azure's guidance says scheduled tasks should be idempotent, so running the same task twice does not create duplicate results. A job that can run more than once without causing harm is far less risky when a scheduler misfires or a retry happens.

The sample below shows the idea in a small, runnable form. The cleanup rule removes carts older than 30 days. Running it a second time changes nothing, because the carts that were already removed stay removed.

```python
def run_cleanup(carts):
    # Keep only carts that are 30 days old or newer.
    return {cid: age for cid, age in carts.items() if age <= 30}

carts = {"cart-1": 45, "cart-2": 3, "cart-3": 31}
once = run_cleanup(carts)
twice = run_cleanup(once)
print(once)
print(twice)
print(once == twice)

```

Output:

```text
{'cart-2': 3}
{'cart-2': 3}
True

```

> **Tip:** Before a job writes anything, check whether the work is already done. For example, look for an existing statement for this account and month. A database rule that forbids duplicate rows is another strong guard.

### Add retries and alerts

The second safeguard is retries and alerts. A plain cron job does not retry a failed run on its own, so the job or its tooling has to handle that. A retry tries the failed step again, often after a short wait. Pair retries with a heartbeat: the job sends a short signal when it finishes, and a monitor raises an alert when that signal does not arrive on time. This catches jobs that never ran at all, which a failing job cannot report by itself.

### Run heavy jobs off-peak

The third safeguard is timing. Run heavy jobs in off-peak hours, when fewer people use the system. Give each big job a clear time limit, so a stuck run does not hold resources all night. If a job must still run during busy hours, split it into smaller batches so it does not compete with customers for all the capacity at once.

**Quick check:** Every night a scheduler starts a job that emails invoices. Two scheduler servers are running by mistake, so some customers receive the same invoice twice. What is the best first fix?

A. Add more servers so the invoice job finishes faster.
B. Move the invoice job to run on each page view.
C. Make the job safe to repeat, so a second run sends nothing a customer has already received.
D. Turn off the scheduler and send invoices by hand each week.

**Answer:** C. Two schedulers can start the same job, so the job must be safe to run more than once. Making it idempotent stops duplicates at the source. Faster servers or manual work would not stop the double send.

## Recap

A schedule-driven job starts because the clock says so. A scheduler watches the time, a worker does the work, and the user is never kept waiting. Choose this style for recurring, predictable, batch-style work, and choose an event-driven trigger when something must react at once. Protect every scheduled job by making it safe to repeat, adding retries and alerts, and running heavy work off-peak.

> **Tip:** Before you launch any scheduled job, write down three things: its exact time, what happens if it runs twice, and who gets alerted if it never reports success.

## Sources

- [Microsoft Azure Architecture Center: Best Practices for Background Jobs](https://learn.microsoft.com/en-us/azure/architecture/best-practices/background-jobs)
- [Microsoft Power Platform Well-Architected: Develop background jobs recommendation](https://learn.microsoft.com/en-us/power-platform/well-architected/reliability/background-jobs)
- [Microsoft Azure Well-Architected Framework: Recommendations for developing background jobs](https://learn.microsoft.com/en-us/azure/well-architected/design-guides/background-jobs)
- [Wikipedia: Cron](https://en.wikipedia.org/wiki/Cron)
- [Xano: What Is a Cron Job?](https://www.xano.com/learn/what-is-a-cron-job/)
- [Back4app: Background Jobs & Task Schedulers](https://www.back4app.com/glossary/background-jobs-task-schedulers/)
- [nixx.dev: Cron Jobs vs Background Workers: What Should You Use?](https://nixx.dev/blog/cron-jobs-vs-background-workers-what-should-you-use-767cc637)
- [Khimananda: Background Jobs vs Cron Jobs Design Choice (2026 Guide)](https://khimananda.com/blog/background-jobs-vs-cron-jobs-design-choice)
- [DevOpsSchool: What is cronjob? Meaning, Examples, Use Cases & Complete Guide](https://www.devopsschool.nl/cronjob/)
- [Uptimepage: Your cron job can fail for months and nothing will tell you](https://uptimepage.dev/blog/cron-jobs-fail-silently)
- [Cronmint: What Happens When Scheduled Jobs Silently Fail](https://www.cronmint.com/blog/when-scheduled-jobs-silently-fail)
- [QuietPulse: Cron Job Failed Silently? Here's How to Actually Detect It](https://quietpulse.xyz/blog/cron-job-failed-silently-how-to-detect-it)
