esc
  • WorkflowsPages
  • Case studiesPages
  • Machine Learning FoundationsLearn
  • AI EngineeringLearn
  • AI Guardrails & Red TeamingLearn
  • UI/UXLearn
  • Design SystemsLearn
  • Production System DesignLearn
  • Book a callContact
  • Download ResumeContact
Let’s talk

I’ll build the agent for it.

AI engineering / Beginner / 01.37

Closed vs Open Source Models

Rent a model through a company's API, or download one and run it yourself? What each choice means for cost, privacy, control and effort.

14 min readFree

Rent one, or run your own

The last lesson, Type of Models, ended with a question: closed vs open models, renting a model or running your own. This lesson answers it.

Think of coffee. You can buy a cup at a cafe and never see the machine. Or you can buy the machine and make coffee at home. The cafe is easy to start with. The machine costs more and needs cleaning, but the coffee is yours to tweak. It depends on how much you drink and what you care about.

Models work the same way. By the end you will know the difference between a closed model and an open-weight model, why "open weights" is not the same as "open source", how to compare the two on seven points, and how to run a small cost sum that shows when each one costs less. You will run two small Python programs.

Key words

Closed model
A model you can only use through the company that owns it, usually by sending requests to its API. You never get the model's files.
API
A way for your program to send a request to someone else's computer and get an answer back.
Weights
The big list of numbers a model learned during training. They are what the model "knows". The Machine Learning Foundations course explains how they are learned.
Open-weight model
A model whose weights you can download and run on your own machine.
Open source
A stricter label. It means more than open weights: the code and information about the training data are shared too, and you are free to use, study, change and share it.
Licence
The written rules for what you may do with something. For a model, it says if you can use it in a product, change it or share it.
Self-hosting
Running a model on machines you control, instead of calling someone else's.

Closed models: renting

With a closed model, your app sends text to the provider's computers over the internet. Their computers run the model and send an answer back. You pay for what you use, usually counted in tokens. The Tokens lesson explains that count.

The Large language model page on Wikipedia calls this hosted inference. Inference means feeding a trained model some input and getting its output. It says hosted inference "runs a language model on remote AI data centers and makes the output available to users over the Internet".

So with a closed model you never hold the weights. You hold a key, a bill and the provider's rules. The company looks after the machines, the updates and the speed. In return you depend on them: for price, for rules about what you may send, and for the model staying available.

Open-weight models: owning

With an open-weight model, the weights are a file you can download. The same Wikipedia page says that "some open-weight language models can be downloaded and run locally on a personal computer". While the model answers, it says, the weights and other data are held in memory.

Running it yourself also means more jobs land on you: finding a computer powerful enough, installing the model, keeping it running, and updating it when a better version appears. The next lesson, Self-hosted Models, goes into that.

Open weights is not open source

You will often see a downloadable model called "open source". Often that is not accurate. The two words mean different things, and the difference matters if a client asks you what you used.

The Open Source Initiative (OSI) is the group that looks after the meaning of "open source". In 2024 it published the Open Source AI Definition 1.0. It says an Open Source AI is a system made available under terms that grant four freedoms: to use it for any purpose without asking permission, to study how it works and inspect its parts, to modify it for any purpose, and to share it with or without changes.

To make changes possible you need the right ingredients. The definition lists three that must be shared as the "preferred form to make modifications".

  • Data information

    Detailed information about the data used to train the system, so that a skilled person could build a substantially equivalent one. It does not have to be the data itself.

  • Code

    The complete source code used to train and run the system.

  • Parameters

    The model parameters, such as the weights.

Now the key line. The definition says "Open Source models" and "Open Source weights" must include the data information and code used to derive those parameters. So weights alone do not qualify.

Wikipedia's page on open-source AI sums up the OSI's view in a table. With open weights, the weights are released, the training code is not shared, and the training data is not shared or disclosed. With open source, the weights are released, the training code is fully shared and the training data's makeup is fully disclosed.

People also have a word for calling a mostly closed project open source. That page says it is called "openwashing". It also says that Meta's description of Llama as open-source "has been disputed due to Llama's software license, which prohibits it from being used for some purposes". That is a fact about one debate, not a verdict on anyone's model.

Quick check

A company releases a model's weights for anyone to download, but shares no training code and no details about the training data. Under the OSI definition, what is it?

Seven things to compare

Here is the comparison as a table. Read it as questions to ask, not as a scoreboard. Nothing in it says one side wins.

Text
Topic            | Closed (API)                  | Open weights (you run it)
-----------------+-------------------------------+------------------------------
Top-end quality  | Check it for your task        | Check it for your task
Cost model       | Pay per token                 | Pay for hardware, fixed
Privacy, data    | Your text goes to the company | Text stays on your machines
Customising      | Whatever the provider allows  | Change it, if the licence lets you
Speed (latency)  | Internet trip each request    | Depends on your hardware
Upkeep           | Provider does it              | You do it
Licence          | Provider's terms of service   | The model's licence

The first row looks odd, so a word on it. Which kind of model gives the best answers at the top end changes from month to month, and it differs by task. A claim like "closed models are smarter" or "open models have caught up" can be true for one task and false for the next. So this lesson makes no claim. Test on your own examples, with your own data. The Context Evaluation lesson shows how.

Now the rows, in plain words.

Cost: per token or per month

A closed model charges for what you use. A few requests cost almost nothing, and a million cost a lot. A self-run model charges you for the machine whether it is busy or asleep. That is a fixed monthly bill. We look at this in a moment with real arithmetic.

Privacy and data control

With an API, the text you send leaves your systems and goes to the provider. Their terms say what they may do with it, and you must read them. With a model on your own machines, the text can stay inside your own walls. That can matter a lot for medical, legal or customer data.

The Wikipedia page on open-source AI mentions a Nature editorial that makes this point for health care. It suggests that medical care could come to depend on AI models "that could be taken down at any time, are difficult to evaluate, and may threaten patient privacy", and its authors propose open models that hospitals can fine-tune with their own data. That is one editorial's view, but it shows why some teams care.

Control, speed and upkeep

Customising means changing how a model behaves, for example by training it a little more on your own examples. With open weights you hold the file, so that is possible if the licence allows it. With an API you only get the changes the provider offers.

Speed has two parts: the trip over the internet and the time the model takes to think. An API adds the trip. A model on your own machine skips it, but may think more slowly if your hardware is weaker than the provider's. Measure both with your real requests.

Upkeep is the part beginners forget. A provider fixes outages, patches security holes and upgrades machines. If you self-host, that is your weekend.

Read the licence

A licence is the rules for use. Even a model you can download may come with strings.

Wikipedia reports that as of 2025, a plurality (39%) of models released on Hugging Face used the permissive Apache 2.0 licence. It also says some models have licences that give some of the benefits of open-source licences but "contain legal restrictions that deter some companies", who fear a future lawsuit or a change in the terms. It adds that many smaller models do not state any licence at all.

The Hugging Face docs give a one-line rule for all of this: remember to "seek out and respect a project's license if you're considering using their code or data". Their licence page lists many licence types, from Apache 2.0 and MIT to families with names like OpenRAIL and Creative Commons variants. They are not all the same, and the names alone do not tell you what you may do.

  1. 1

    Find it

    Look on the model's page for its licence. If there is none, treat that as a warning, not a green light.

  2. 2

    Read it

    Look for limits on commercial use, limits on certain fields, and rules about sharing changes.

  3. 3

    Check your plan

    Does your product fit inside those rules? Ask a lawyer when money or clients are involved. This lesson is not legal advice.

  4. 4

    Write it down

    Save the licence name and version next to the model name in your project notes.

A cost sum you can run

Let's put numbers on the cost row. Warning first: every price below is made up. Real prices change often and differ a lot between providers and machines. Real choices also weigh quality, privacy and effort, which this sum ignores.

Our made-up world has two options. The API charges $4.00 for every 1,000 requests, and you pay only for what you use. The server costs $1,200.00 a month, fixed, whether busy or not. We try six volumes, from 1,000 up to 1,000,000 requests a month, then find where the lines cross.

The code counts money in whole cents. That keeps the arithmetic exact, because small decimals can drift in a computer.

Python
# All prices are made up. Real prices change and differ a lot.
# Money is counted in whole cents so the sums stay exact.
API_CENTS_PER_1000 = 400    # $4.00 per 1,000 requests, pay as you go
SERVER_CENTS = 120_000      # $1,200.00 per month, fixed, used or not

volumes = [1_000, 10_000, 100_000, 300_000, 500_000, 1_000_000]

print("requests/month   api cost   server cost   cheaper")
for n in volumes:
    api = n * API_CENTS_PER_1000 // 1000
    if api < SERVER_CENTS:
        cheaper = "api"
    elif api > SERVER_CENTS:
        cheaper = "server"
    else:
        cheaper = "same"
    print(f"{n:>14,}   ${api / 100:>8,.2f}   ${SERVER_CENTS / 100:>9,.2f}   {cheaper}")

# Where the two lines cross: server cost divided by cost per request.
cross = SERVER_CENTS * 1000 // API_CENTS_PER_1000
print()
print(f"Both cost the same at {cross:,} requests a month.")
print("Above that, the fixed server is cheaper.")

Output

requests/month   api cost   server cost   cheaper
         1,000   $    4.00   $ 1,200.00   api
        10,000   $   40.00   $ 1,200.00   api
       100,000   $  400.00   $ 1,200.00   api
       300,000   $1,200.00   $ 1,200.00   same
       500,000   $2,000.00   $ 1,200.00   server
     1,000,000   $4,000.00   $ 1,200.00   server

Both cost the same at 300,000 requests a month.
Above that, the fixed server is cheaper.

Read it row by row. At 1,000 requests the API costs $4.00 and the server costs $1,200.00, so renting wins by a huge margin. At 100,000 requests the API costs $400.00, still cheaper. At 300,000 requests both cost exactly $1,200.00. Past that point the API bill keeps climbing while the server bill stays flat: at 1,000,000 requests the API costs $4,000.00 and the server is still $1,200.00.

break-even requests=server cost per monthAPI cost per request\text{break-even requests} = \frac{\text{server cost per month}}{\text{API cost per request}}

In words: divide what the server costs each month by what one API request costs, and you get the number of requests where the two cost the same.

What this sum leaves out

The sum is a teaching toy. A real decision has more in it.

  • People time

    Someone must set up, watch and fix the server. Their time costs money, and the sum above counts none of it.

  • Spare capacity

    A fixed server has a limit. Past some number of requests you need a second server, so the flat line becomes steps.

  • Quality and rules

    If only one kind of model does your task well enough, or your data may not leave the building, cost is not the deciding point at all.

Tip

A low volume almost always means renting is simpler. Start with an API, learn how much you really use, and only then price a server.

Keep your app able to switch

Because the best choice can change, a good habit is to keep the model behind one small function. Your app calls the function. Only that function knows if the model is rented or owned.

The code below uses two stand-ins, small fake functions that return fixed text. They are there so the code runs free, with no keys and no internet. A real model is slower, costs money for every token, can give a different answer each run, and can fail or refuse. The stand-ins pretend none of that.

Python
# Two stand-ins, so this runs free. Real models are slower, cost money
# for every token, can answer differently each run, and can fail or refuse.
def closed_model(messages):
    return "Stand-in reply from a model behind a provider's API."

def open_model(messages):
    return "Stand-in reply from a model running on our own machine."

def summarise(model, text):
    messages = [{"role": "user", "content": "Summarise: " + text}]
    return model(messages)

for model in (closed_model, open_model):
    print(summarise(model, "Our refund policy is thirty days."))

Output

Stand-in reply from a model behind a provider's API.
Stand-in reply from a model running on our own machine.

The function summarise does not care which model it gets. We handed it the first stand-in, then the second, and the rest of the program did not change. In a real app, closed_model would call a provider's API and open_model would call your own server. Switching later would then mean changing one line, not rewriting the app.

How to choose

There is no formula, but there is a sensible order of questions.

  1. 1

    What must stay private?

    If the data cannot leave your systems, that points toward running a model yourself, or toward a provider whose terms you have read and accepted.

  2. 2

    How much will you use it?

    Low or unknown volume is easy to start with an API. Steady, heavy volume is when the cost sum is worth redoing with real prices.

  3. 3

    Do you need to change the model?

    If you must train it further or run it in a special place, check whether an open-weight model and its licence allow that.

  4. 4

    Who will look after it?

    Be honest about your team's time. A server nobody maintains is worse than a bill you can pay.

  5. 5

    Does it pass your test?

    Try both on your own examples. Choose on results, not on reputation.

Quick check

Your app sends about 2,000 requests a month, and nobody on the team has server experience. What is the likely first step?

What to remember

  • Two ways to get a model

    Closed: you use it through the owner's API and never hold the weights. Open weights: you download the weights and run them yourself.

  • Open weights is not open source

    The OSI definition asks for data information, code and parameters, with the rights to use, study, modify and share. Weights alone fall short.

  • Compare, don't crown

    Weigh quality for your task, cost model, privacy, customising, speed, upkeep and licence. Test with your own examples and read every licence.

Next: self-hosted models, running a model on your own machines.

Sources

How is this lesson?

Last updated on 16 November 2026

Learn with me

Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.