AI engineering / Core / 02.04
OpenAI-compatible APIs
Many model providers, routers and local servers accept the same kind of request. Learn what changes when you switch, and what still breaks.
13 min readFree
One shape, many doors
The last lesson, Model APIs side by side, ended with a promise: next, the OpenAI-compatible shape, one shape many providers copy. Here it is.
Think of a wall socket. Every lamp, kettle and phone charger is different inside, but they all plug into the same hole. If two countries use the same socket, one lamp works in both. Model APIs have something like that. A lot of services accept requests that look the same, so the code you write for one can often talk to another.
By the end you will know what that shared request looks like, which three things you change to switch provider, what a router like OpenRouter does, and why the word compatible needs a pinch of salt. You will also run a small Python program that builds requests for three made-up providers.
Key words
- API
- A way for one program to ask another program to do something. Here, your app asks a model to write a reply.
- Endpoint
- One web address that does one job. The chat endpoint takes a conversation and gives back a reply.
- Base URL
- The first part of every address a service uses. Change it, and the same code talks to a different service.
- API key
- A secret password string that tells the service who is asking. It is sent with each request.
- Chat completions
- A common request shape: you send a list of messages, and the service sends back the next message. The endpoint is usually named /chat/completions.
- Router
- A service that sits in front of many models. You send one request to it, and it passes the request on to the model you named.
- Drop-in
- Able to replace something without changing the rest of your code.
What the shared shape looks like
A chat-completions request has two big parts: a model name, and a list of messages. Each message has a role, such as user, and some content. That is the heart of it. You post it to an address that ends in /chat/completions, and you send your key in a header.
Look at how three different docs describe it. The Ollama docs open with this line: "Ollama supports a subset of the OpenAI API." The vLLM docs say: "vLLM provides an HTTP server that is compatible with many interfaces!" The OpenRouter docs say its request and response shapes are "very similar to OpenAI's APIs, with a few small differences." Three services, one shape they copy.
A likely reason to copy a shape is that people already have code written for it. Copy the shape, and their code can work with you after a small change. It is like making a phone charger that fits the socket people already own.
Tip
You do not need to know any one provider's history. What matters in practice is simpler: the request looks the same, so your code can stay the same.
The three things you change
When code written for one provider is pointed at another, usually three values change. Everything else stays.
Base URL
Where the request goes. A local program, a router and a cloud company each have their own address.
API key
Your secret for that service. Never write it into your code; keep it in an environment variable, which is a named setting outside your code.
Model name
Which model should answer. Names differ between services, so a name that works on one may not exist on another.
Here is a real example from the Ollama docs. It points an OpenAI-style client at a program running on your own computer. Read the base_url, the api_key and the model line. Those are the three values.
# From the Ollama docs, 'OpenAI compatibility' page, shown without its output (not run here)
from openai import OpenAI
client = OpenAI(
base_url='http://localhost:11434/v1/',
api_key='ollama', # required but ignored
)
chat_completion = client.chat.completions.create(
messages=[
{
'role': 'user',
'content': 'Say this is a test',
}
],
model='gpt-oss:20b',
)
print(chat_completion.choices[0].message.content)Look at the api_key line. The docs mark it as required but ignored. The page says: "The client requires an API key value, but Ollama ignores it." The client library insists on a key, so you give it any text. A local program on your own machine has no need to check who you are.
Now the same idea on the other side. The OpenRouter Quickstart says you can use the OpenAI SDK, a ready-made code library for these requests, "pointed at OpenRouter as a drop-in replacement." Its example sets a base URL of https://openrouter.ai/api/v1 and a key from OpenRouter. The rest of your code does not change.
Quick check
You move your app from one provider to another that copies the same request shape. Which three values usually change?
Three kinds of places to point at
Providers that copy the shape come in a few kinds. Here we sort them by where the model runs.
A local server
A program on your own computer or your own server. Ollama and vLLM are two examples. You run the model yourself, so your text does not have to leave your machine.
A router
One address in front of many models. You pick the model by name in each request. OpenRouter is an example.
A cloud company
A company that runs its own models on its own computers and lets you call them over the internet for a fee.
Local servers are covered in the Self-hosted models lesson, so here is only the AI-engineering angle: because they copy the shape, you can build your app against a local model while you are learning, with no bill. Later you can point the same code at a bigger service.
OpenRouter, one door to many models
A router is the idea that you should not have to open an account with ten companies to try ten models. The OpenRouter Quickstart puts it this way: it "gives you access to hundreds of AI models through a single API endpoint." It also says it "handles fallbacks automatically and picks the most cost-effective option for each request." A fallback means a backup: if one choice fails, it tries another.
The API Reference says OpenRouter "normalizes the schema across models and providers so you only need to learn one." Normalize means to make things the same size and shape. Different companies answer in slightly different formats; the router tidies them into one.
Two details to know. First, OpenRouter supports two request shapes: a newer one called the Responses API, and the Chat Completions API at POST /api/v1/chat/completions. The Quickstart's own examples now use the Responses one, so check the page when you follow along. This lesson sticks to chat completions, because the others you will meet copy that one. The Ollama and vLLM docs also list a Responses endpoint beside it.
Second, the model name is how you choose. The Quickstart says: "You can substitute any model slug here", and points to a catalog and a list endpoint for finding the slugs. A slug is just the short name the service uses for a model. So with a router, switching models can be as small as changing one word.
Tip
A router is handy for trying many models fast. It adds one more company between you and the model, so read what it says about prices and privacy before you send anything private.
Try it: a config table and a request builder
Now let us build the idea in plain Python. The program has three made-up providers. Their names, addresses and models are invented, so none of it points at anything real.
- 1
Write a config table
One row per provider: base URL, the name of the environment variable that holds the key, and the model name.
- 2
Build the request
One function takes a provider name and some messages, and returns the address, the header and the JSON body. It only builds them. It sends nothing.
- 3
List what each provider supports
A made-up set of features for each row: streaming, tools and JSON mode.
- 4
Check before you call
A second function compares what you want with what the provider lists, and the program prints a warning when something is missing.
Two notes before you run it. This code sends nothing and uses no network. There is also no model in it at all, so there is nothing to pretend. A real call would be slower, would cost money for every token, could give a different answer each time, and could fail or refuse. And the header shows a dollar-sign name like $HOMEBOX_KEY as a placeholder, because we never put a real key in a lesson.
import json
# Three made-up providers. None of these names or URLs is real.
PROVIDERS = {
"homebox": {
"base_url": "http://localhost:9000/v1",
"key_env": "HOMEBOX_KEY",
"model": "tiny-chat",
"features": {"streaming", "tools"},
},
"hubly": {
"base_url": "https://api.hubly.example/v1",
"key_env": "HUBLY_API_KEY",
"model": "maker-a/mid-chat",
"features": {"streaming", "json_mode"},
},
"bigcloud": {
"base_url": "https://api.bigcloud.example/v1",
"key_env": "BIGCLOUD_KEY",
"model": "grand-chat-1",
"features": {"streaming", "tools", "json_mode"},
},
}
def build_request(name, messages):
p = PROVIDERS[name]
url = p["base_url"] + "/chat/completions"
headers = {"Authorization": "Bearer $" + p["key_env"]}
body = {"model": p["model"], "messages": messages}
return url, headers, body
def check_features(name, wanted):
missing = [f for f in sorted(wanted) if f not in PROVIDERS[name]["features"]]
return missing
messages = [{"role": "user", "content": "Say hello in one sentence."}]
for name in PROVIDERS:
url, headers, body = build_request(name, messages)
print(name, "->", url)
print(" header:", headers)
print(" body: ", json.dumps(body))
print()
wanted = {"streaming", "tools", "json_mode"}
for name in PROVIDERS:
missing = check_features(name, wanted)
if missing:
print("WARNING", name, "does not list:", ", ".join(missing))
else:
print("OK ", name, "lists everything we asked for")
print("Nothing was sent. No network was used.")Output
homebox -> http://localhost:9000/v1/chat/completions
header: {'Authorization': 'Bearer $HOMEBOX_KEY'}
body: {"model": "tiny-chat", "messages": [{"role": "user", "content": "Say hello in one sentence."}]}
hubly -> https://api.hubly.example/v1/chat/completions
header: {'Authorization': 'Bearer $HUBLY_API_KEY'}
body: {"model": "maker-a/mid-chat", "messages": [{"role": "user", "content": "Say hello in one sentence."}]}
bigcloud -> https://api.bigcloud.example/v1/chat/completions
header: {'Authorization': 'Bearer $BIGCLOUD_KEY'}
body: {"model": "grand-chat-1", "messages": [{"role": "user", "content": "Say hello in one sentence."}]}
WARNING homebox does not list: json_mode
WARNING hubly does not list: tools
OK bigcloud lists everything we asked for
Nothing was sent. No network was used.Read the output. The first three blocks show that the request body is almost the same for all three providers. Only the address, the key name and the model name differ. The made-up hubly model name has a maker name and a slash in front, a reminder that model names are not the same everywhere. This is the whole trick of the shared shape, and it fits in one small function.
Now the last four lines. We asked for streaming, tools and JSON mode. The program found that homebox does not list JSON mode, that hubly does not list tools, and that only bigcloud lists all three. The table is made up, but the habit is real: before you switch, check the features you need.
Quick check
In the program, what does build_request do?
Compatible is rarely complete
Here is the catch. Compatible sounds like identical, but it usually means mostly the same. The docs are open about it, and the wording is worth reading.
Ollama says it supports "a subset", so it does not take every field. Its page lists what the chat endpoint supports: streaming, JSON mode, tools and vision. It also shows a box left empty for logprobs, which are the model's scores for each word it might say. For images, it takes a base64-encoded image, which is the picture turned into text, but not an image URL.
vLLM has notes of its own. For its chat endpoint, it says the endpoint is "Only applicable to text generation models with a chat template", so a model needs a chat setup to work. It also says: "user parameter is ignored." And on tools it says that not all models are designed to support parallel tool calls, which means asking for several tool calls at once.
OpenRouter has a quiet one. Its API Reference says: "If the chosen model doesn't support a request parameter (such as logit_bias in non-OpenAI models, or top_k for OpenAI), then the parameter is ignored." Ignored means there is no error. Your request goes through, but a setting you asked for did nothing. For JSON output it says to see the models page for which models support it.
So the same request can behave differently from one door to the next. Ollama's docs even show a trick for tools that expect a default OpenAI model name: copy a model to that name with ollama cp. That is a patch for the model-name difference, and the docs include it because it happens.
A field can be left out
The service takes your request but does not list that field, as with logprobs on Ollama's chat endpoint.
A field can be ignored
No error appears, but the setting does nothing, like the user parameter in vLLM.
A feature can depend on the model
Tools, JSON output and parallel tool calls can work for one model and not for another, even on the same server.
Tip
A silent ignore is the sneaky one. If a setting seems to do nothing after you switch provider, check whether the new service lists it before you blame your code.
A checklist for switching
- 1
Read the compatibility page
Look for the list of supported fields, and for notes about what is ignored.
- 2
Change the three values
Base URL, key and model name. Keep the key out of your code.
- 3
Run a tiny test
Send one short message and check that a reply comes back in the shape you expect.
- 4
Test each feature you use
Streaming, tools, JSON output. Try each one on its own, so you know which one fails.
- 5
Test the failures too
Try a bad key and a wrong model name. Make sure your app shows a polite message, not a crash.
Streaming means getting the reply in small pieces as it is written; the Streaming responses lesson covers it. Tools are covered in Function calling, and JSON output in Structured output. Each one is exactly the kind of feature that can differ from door to door.
Quick check
A provider says it is OpenAI-compatible, and your app uses tools and JSON output. What is the safest next step?
What to remember
The shape is shared
A model name and a list of messages, sent to a chat-completions address with a key.
Three values change
Base URL, API key and model name. Keep keys in environment variables.
Test what you use
Compatible is rarely complete. Fields can be left out or ignored, and features can depend on the model.
Next: using SDKs directly, the libraries providers ship for calling their models.
Sources
Last updated on 16 November 2026
Learn with me
Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.