esc
  • WorkflowsPages
  • Case studiesPages
  • Machine Learning FoundationsLearn
  • AI EngineeringLearn
  • AI Guardrails & Red TeamingLearn
  • UI/UXLearn
  • Design SystemsLearn
  • Production System DesignLearn
  • Book a callContact
  • Download ResumeContact
Let’s talk

I’ll build the agent for it.

AI engineering / Core / 02.05

Using SDKs Directly

Provider libraries let you call a model in a few lines. See what they add, what they hide, and when to pick one over raw HTTP or a framework.

14 min readFree

The libraries providers ship

The last lesson, OpenAI-compatible APIs, ended with a promise: next, using SDKs directly, the libraries providers ship for calling their models. Here it is.

Think of a restaurant. You can walk into the kitchen, find the chef, and shout your order in the chef's own shorthand. Or you can sit down and talk to a waiter. The waiter knows the shorthand, carries the order, and comes back when something goes wrong. An SDK is the waiter. It is code written by the provider so that you do not have to speak the raw language of their server.

By the end you will know what an SDK adds, how you install one and give it your key, what a minimal call looks like in three SDKs, and how to choose between an SDK, a big framework and plain HTTP. You will also run a small program that builds a raw request, so you can see exactly what an SDK hides.

Key words

SDK
Short for software development kit. Here it means a library the provider ships so your code can call their models in a few lines.
Client
The object you create from an SDK. You set it up once, then call methods on it, such as "make a message".
HTTP request
A message your program sends over the internet to a server. It has a method, an address, headers and sometimes a body.
Header
A labelled extra line on a request, like a stamp on an envelope. The API key usually travels in one.
Retry
Trying a failed call again after a short wait.
Exception
Python's way of saying something went wrong. Your code can catch it and decide what to do.
Pinning
Fixing the exact version of a library you use, so an upgrade cannot change your app by surprise.

Tip

SDK details here were checked on 16 November 2026. Libraries change between versions, so treat every snippet as a picture of that day and check the current docs before you copy it.

What an SDK adds

You could call a model with no SDK at all. A model API is a web address, and any program that can send a web request can talk to it. So why do providers bother to ship a library? Their own pages give some answers.

The OpenAI Python README says its library "includes type definitions for all request params and response fields, and offers both synchronous and asynchronous clients". A type definition tells your editor what fields exist, so it can warn you about a typo before you run anything. Synchronous means your code waits for the answer. Asynchronous means it can do other work while it waits.

The same README lists more. It has a streaming section, built on Server-Sent Events, which is the way a server pushes an answer to you in pieces. It has a retry section and an error section. Let us look at those two, because they are the parts you would otherwise have to build yourself.

  • Retries

    The OpenAI README says certain errors are "automatically retried 2 times by default, with a short exponential backoff". Backoff means waiting a little longer after each failure, like not knocking again straight away.

  • Errors as exceptions

    That README says a failed call to the API raises a subclass of APIStatusError, which carries a status_code. A lost connection raises a subclass of APIConnectionError. All errors inherit from APIError, so you can catch them in one place.

  • Streaming

    You pass stream=True and loop over the events as they arrive, instead of waiting for the whole answer. The streaming lesson showed why that feels faster to a person.

Notice what is not on that list. An SDK does not make the model smarter, cheaper or more truthful. It is plumbing. It makes the calling code shorter and safer. Everything the model does is still decided by the server on the other side.

Quick check

What does an SDK mainly give you?

Install it and set the key

Installing an SDK is one line, and the setup order is almost the same for every provider.

  1. 1

    Install the library

    The Anthropic README says to run pip install anthropic. The OpenAI README says pip install openai. Hugging Face's guide uses a library called huggingface_hub.

  2. 2

    Get a key from the provider

    A key is a long secret string that proves a request is yours. It is also how your account is billed, so treat it like a bank card number.

  3. 3

    Keep the key out of your code

    Store it in an environment variable, a named setting that lives outside your files, so it never lands in a repository by accident. The OpenAI README recommends python-dotenv to put the key in a .env file "so that your API key is not stored in source control".

  4. 4

    Create a client

    Make one client object and reuse it. It reads your key and holds the settings.

The OpenAI README also says the library needs Python 3.10 or higher, and the Anthropic README says Python 3.10+. Check this before you install. An old Python will not run a new SDK.

Tip

Never paste a real key into a lesson, a screenshot or a chat. If one leaks, the provider lets you delete it and make a new one. Do that straight away.

A minimal call in each SDK

Here are the smallest getting-started examples from the two READMEs. They are copied from the docs, so they use the model names the docs used on the day we fetched them. We do not run them in this course. Running them needs a real key and costs real money. Model names get replaced often, so look up the current one yourself.

Text
# From the Anthropic Python SDK README on GitHub, shown without its output (not run here)
import os
from anthropic import Anthropic

client = Anthropic(
    api_key=os.environ.get("ANTHROPIC_API_KEY"),  # This is the default and can be omitted
)

message = client.messages.create(
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": "Hello, Claude",
        }
    ],
    model="claude-opus-5-5",
)
print(message.content)

Read it top to bottom. It imports the client, builds one, and gives it the key from an environment variable. The comment in the docs says that reading ANTHROPIC_API_KEY is the default, so you could leave that line out. Then it calls client.messages.create with a list of messages, a model name and a limit on the answer length, max_tokens.

Text
# From the OpenAI Python SDK README on GitHub, shown without its output (not run here)
import os
from openai import OpenAI

client = OpenAI(
    # This is the default and can be omitted
    api_key=os.environ.get("OPENAI_API_KEY"),
)

response = client.responses.create(
    model="gpt-5.5",
    instructions="You are a coding assistant that talks like a pirate.",
    input="How do I check if a Python object is an instance of a class?",
)

print(response.output_text)

The shape is nearly the same: import, create a client, call one method, print the result. The names differ. One says messages.create, the other says responses.create. The OpenAI README calls the Responses API "the primary API for interacting with OpenAI models", and it says the older Chat Completions API is supported indefinitely. That is the kind of detail you only learn by reading the docs of the SDK you chose.

The Hugging Face Inference client

Hugging Face ships its client inside a library called huggingface_hub. Its guide explains that inference "is the process of using a trained model to make predictions on new data". It lists three places the client can send your request to.

  • Inference Providers

    The guide calls this "a streamlined, unified access to hundreds of machine learning models, powered by our serverless inference partners".

  • Inference Endpoints

    A product to deploy models to production. The guide says inference there runs in a dedicated, fully managed setup on a cloud provider you choose.

  • Local endpoints

    Servers on your own machine, such as llama.cpp, Ollama or vLLM. You connect the client to them. The next lesson is about running models locally.

One client class, InferenceClient, covers all of that. Here is its chat example from the guide.

Text
# From the Hugging Face huggingface_hub inference guide, shown without its output (not run here)
from huggingface_hub import InferenceClient
messages = [
    {
        "role": "user",
        "content": "What is the capital of France?",
    }
]
client = InferenceClient(
    provider="together",
    model="meta-llama/Meta-Llama-3-8B-Instruct",
    api_key="my_together_api_key",
)
client.chat_completion(messages, max_tokens=100)

The guide notes that when you use a provider, you must say which model you want, and the model id must be the one on the Hugging Face Hub, not the provider's own id. It also describes two ways to pay. Calls can be routed through Hugging Face, billed to your Hugging Face account, using a Hugging Face token. Or you can use your own key and talk to the provider directly.

There is a bridge to the last lesson too. The guide says that chat_completion "follows OpenAI's Python client" syntax, so if you are used to OpenAI's code you can switch to InferenceClient by changing a few lines. Same idea, one more library.

What the SDK hides

The Anthropic and OpenAI examples are short because the SDK does the boring part. To see that part, we will build the same kind of call by hand. We will not send it. Python's urllib.request module can make a Request object, which is just a filled-in envelope. Building the envelope sends nothing. Only opening it would.

The address below, api.example.com, is a made-up placeholder, and so are the model name and the key. A real provider's docs give the exact address, headers and body, and they differ from one provider to the next.

Python
import json
import os
import urllib.request

os.environ["DEMO_API_KEY"] = "demo-key-1234567890"  # a made-up key, so the code runs free

def mask(secret):
    return secret[:4] + "..." + secret[-2:]

key = os.environ["DEMO_API_KEY"]
body = {
    "model": "some-model-name",
    "max_tokens": 100,
    "messages": [{"role": "user", "content": "Hello"}],
}

req = urllib.request.Request(
    "https://api.example.com/v1/chat",
    data=json.dumps(body).encode("utf-8"),
    headers={"Authorization": "Bearer " + key, "Content-Type": "application/json"},
    method="POST",
)

print(req.get_method(), req.full_url)
for name, value in req.header_items():
    print(name + ":", value.replace(key, mask(key)))
print(req.data.decode("utf-8"))
print("bytes in body:", len(req.data))
print("sent over the network:", False)

Output

POST https://api.example.com/v1/chat
Authorization: Bearer demo...90
Content-type: application/json
{"model": "some-model-name", "max_tokens": 100, "messages": [{"role": "user", "content": "Hello"}]}
bytes in body: 99
sent over the network: False

Nothing was sent. There is no network call in this code, and the last line says so. The printed request has four pieces. The method is POST, which means we are sending something in. The address says where. The headers carry the key and say the body is JSON. The body holds the model name, the length limit and the messages. It came to 99 bytes.

Now count what an SDK does around those four pieces. It builds the headers and the body from your arguments. It sends the request. It reads the reply and turns it into an object with named fields. It turns a failure into an exception. It retries when the failure was a passing one. Doing all of that by hand, for every provider, is the work you skip by installing the library.

Retries, shown with a stand-in

The code below is a stand-in. fake_model returns fixed text and the failures are pretended, so the code runs free with no key. A real model is slower, costs money for every token, can give a different answer each run, and can fail or refuse. This code only shows the shape of a retry loop, the idea the OpenAI README describes. The count of two retries is borrowed from that README. The code is our own, not the SDK's.

Python
def fake_model(messages):
    return {"text": "Hello! (fixed stand-in answer)", "tokens": 6}

def fake_sdk_call(messages, max_retries=2):
    for attempt in range(1, max_retries + 2):
        try:
            if attempt < 3:
                raise ConnectionError("pretend network blip")
            return fake_model(messages)
        except ConnectionError as err:
            print("attempt", attempt, "failed:", err)
    raise RuntimeError("gave up")

reply = fake_sdk_call([{"role": "user", "content": "Hello"}])
print(reply["text"])

Output

attempt 1 failed: pretend network blip
attempt 2 failed: pretend network blip
Hello! (fixed stand-in answer)

Two attempts failed, and the third worked, so the caller only saw the good answer. That is the quiet comfort an SDK gives. Be careful with it, though. The OpenAI README says retries only happen when the request body can be safely resent, and that stream consumption is not automatically retried, because replaying a request could duplicate output already delivered to your application. The README lets you change it: its max_retries option has a default of 2, and 0 turns retries off.

The same README says requests time out after 10 minutes by default, and you can set your own timeout.

SDK, framework or raw HTTP

You now have three ways to call a model. A framework is a bigger library that sits on top of SDKs and adds ready-made parts for chains of steps, tools and memory. Later lessons look at some. Here is how to choose.

  • Raw HTTP

    You build the request yourself, as in our example. Pick it when there is no SDK for your language, when you need a tiny program with no extra libraries, or when you want to see exactly what goes over the wire. You write retries, errors and streaming yourself.

  • The provider's SDK

    You get types, retries and errors for the one provider. Pick it for most apps that talk to one or two providers. It stays close to the provider's docs, so new features arrive here first.

  • A framework

    You get many providers behind one shape, plus pieces for bigger flows. Pick it when you really switch providers often, or when your app has many moving parts. You also take on its size and its own rules.

A simple rule: start with the smallest thing that works. Most first apps need one SDK and no framework. Move up when you feel a real pain, not because a tutorial used a framework.

Quick check

You are building a small app that talks to one provider. What is a sensible first choice?

Pin versions and read the changelog

A library changes. Providers add features, rename things and drop old ones. If your app installs whatever is newest on the day, a quiet upgrade can break it on a day you did not choose.

The OpenAI README says its package "generally follows SemVer conventions", with a few exceptions. SemVer is a way of numbering versions so the numbers hint at how big a change is. The README lists the exceptions: changes that affect only static types, changes to internals that are technically public but not meant for outside use, and changes not expected to affect most users. So even with that policy, a small version bump can change something you rely on. The Anthropic README, for its part, points readers upgrading from a 0.x release to a v1 migration guide. That shows version jumps can need real changes in your code.

  1. 1

    Pin the version

    Write the exact version in your requirements file, so every install gets the same code.

  2. 2

    Read the changelog before you upgrade

    A changelog is the list of what changed in each release. Read it, and the migration guide if one exists.

  3. 3

    Upgrade on purpose

    Change one thing at a time, run your checks, then commit. Do not upgrade on the day of a launch.

  4. 4

    Check the version you are running

    The OpenAI README shows that you can print openai.__version__ at runtime. If new features seem missing, your environment may still be on an older version.

The same care applies to model names. A model name in a snippet is a snapshot. Treat it as a setting you can change, keep it in one place, and check the provider's docs for the current list.

Recap

  • An SDK is plumbing

    It adds types, retries, streaming helpers and errors as exceptions, as the READMEs describe. It does not change the model.

  • Keep the key safe

    Put it in an environment variable, never in your code, and mask it before you print it.

  • Choose small, then pin

    Start with the provider's SDK, move to a framework only when you need one, and pin the version.

Quick check

Why do you pin the version of an SDK?

The Tokenization lesson in Machine Learning Foundations goes deeper on tokens, the unit those calls are counted and billed in.

Next: running models locally, a model on your own laptop.

Sources

How is this lesson?

Last updated on 16 November 2026

Learn with me

Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.