esc
  • WorkflowsPages
  • Case studiesPages
  • Machine Learning FoundationsLearn
  • AI EngineeringLearn
  • AI Guardrails & Red TeamingLearn
  • UI/UXLearn
  • Design SystemsLearn
  • Production System DesignLearn
  • Book a callContact
  • Download ResumeContact
Let’s talk

I’ll build the agent for it.

AI engineering / Beginner / 01.30

Vector DBs

How a vector database keeps embeddings with their text and tags, finds the closest ones fast, and when to use a database you already have.

14 min readFree

A place to keep your embeddings

The last lesson, Embeddings, ended with a promise: next, vector databases, storing embeddings so you can search them fast. Here it is.

An embedding is a list of numbers that stands for the meaning of a piece of text. You made some in the last lesson. Now imagine you have ten thousand of them, one for every help article on your site. A customer types a question. You turn the question into an embedding too. Which of your ten thousand is closest? You need somewhere to keep them, and a quick way to ask.

Think of a huge library where books are shelved by topic instead of by title. You walk in with a question, and the librarian walks you straight to the shelf where similar books sit. A vector database is that library. By the end you will know what it stores, why searching it can be done two ways, how filters and deletes work, and how to choose between a new store and one you may already have. You will also run a tiny store of your own.

Key words

Vector
A list of numbers. An embedding is a vector.
Vector database
A database built to keep vectors and find the ones closest to a vector you give it.
Metadata
Extra facts stored beside a vector, like a date, a category or an author. It is data about the data.
Nearest neighbour search
Finding the stored items closest to your query. The closest ones are the nearest neighbours.
Exact search
Comparing the query with every stored vector, so the answer is always the true closest one.
Approximate search
Using a shortcut that skips most vectors. It is much faster, and it may miss a true closest match now and then.
Index
An extra structure built beside the data so a search can skip most of it, like the index at the back of a book.
Recall
Of the true closest matches, the share your search actually found. Perfect recall means none were missed.

What it stores

The Wikipedia page on vector databases says one is "a database that stores and retrieves embeddings of data in vector space". Plain words: you put vectors in, you ask for the closest ones back.

But a vector alone is not much use. It is just numbers, and you cannot show numbers to a customer. So each entry keeps a few things together.

  • The vector

    The embedding itself. This is what the search compares.

  • The text and an id

    The original words (or a link to them) and a short name for the entry. The Wikipedia page describes the usual setup for answering questions from documents: each document's vector is stored "along with a link to the document".

  • Metadata

    Tags such as language, date or which customer the entry belongs to. They let you narrow the search later.

One thing to keep in mind. The vectors only make sense next to the model that made them. The goal, in the page's words, is "that semantically similar data items receive feature vectors close to each other". If you switch to a different embedding model, you must make new vectors for everything. Old and new numbers do not mix.

How closeness is measured

"Closest" needs a ruler. A common one for text is cosine similarity. It asks how much two lists of numbers point the same way. Two vectors pointing the same way score 1. Two at right angles score 0.

cosine(a,b)=a⋅b∥a∥ ∥b∥\text{cosine}(a, b) = \frac{a \cdot b}{\lVert a \rVert \, \lVert b \rVert}

In words: multiply the two lists number by number and add it all up (that is the dot product, the top of the fraction). Then divide by the length of each list so that long vectors do not win just for being long. Cosine is one choice. Euclidean and Manhattan distance are others.

Exact search: check everything

The simplest search has no cleverness. Compare the query with every stored vector and keep the best so far. Wikipedia's page on nearest neighbour search describes it as computing "the distance from the query point to every other point in the database, keeping track of the 'best so far'".

It has two good points. The answer is always the true closest match. And there is nothing extra to build or keep up to date. The page adds that this simple way "has no space complexity beyond the storage of the database", which means it needs no extra storage.

Its weakness is the work. If you have N vectors with d numbers each, the page gives the running time as O(dN). That is a short way of saying the time grows in step with N times d. Double the vectors and you double the work.

Below is a tiny vector store in plain Python. It has add, delete and search with cosine. Search can take an optional filter on the metadata. The eight vectors have only four numbers each, and they are made up so you can see the idea. A real embedding has far more numbers (Wikipedia says from a few hundred to tens of thousands), and a real model would make them. The four axes are invented too: fruit, tech, outdoors and food.

Python
import math

class VectorStore:
    def __init__(self):
        self.items = {}  # id -> (vector, text, metadata)

    def add(self, item_id, vector, text, metadata=None):
        # Adding with an id that already exists replaces the old entry.
        self.items[item_id] = (vector, text, metadata or {})

    def delete(self, item_id):
        self.items.pop(item_id, None)

    def search(self, query, k=3, where=None):
        scored = []
        for item_id, (vector, text, meta) in self.items.items():
            if where and any(meta.get(key) != value for key, value in where.items()):
                continue
            scored.append((cosine(query, vector), item_id, text))
        scored.sort(reverse=True)
        return scored[:k]

def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))

# Made-up vectors. Axes (invented): [fruit-ness, tech-ness, outdoors-ness, food-ness]
store = VectorStore()
store.add("a1", [0.9, 0.0, 0.1, 0.8], "Apple pie recipe", {"kind": "food"})
store.add("a2", [0.8, 0.1, 0.0, 0.2], "How to pick a ripe mango", {"kind": "food"})
store.add("a3", [0.1, 0.9, 0.0, 0.0], "Choosing a new laptop", {"kind": "tech"})
store.add("a4", [0.0, 0.8, 0.2, 0.0], "Phone battery tips", {"kind": "tech"})
store.add("a5", [0.1, 0.0, 0.9, 0.1], "Best trails for a day hike", {"kind": "travel"})
store.add("a6", [0.0, 0.1, 0.8, 0.3], "Camping food ideas", {"kind": "travel"})
store.add("a7", [0.2, 0.7, 0.1, 0.1], "Apple unveils a phone", {"kind": "tech"})
store.add("a8", [0.7, 0.0, 0.1, 0.9], "Baking with fresh fruit", {"kind": "food"})

query = [0.8, 0.1, 0.0, 0.5]  # a made-up query about "fruit you can eat"

print("No filter:")
for score, item_id, text in store.search(query, k=3):
    print(f"  {score:.3f}  {item_id}  {text}")

print("Only kind = tech:")
for score, item_id, text in store.search(query, k=3, where={"kind": "tech"}):
    print(f"  {score:.3f}  {item_id}  {text}")

store.delete("a1")
store.add("a8", [0.7, 0.0, 0.1, 0.9], "Baking with fresh fruit (updated)", {"kind": "food"})
print("After a delete and an update:")
for score, item_id, text in store.search(query, k=3):
    print(f"  {score:.3f}  {item_id}  {text}")
print(len(store.items), "items left")

Output

No filter:
  0.977  a1  Apple pie recipe
  0.952  a2  How to pick a ripe mango
  0.930  a8  Baking with fresh fruit
Only kind = tech:
  0.398  a7  Apple unveils a phone
  0.198  a3  Choosing a new laptop
  0.102  a4  Phone battery tips
After a delete and an update:
  0.952  a2  How to pick a ripe mango
  0.930  a8  Baking with fresh fruit (updated)
  0.398  a7  Apple unveils a phone
7 items left

Read the output from the top. With no filter, the closest three are all food, and "Apple pie recipe" scores highest at 0.977. Note that "Apple unveils a phone" is not in the top three even though it shares the word "apple". Vectors match on meaning, not on shared words. That is the whole point.

With the filter kind = tech, the store skips everything that is not tagged tech before it scores anything. The best tech item scores only 0.398, which shows that a filter can force you to accept weaker matches. Last, we deleted "a1" and re-added "a8" with the same id. The old entry was replaced, and now seven items are left.

What a search does, step by step

  1. 1

    Embed the query

    Turn the user's question into a vector with the same model that made the stored ones.

  2. 2

    Filter

    Optionally keep only entries whose metadata matches, such as one language or one customer.

  3. 3

    Compare

    Score the query against each remaining vector, or against the few that an index points to.

  4. 4

    Rank and return

    Sort by score, take the top few, and hand back their ids, text and metadata.

Tip

Always keep the original text next to the vector. The vector is for finding. The text is what you show the user or give to a model.

When checking everything gets slow

Our eight-item store is instant. Real stores are not that small. The next program makes 20,000 random vectors with 64 numbers each and a fixed seed, so every run makes the same ones. It then does one exact search by comparing the query with all of them. It times the search too, but the time is not printed, because it changes from one computer to the next.

Python
import math
import random
import time

random.seed(7)
dims = 64
vectors = [[random.random() for _ in range(dims)] for _ in range(20000)]
query = [random.random() for _ in range(dims)]

def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    return dot / (math.sqrt(sum(x * x for x in a)) * math.sqrt(sum(y * y for y in b)))

start = time.perf_counter()
best_score, best_index, searched = -1.0, -1, 0
for index, vector in enumerate(vectors):
    score = cosine(query, vector)
    searched += 1
    if score > best_score:
        best_score, best_index = score, index
seconds = time.perf_counter() - start  # measured, but not printed: it differs on every computer

print("vectors searched:", searched)
print("positions compared per query:", searched * dims)

Output

vectors searched: 20000
positions compared per query: 1280000

One search lined up 1,280,000 positions, which is 20,000 vectors times 64 numbers. Run it yourself and watch how long it takes on your computer. Now picture ten million vectors. That is 640,000,000 positions for every single question, and it grows with every vector you add.

Approximate search: skip most of it

An approximate search does not promise the true closest vector. It promises one that is almost as close, found far faster. Wikipedia's page on nearest neighbour search says such an algorithm may not return the actual nearest neighbour every time, "in return for improved speed or memory savings". It adds that in many cases an approximate nearest neighbour "is almost as good as the exact one".

Picture a big city and a question like "where is the nearest bakery?". Exact search walks every street. Approximate search starts on the main roads, heads toward your area, then checks only the side streets there. You almost always find a great bakery. Very rarely, a closer one on a street you skipped is missed.

The Wikipedia page on vector databases says they "typically implement approximate nearest neighbor algorithms", and unlike traditional databases, which "primarily look up records by exact match", they let you find records that are semantically similar to a given input. It names several families of techniques, among them graph methods, locality-sensitive hashing, product quantization and inverted files.

Key words

HNSW
Short for Hierarchical Navigable Small World. A graph method: stored vectors are linked to their close neighbours, in several layers.

The most talked-about method has a paper behind it. Its abstract says HNSW "incrementally builds a multi-layer structure consisting from hierarchical set of proximity graphs (layers)". In plain words: dots joined to nearby dots, stacked in layers. A search starts at the top layer, which has few dots and long links (like main roads), then drops layer by layer to the bottom, which has every dot and short links (like side streets). The abstract says starting from the upper layer allows "a logarithmic complexity scaling", so the work grows slowly as the data grows. It also notes that the design is similar to a skip list.

There is a price. An index must be built, it takes time and memory to keep up to date, and the answer can differ from an exact search. The pgvector README says it directly: by default it "performs exact nearest neighbor search, which provides perfect recall", and you can add an index to use approximate search, "which trades some recall for speed". It also warns that, unlike typical indexes, "you will see different results for queries after adding an approximate index".

Tip

A small collection, say a few thousand entries, may not need an index at all. Exact search is simple, always right, and fast enough. Add an index when the search starts to feel slow.

Metadata filters

Most real questions have a condition. "Show me articles about refunds, but only in English." "Only this customer's files." That is a metadata filter, and it is how you keep one customer from ever seeing another's text.

Filtering and an approximate index do not always fit together neatly. The pgvector README explains one case: with an approximate index, "filtering is applied after the index is scanned". Its example is worth reading slowly. If a condition matches 10% of rows, with HNSW and the default hnsw.ef_search of 40, "only 4 rows will match on average". Of 40 candidates the index looked at, only about 4 survive your filter.

The README names ways to cope: an ordinary index on the filter column, which it says "can provide fast, exact nearest neighbor search in many cases", and iterative index scans, which "automatically scan more of the index when needed". Other products handle filters in their own ways. The lesson to take away is simple. Test your filters with real questions, and count how many results come back.

Updates and deletes

Your content changes. A help article is rewritten. A customer asks you to remove their files. A vector database has to cope with that, and your side of the job is to keep three things in step.

  • Rewritten text

    Make a new embedding and replace the old entry under the same id.

  • Removed content

    Delete the entry by its id, as our store did for "a1". In pgvector it is the ordinary SQL line DELETE FROM items WHERE id = 1.

  • A changed model

    Re-embed everything with the new model. Vectors from two different models are not comparable.

Tip

Give every entry an id you can trace back to the source document. Without one, you cannot find the right entry to update or delete.

Choosing: extension or dedicated store

There are two broad paths, and neither is the "right" one for everybody. No ranking here. Specific products get their own Core lessons.

The first path is to add vector search to a database you already use. pgvector is an example. Its README calls it "open-source vector similarity search for Postgres" and says "store your vectors with the rest of your data". It lists perks of staying inside Postgres: "ACID compliance, point-in-time recovery, JOINs, and all of the other great features of Postgres". The second path is a dedicated vector store, a system built only for vectors. The Wikipedia page lists many, and it notes that its own list is a dynamic one and may never be complete.

  • Add-on to a database you have

    Fewer moving parts. Your vectors sit beside the rest of your data, so one query can mix a vector search with normal conditions. Your team already knows the tool, and your usual backups and permissions still apply.

  • Dedicated vector store

    Built around vector search from day one, so the features you need for large collections, filters and index tuning may be richer. It is one more system to run, pay for, secure and keep in step with your main data.

  • A simple file or in-memory store

    Fine for learning and small tools, like the class in this lesson. You lose the safety features of a real database, so move up when the data matters.

If you are unsure, start with what you already run. Moving later means re-loading the same vectors, which is easy if you kept the original text and ids.

Quick checks

Quick check

A search skips most vectors and returns an answer that is almost as close as the true one. What is it called?

Quick check

You rewrite a help article. What should you do with its entry in the vector store?

Next

Next: retrieval-augmented generation, answering from your own documents.

Sources

How is this lesson?

Last updated on 16 November 2026

Learn with me

Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.