Key-Value Store: Fast Lookups by Key, and Why Queries Are Hard
Learn how a key-value store saves data under unique keys, why it is so quick, where it falls short, and how to tell when it fits a design.
In short
A key-value store saves each value under a unique key, so one key finds one value quickly, but it is weak at searching across many records.
10 min readFree
What You Will Learn
This lesson explains the key-value store. Many system designers pick this tool when the main job is to fetch one item by its name, such as a user's settings or a visitor's shopping cart. You will learn what a key and a value are, why this design is so quick, where it falls short, and how to tell when it fits a problem.
Key words
- Database
- A system that saves data so programs can read it later and change it.
- Key
- A unique name for one piece of data, like a label on a box. No two keys are the same.
- Value
- The data stored under a key. It can be a number, a line of text, or a block of JSON (a text format for structured data).
- Lookup
- Asking the store for the value that belongs to one key.
- Hash map
- A table that jumps straight to a storage slot using a number made from the key, so it does not check every entry one by one.
- O(1)
- A way to say that the time to do something stays about the same no matter how much data exists. Average O(1) means this holds in most cases.
- In-memory
- Kept in the computer's main memory (RAM). This is faster than a disk but usually holds less data.
- Query
- A question sent to a database, such as 'find every order over $500'.
- SQL
- The common language for asking questions of databases that store data in tables.
- Session
- The saved details about one visitor's current visit, such as who is logged in and what is in the cart.
- Cache
- A fast copy of data kept close to where it is used, so the slow original does not have to be read every time.
What Is the Basic Model?
Picture a giant dictionary. You give it a word, and it hands back the entry for that word. A key-value store works the same way. Each record is a pair with one key and one value. The key must be unique, so there is exactly one place to look for any value. <cite index="1-4">A key-value store is a database that stores data as pairs of keys and values.</cite>
The store does not care what the value holds. Whatever you put in, the store treats it as one sealed package. <cite index="6-10">You can store JSON, a serialized Python object, or binary data, and the database does not care.</cite> Because the store never looks inside, it cannot tell you what is in the value unless your program reads it first.
Most key-value stores offer only a few operations. The three most common are shown below.
Put
Save a value under a key. If the key already holds a value, the new value replaces the old one.
Get
Fetch the value for a key.
Delete
Remove the key and its value from the store.
Because every request names one exact key, the work is small and predictable. <cite index="6-15">Key-value databases provide the simplest data model optimized for direct key lookups with O(1) access performance for point reads.</cite> A point read means fetching one item by its key. The Python sample below uses a plain dictionary, which is Python's built-in hash map, to show the three operations.
store = {}
def put(key, value):
store[key] = value
def get(key):
return store.get(key)
def delete(key):
store.pop(key, None)
put("session:8f3a21", {"user_id": 42, "cart": ["book", "pen"]})
print(get("session:8f3a21"))
delete("session:8f3a21")
print(get("session:8f3a21"))Output
{'user_id': 42, 'cart': ['book', 'pen']}
None
Tip
Read the key as a question the app asks: 'Give me the thing called this.' If your app needs a different question, such as 'give me every item above a price', a key-value store is probably the wrong choice.
Why Is It Fast and Easy to Grow?
The speed comes from the simple request. The app sends one key, and the store finds one value. <cite index="19-5">The store looks for a single key and returns its value, instead of running complex calculations over many records.</cite> Less work per request means more requests can be answered each second.
Growth is also simpler to plan. A key-value store needs no fixed table layout, so a new kind of value can be saved without changing the structure first. Each value is also found by its own key, which makes it easier to share the data out across several machines. The next two sub-topics explain the two parts of this: where the data lives, and how it spreads out.
Where Does the Data Live?
Some key-value stores keep their data in memory, and some keep it on disk. <cite index="1-14">In-memory stores like Redis and Memcached are fastest, while persistent stores like DynamoDB and etcd are the lasting home for data.</cite> Redis is a useful middle case, because <cite index="1-9">Redis can optionally persist data to disk</cite>, so its data can survive a restart when that option is on. Choose memory when speed matters most and losing the data is acceptable, or choose disk when the data must last.
How Are Keys Spread Across Machines?
Each key can be turned into a number by a formula called a hash. The hash shows which machine holds that key, so different machines can hold different keys. <cite index="9-1">Key-value stores are often described as distributed hash tables.</cite> A single machine has limits. Spreading the keys helps with all three.
Be careful not to overstate how easy this is. Key-value stores scale out by partitioning keys across machines. The idea is simple to state, but the details take real engineering. Those details belong to a later lesson on sharding.
Worked Example: A Shopping Session in Redis
Here is one real pattern from an online shop. When a visitor logs in, the site keeps a session, which is the record of who they are and what is in their cart. The site stores that session in Redis, an in-memory key-value store. Follow the steps to see how the idea is used on each page.
- 1
Create the session
The visitor logs in as user 42 and adds a book and a pen. The app builds one record holding the user ID and the cart items.
- 2
Choose a unique key
The app names the record session:8f3a21. The key is unique to this visit, and the prefix 'session:' tells a reader what kind of data it is.
- 3
Put the value
The app saves the record under that key. The value is a JSON block with the user ID and the cart list. Redis does not need to know the cart's layout to store it.
- 4
Get the whole session on each page
Each page load sends one request for session:8f3a21. Redis returns the full session in one lookup, so the page has what it needs to show the user's name and cart without a second query.
- 5
Update and save again
When the visitor adds an item, the app reads the session, changes the cart list, and puts the new value back under the same key.
The commands below show the same steps in Redis's own command style. The text is shown only to illustrate the idea, not as a program to run.
SET session:8f3a21 '{"user_id": 42, "cart": ["book", "pen"]}'
GET session:8f3a21
SET session:8f3a21 '{"user_id": 42, "cart": ["book", "pen", "lamp"]}'Tip
Notice that the shop never asks Redis to search. It always knows the key, which is why the design is quick. If the shop later wanted 'every visitor with a lamp in the cart', it would need a different tool.
What Trade-offs Must You Accept?
Simple designs give up some features. The key-value model is strong at one job, so it is weak at others. These are the costs to weigh before you choose it.
No rich queries
The store cannot search inside values. <cite index="6-11">You cannot run a question like 'find all users where age is over 25' without pulling the data into your application first.</cite>
Values are a black box
The store does not describe what a value contains. <cite index="18-4">When you fetch a value by its key, you have no guarantee about what shape or content you will get back.</cite> Your code must check the data itself.
Few extra features
Some stores add small extras. These are narrow, so do not expect the wide set of tools a relational database offers.
Growth takes planning
A single machine has limits. Once data spreads across machines, you must decide how keys are placed and what happens when one machine fails. Those decisions add work.
The trade is clear. You get speed and simple growth, and you pay with weak searching. If your team needs to answer many different questions about the same data, the cost of that trade grows quickly.
When a Key-Value Store Fits
Use this design when every request names the item it wants. The following uses are common in production systems.
Common Good Fits
Sessions
Logged-in state for a visit. <cite index="17-7">Key-value works well for session management in web applications.</cite>
Caches
<cite index="14-9">Key-value stores are often used for in-memory data caching to make application responses faster.</cite>
Shopping carts
<cite index="17-3">For carts and similar data, a key-value store is cheap to use, since reading and writing the data each takes one request.</cite>
User preferences
Settings such as language or theme. <cite index="10-11">Typical use cases include session management and user preference and profile storage.</cite>
Leaderboards
Ranked scores for a game or contest. Telnyx lists <cite index="12-2">caching, session storage, configuration, feature flags, and real-time leaderboards</cite> among the main uses.
Poor Fits
Avoid this design when the main job is reporting or searching across many records. Data with links between tables, such as customers, orders, and products that must be combined in one report, is a poor match. The store can only fetch by key, so a question like 'every order above $500 from last week' would force the app to read far too much data. That kind of work fits a database built for queries.
Quick check
A game must show each player's profile settings right away, by player ID, millions of times per minute. It never needs to search across players by those settings. Would a key-value store fit, and why?
Quick check
A web shop wants a report of every order above $500 placed last week. Should its main database be a key-value store?
Key Takeaways
The model
Each unique key maps to one value. The store does not look inside the value, and the common operations are put, get, and delete.
The speed
Fetching by an exact key is quick and predictable, so lookups are usually O(1) on average.
The limits
You cannot search by value contents without reading the data into your app, so plan queries carefully.
The fit
Choose it for sessions, caches, carts, preferences, and leaderboards, where every request names the key it needs.
Tip
Before you pick a key-value store, write down the exact questions your app asks. If every question starts with a known key, the design is a good match. If you find yourself asking 'which items match this rule?', look at a database built for queries.
Sources
- Algomaster: Key-Value Stores
- Redis: What is a key value database? A complete guide
- Telnyx: What is a key value store and when should you use one?
- KDnuggets: Key-Value Databases, Explained
- ScyllaDB glossary: What is a Key Value Database?
- InfluxData: A Guide to Key-Value Databases
- Hazelcast: Key-Value Stores Explained. Advantages & Use Cases
- Codelit: Design a Key-Value Store, From Simple Hash Map to Distributed Database
- Software Craftsperson: Design a Distributed Key-Value Store
- ByteLearn: Key-Value Store at Scale
- USPTO patent publication 12265473 (key value store description)
Last updated on 17 November 2026
Related lessons
Learn with me
Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.