esc
  • WorkflowsPages
  • Case studiesPages
  • Machine Learning FoundationsLearn
  • AI EngineeringLearn
  • AI Guardrails & Red TeamingLearn
  • UI/UXLearn
  • Design SystemsLearn
  • Production System DesignLearn
  • Book a callContact
  • Download ResumeContact
Let’s talk

I’ll build the agent for it.

Wide Column Store: Flexible Rows for Huge Write Loads

A wide column store keeps each row under one key, lets rows hold different columns, and spreads the data across many servers.

In short

A wide column store keeps each row under one key, lets rows hold different columns, and spreads the data across many servers to handle huge write loads.

10 min readFree

What You Will Learn

Some apps receive a steady flood of small records. Think of a chat app, a fitness watch, or a sensor in a factory. A wide column store is a kind of database built for this kind of load. In this lesson you will learn how its data model works, how it stores and finds data, when it is a good choice, and what you give up to get its strengths.

Key words

Wide column store
A NoSQL database that keeps each row under a key, where each row can hold its own set of named columns.
NoSQL
A family of databases that do not rely mainly on fixed tables and SQL queries. They often trade some features for speed and scale.
Row key
The label that identifies one row. The database uses it to find that row.
Column family
A named group of related columns that are stored together.
Column
One named value inside a row, such as an event time or a field name.
Sparse
Most possible columns are empty for most rows. Empty cells take no space.
Node
One server in a group of servers that work together as one database.
Join
Combining rows from two tables by matching values in them.
Ad hoc query
A question you did not plan for when you designed the tables.

What Is the Data Model?

A wide column store uses tables, but they are looser than the tables in a relational database. Every row has a row key. Columns are grouped into column families. Each column has a name and a value. A helpful picture is a map inside a map. The outer map finds a row by its key. The inner map holds that row's columns.

  • Row key

    The single label for one row. It is the main way to find data, so choosing it is the most important design step.

  • Column family

    A named bundle of related columns. Columns in one family are stored together, so an app can read them together. People often compare a family to a table in a relational database.

  • Columns

    Named values inside a family. You can write a new column to a row at any time, and another row can simply skip it.

This is where the flexibility comes from. Two rows in the same table can have completely different columns. Because empty cells are not stored, a table can be sparse without wasting space. This is why many writers describe the model as a two-level map rather than a grid of fixed columns.

Put simply, the whole table is a map from each row key to that row's own map of columns. Keep this picture in mind, because the storage and query rules below all follow from it.

How Do Columns Work Without a Fixed Schema?

A relational table needs its columns declared before any row is saved, and every row follows the same list. A wide column store lets an app add a column simply by writing it. Bigtable allows this. The cost is that nothing checks that rows look alike, so the app must handle missing columns on its own.

Tip

Flexible does not mean careless. Agree on column names and value types in your app code. Otherwise the same idea may be written in three different ways across your data.

How Is Data Stored and Reached?

A wide column store spreads its data across many nodes. It splits the data into parts, called partitions, and places each part on a set of nodes. Bigtable keeps all of its rows in key order, so rows with neighboring keys sit close together.

Writes are the strength of this design. When a write arrives, the database follows a short path that keeps it fast.

  1. 1

    Receive the write

    The request carries a row key, a column name, and a value. The node that owns that row key accepts it.

  2. 2

    Hold it in memory

    The write goes into an in-memory buffer called a memtable. Memory is much quicker to write to than disk.

  3. 3

    Add it to a log

    The write is also added to an append-only log on disk. Append-only means new entries are only added at the end, which is quick. The log also helps protect the write if the server crashes.

  4. 4

    Flush to sorted files

    When the buffer is full, its contents are written to disk as a sorted file. Each file is not edited after it is written. New changes go into new files.

Reads take more work. A read checks the memory buffer and the sorted files, then combines what it finds. In the background, the database merges older files to reclaim space. This merging step is called compaction. It is necessary, but it uses resources that the app cannot see.

Tip

A wide column store is built for writes first. If your app reads far more than it writes, check whether this design still makes sense before you commit to it.

Why Design Around Key Lookups?

The fastest reads are lookups by row key, or by a range of row keys that sit next to each other. Bigtable does not have secondary indexes, which are extra lookup paths on other fields. So a question such as find every row where the city is Paris has no quick answer on the main table. The usual fix is to store a second copy of the data in a table keyed the way that question is asked. Plan the questions first, then build the tables.

Worked Example: An Activity Feed in Cassandra

Imagine a messaging or activity app. Each user has a feed of events such as likes, replies, and new follows. The app needs one screen that shows a user's newest events. Here is how you would design that screen's table in Apache Cassandra.

  1. 1

    Pick the row key

    Use the user ID as the row key. All events for one user are kept under that key, so one lookup can find them.

  2. 2

    Make each event a timestamped column

    Each event is stored under its time. Times written in one fixed text format, such as 2026-11-01T09:05, sort correctly as plain text.

  3. 3

    Define the table

    The table below names the user ID as the partition key. The event time sorts the events inside each user's row, with the newest first.

  4. 4

    Run the screen's query

    The app asks for the newest events for one user. The database goes to the nodes that hold that user's row and reads only the first few entries. It does not search the whole cluster.

  5. 5

    Let other rows differ

    Another user may have only a follow event and a device name. Their row does not need the same columns as the first user's row.

Text
CREATE TABLE user_events (
  user_id text,
  event_time timestamp,
  event text,
  PRIMARY KEY (user_id, event_time)
) WITH CLUSTERING ORDER BY (event_time DESC);

Tip

Cassandra's query language calls each event a row inside the user's partition. The idea is the same as the timestamped column described above.

The Python sample below copies the same idea with a plain dictionary. It is not Cassandra, but it shows the two-level map and the sorted columns. It runs as is with the standard library.

Python
# A toy wide-column table: row key -> {column name: value}
table = {}

def put(row_key, column, value):
    table.setdefault(row_key, {})[column] = value

put("user:42", "2026-11-01T09:00", "liked a photo")
put("user:42", "2026-11-01T09:05", "posted a comment")
put("user:7", "2026-11-01T08:30", "followed a user")
put("user:7", "device", "phone")

# Rows can hold different columns
print(sorted(table["user:42"]))
print(sorted(table["user:7"]))

# Newest two events for user:42, newest first
events = table["user:42"]
for column in sorted(events, reverse=True)[:2]:
    print(column, events[column])

Output

['2026-11-01T09:00', '2026-11-01T09:05']
['2026-11-01T08:30', 'device']
2026-11-01T09:05 posted a comment
2026-11-01T09:00 liked a photo

Look at the output. The first user's columns sort by time, so the newest event comes last in the list. The second user's row holds a column the first row does not have, and nothing had to be declared first. In a real cluster, the same lookup by user ID goes straight to the right nodes.

When Should You Choose a Wide Column Store?

This type of database fits a specific kind of workload. It is not a general upgrade over relational databases. Many teams start with a relational database and move only the heavy data when the load demands it.

  • Huge write volume

    Many records arrive every second, and writes far outnumber reads. Examples include sensor readings, logs, and event streams.

  • Time-based data

    Readings or events are stored under a device or user plus a time, and then read back for a time range.

  • Sparse, varied records

    Different rows need different fields, and most fields are empty for most rows.

Tip

Ask one question before you choose: can I write down every question the app will ask before I build the tables? If the answer is no, a relational database is usually the safer first choice.

Trade-Offs to Accept

  • Limited joins

    Combining data from many tables on the fly is hard or not supported. You store data in the shape each screen needs, sometimes in more than one place.

  • Fixed query plan

    You design tables around known questions. A new question may mean a new table and a second copy of the data.

  • Eventual consistency

    Copies of the data can disagree for a short time after a write. A read right after a write may see older data until the copies catch up.

  • Single-row changes

    Bigtable makes changes to one row all-or-nothing, which it calls atomic. It does not offer transactions that span several rows.

Real Systems That Use This Model

  • Apache Cassandra

    An open source wide column store.

  • Apache HBase

    An open source wide column store. It is modeled on Bigtable.

  • Google Cloud Bigtable

    Google's hosted service, built on the Bigtable design that Google described in its 2006 paper. Google Search, Maps, and Gmail were among the services that used Bigtable.

Check Your Understanding

Try these two short scenarios. Each one asks you to use the ideas from this lesson.

Quick check

A fitness app collects millions of heart-rate readings per minute from smartwatches. Most screens show one user's readings for a chosen time range. Which setup fits best?

Quick check

A small online shop stores orders in a wide column store keyed by customer ID. The manager now wants a report of all orders for one product, across every customer. What is the best first step?

Key Takeaways

A wide column store keeps rows under a row key, groups columns into families, and lets each row hold its own columns. It spreads the data over many nodes and handles heavy writes well. The price is fewer joins, a query plan you must set up front, and copies of data that may briefly disagree. Choose it when your data is large, write-heavy, or time-based, and when you know your questions in advance.

Sources

How is this lesson?

Last updated on 17 November 2026

Learn with me

Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.