esc
  • WorkflowsPages
  • Case studiesPages
  • Machine Learning FoundationsLearn
  • AI EngineeringLearn
  • AI Guardrails & Red TeamingLearn
  • UI/UXLearn
  • Design SystemsLearn
  • Production System DesignLearn
  • Book a callContact
  • Download ResumeContact
Let’s talk

I’ll build the agent for it.

Machine Learning Foundations / Beginner / 01.52

Preprocessing: getting data ready for a model

Models only do arithmetic. Preprocessing is how raw, messy data becomes a clean table of numbers, and the one rule that keeps your results honest.

14 min readFree

What preprocessing is

Think about cooking. You do not toss a whole muddy carrot into a pot. You wash it, peel it, cut it and measure it first. Data is the same. Raw data arrives muddy: missing cells, words where numbers should be, columns that use different units. A model cannot cook with that. Getting it ready is called preprocessing.

The scikit-learn user guide, a well-known Python toolkit for machine learning, describes its preprocessing tools like this: they 'change raw feature vectors into a representation that is more suitable for the downstream estimators.' In plain words, preprocessing turns raw data into a clean table of numbers that a model can learn from.

The last lesson covered file formats. This one is the map of what happens next. Later lessons zoom into cleaning, feature engineering, scaling, selection and dimensionality reduction, so here each gets a short introduction and no more.

Key words

Preprocessing
Everything you do to raw data before a model sees it, so that it becomes a clean table of numbers.
Feature
One column of the table: a single thing you know about each row, such as the number of rooms in a house.
Categorical
A column whose values are labels from a list, such as a city name or a size of small, medium or large.
Encoding
Turning labels or other non-number values into numbers.
Training data / test data
Training data is what the model learns from. Test data is kept aside and used afterwards to check the model on rows it has never seen.
Data leakage
When information from the test data sneaks into the model's learning, which makes the results look better than they really are.
Pipeline
A fixed chain of steps that always runs in the same order, so you can repeat it exactly.

Why models need it

A model is a machine that does arithmetic. Words like 'Leeds' or 'large' mean nothing to it, and a gap where a number should be breaks the sum. Google's Machine Learning Crash Course says it plainly: 'Machine learning models can only manipulate floating-point numbers.' A floating-point number is just a number with a decimal point, like 3.0 or 0.25.

There is a second reason: scale. A house has 3 rooms but costs 250,000. The scikit-learn guide warns that if a feature 'has a variance that is orders of magnitude larger than others, it might dominate the objective function and make the estimator unable to learn from other features correctly as expected.' A column of huge numbers can drown out a column of small ones.

So preprocessing has two jobs: make everything a number, and make those numbers behave well together.

Tip

Whenever a step 'learns' something from the data, ask: learned from which rows? The golden rule below is the answer.

The usual steps

Every project differs, but the journey has a common shape. Here it is as a walk-through.

  1. 1

    Inspect

    Look before you touch. How many rows and columns? What type is each column? Which cells are empty? Which values look odd?

  2. 2

    Split

    Set some rows aside as test data before you learn anything from the data. More on why in a moment.

  3. 3

    Clean

    Fix or fill the empty cells, remove duplicates, correct obvious mistakes. The next lesson, data cleaning, goes deep here.

  4. 4

    Transform and encode

    Turn labels, text and dates into numbers. Create new useful columns from old ones. This is the heart of feature engineering.

  5. 5

    Scale

    Bring numeric columns onto similar ranges so that none shouts over the others.

  6. 6

    Select or reduce

    Keep the columns that help and drop the ones that do not, or squeeze many columns into fewer.

Notice that Split comes early, before any step learns from the data. The golden rule below explains why.

Here is the first step on a tiny made-up table of three houses. Each row is a dictionary: a name for each value. The program counts the rows and finds the missing number. In Python, a missing value is often written None.

Python
from statistics import correlation, pvariance

table = {
    "size_m2":   [50, 62, 75, 80, 95, 110, 120, 140],
    "size_ft2":  [538, 667, 807, 861, 1023, 1184, 1292, 1507],
    "rooms":     [2, 3, 3, 4, 4, 5, 4, 6],
    "country":   [1, 1, 1, 1, 1, 1, 1, 1],
    "door_code": [7, 2, 9, 4, 1, 8, 3, 5],
}
price = [150, 190, 220, 250, 280, 340, 350, 420]

print("variance of each feature")
for name, col in table.items():
    print(f"{name:10} {pvariance(col):10.2f}")

print()
print("correlation with price")
for name, col in table.items():
    if pvariance(col) > 0:
        print(f"{name:10} {correlation(col, price):6.3f}")

print()
print("size_m2 vs size_ft2:", round(correlation(table["size_m2"], table["size_ft2"]), 3))
print("size_m2 vs rooms:", round(correlation(table["size_m2"], table["rooms"]), 3))

Output

variance of each feature
size_m2        814.50
size_ft2     94471.36
rooms            1.36
country          0.00
door_code        7.36

correlation with price
size_m2     0.997
size_ft2    0.997
rooms       0.943
door_code  -0.090

size_m2 vs size_ft2: 1.0
size_m2 vs rooms: 0.918

Row number 1 has no value for rooms. Counting starts at 0, so the second row is index 1. Spotting this is inspection. What to do about it is cleaning, and that is the next lesson.

Quick check

A column in your table says 'small', 'medium' or 'large'. What kind of column is this?

Encoding categories

Categorical columns are the most common reason a table is not ready. The scikit-learn guide notes that such features 'can be efficiently coded as integers', and gives an example of a person described as 'male', 'from US', 'uses Internet Explorer' becoming [0, 1, 3]. There are two main ways to encode them, and the difference matters.

  • Ordinal encoding

    Give each label a whole number, such as 0, 1 and 2. This is right only when the labels really have an order, like small, medium, large. The scikit-learn guide warns that integer codes make estimators 'interpret the categories as being ordered, which is often not desired' when the order is arbitrary.

  • One-hot encoding

    Make one new column per label. A row gets a 1 in its own label's column and 0 in all the others. No order is implied. Google's course describes it as turning categorical values into vectors where each category is represented by a unique element with a value of 1.

  • Why not just numbers for everything?

    Google's course gives the risk: if you leave the labels as indexed integers, 'the model would treat the indexed values as continuous floating-point numbers. The model would then consider "purple" six times more likely than "orange."' A code is not a size.

Here is both ideas on tiny lists. The first loop one-hot encodes four house cities. The second part gives sizes an ordered code, because small, medium and large do have an order.

Python
from statistics import correlation

price = [150, 190, 220, 250, 280, 340, 350, 420]
price_with_tax = [round(p * 1.2) for p in price]
size_m2 = [50, 62, 75, 80, 95, 110, 120, 140]
print("size_m2 vs price:", round(correlation(size_m2, price), 3))
print("price_with_tax vs price:", round(correlation(price_with_tax, price), 3))

Output

size_m2 vs price: 0.997
price_with_tax vs price: 1.0

The first line is the column order: Hull, Leeds, York. Leeds is [0, 1, 0] because the 1 sits in its column. The sizes come out as [0, 2, 1, 0]: small is 0, medium 1, large 2.

One-hot encoding has a cost: a column with thousands of different labels would become thousands of new columns. Google's course says that for features with many categories 'embeddings or hashing are recommended', and that a one-hot table is mostly zeros, so it can be stored by recording only the position of the 1. You will meet those ideas in the feature engineering lesson.

Quick check

A column holds three cities with no natural order. Which encoding avoids telling the model that one city is 'bigger' than another?

Text and dates, in short

Text and dates are the two other kinds of column you meet often. Both need turning into numbers too. This is only the first look; feature engineering comes later.

  • Text

    The scikit-learn guide says that 'the raw data, a sequence of symbols, cannot be fed directly to the algorithms themselves as most of them expect numerical feature vectors with a fixed size.' Its basic fix is called a bag of words: split text into words, then count how often each word occurs, 'while completely ignoring the relative position information of the words'.

  • Dates

    A date as one chunk of text is not useful to a model. A common idea is to pull out the parts that matter, such as the year, the month and the day of the week. The scikit-learn guide notes that datasets often have 'features of different types, say text, floats, and dates, where each type of feature requires separate preprocessing'.

  • Same goal

    Different tools, same goal: a plain number in every cell.

This sample does both by hand with the standard library. Python's datetime module gives the date parts; weekday() counts Monday as 0. The word counts line up with a sorted word list.

Python
from statistics import correlation

rooms = [2, 3, 3, 4, 4, 5, 4, 6]
door_code = [7, 2, 9, 4, 1, 8, 3, 5]
price = [150, 190, 220, 250, 280, 340, 350, 420]

train = slice(0, 6)  # the first 6 houses are the training rows
for name, col in [("rooms", rooms), ("door_code", door_code)]:
    r = correlation(col[train], price[train])
    print(f"{name:10} r on training rows = {r:6.3f}")

Output

rooms      r on training rows =  0.977
door_code  r on training rows =  0.019

So 16 November 2026 becomes the numbers 2026, 11, 16 and 0, and it is a Monday. The sentence becomes a list of counts, one slot per word: 'the' appears twice and every other word once.

The golden rule: learn from training data only

Now the most important idea in this lesson. Some preprocessing steps learn something from the data. A scaler learns the smallest and largest value. A filler of missing cells learns the average. An encoder learns which labels exist. The word for learning this is fit. Using what was learned is transform.

The scikit-learn guide explains the danger, called data leakage: 'Data leakage occurs when information that would not be available at prediction time is used when building the model. This results in overly optimistic performance estimates.' Overly optimistic means it looks good in your notebook and then disappoints in real life.

It then states the rule: 'The general rule is to never call fit on the test data.' And it gives the reason in terms of an average: 'if you have a normalization step where you divide by the average value, the average should be the average of the train subset, not the average of all the data. If the test subset is included in the average calculation, information from the test subset is influencing the model.'

An everyday picture: a student who sees the exam questions while studying will score well and prove nothing. The same guide lists how to avoid leakage. First, 'always split the data into train and test subsets first, particularly before any preprocessing steps.' Second, use fit_transform on the training part and transform on the test part.

The sample below shows a tiny scaler that squeezes numbers to the range 0 to 1. The ten values are split into 8 for training and 2 for testing. First we learn the smallest and largest from the training rows only. Then, for contrast, the leaky version learns them from all ten.

Python
def fit_scaler(values):
    lo, hi = min(values), max(values)
    return lo, hi

def apply_scaler(values, lo, hi):
    return [round((v - lo) / (hi - lo), 2) for v in values]

data = [10, 20, 30, 40, 50, 60, 70, 80, 90, 100]
train, test = data[:8], data[8:]

lo, hi = fit_scaler(train)
print("fit on train only:", lo, hi)
print("train:", apply_scaler(train, lo, hi))
print("test: ", apply_scaler(test, lo, hi))

lo2, hi2 = fit_scaler(data)
print("leaky, fit on all:", lo2, hi2)
print("test: ", apply_scaler(test, lo2, hi2))

Output

fit on train only: 10 80
train: [0.0, 0.14, 0.29, 0.43, 0.57, 0.71, 0.86, 1.0]
test:  [1.14, 1.29]
leaky, fit on all: 10 100
test:  [0.89, 1.0]

Tip

Order of work: split first, fit on the training part, then apply the same fitted steps to the test part.

With the correct approach, the scaler learned 10 and 80 from training data. The two test values, 90 and 100, scale to 1.14 and 1.29, above 1, and that is fine and honest: the model really is meeting numbers it has not seen. The leaky version learned 100 from the test set, so the test values land neatly inside 0 to 1 at 0.89 and 1.0. It hid the surprise, which is the leak.

Quick check

You want to fill missing cells with the average. Which average do you use for the test rows?

Keep it repeatable with pipelines

Preprocessing is a recipe, and recipes must be repeatable. By hand, you can forget a step, run steps out of order or fit on the wrong rows. And new data later needs exactly the same treatment.

The answer is a pipeline: a chain of steps run in a fixed order. The scikit-learn documentation says a Pipeline 'can be used to chain multiple estimators into one', and lists three benefits. Convenience: 'You only have to call fit and predict once on your data to fit a whole sequence of estimators.' Safety: 'Pipelines help avoid leaking statistics from your test data into the trained model in cross-validation, by ensuring that the same samples are used to train the transformers and predictors.' Cross-validation is a way of testing a model several times on different pieces of the data.

Every step except the last must be a transformer, meaning it can transform data; the last can be anything. A companion tool, ColumnTransformer, gives different columns different treatment.

scikit-learn is not installed here, so the next block is not run. This is how the library's own documentation shows leakage-safe preprocessing. It is the documentation's example, shown without output.

Text
# from the scikit-learn user guide (Common pitfalls), not run here
from sklearn.pipeline import make_pipeline
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
pipeline = make_pipeline(SelectKBest(k=25),
                         HistGradientBoostingClassifier(random_state=1))
pipeline.fit(X_train, y_train)
y_pred = pipeline.predict(X_test)

The idea is simple enough to build yourself. This toy pipeline in plain Python has two steps: fill a missing number with the mean, then halve every value. It learns the mean from [2, None, 6], which is 4, then is applied to new data [None, 10] without learning again.

Python
class Pipe:
    def __init__(self, steps):
        self.steps = steps
    def fit_transform(self, rows):
        for step in self.steps:
            rows = step.fit(rows).transform(rows)
        return rows
    def transform(self, rows):
        for step in self.steps:
            rows = step.transform(rows)
        return rows

class MeanFill:
    def fit(self, rows):
        known = [r for r in rows if r is not None]
        self.mean = sum(known) / len(known)
        return self
    def transform(self, rows):
        return [self.mean if r is None else r for r in rows]

class Halve:
    def fit(self, rows):
        return self
    def transform(self, rows):
        return [r / 2 for r in rows]

pipe = Pipe([MeanFill(), Halve()])
print(pipe.fit_transform([2, None, 6]))
print(pipe.transform([None, 10]))

Output

[1.0, 2.0, 3.0]
[2.0, 5.0]

The training run gives [1.0, 2.0, 3.0]. The new data becomes [2.0, 5.0], because its gap was filled with the training mean of 4. Same recipe each time: that is what a pipeline guarantees.

What comes next

Each later lesson widens one part of the path.

  • Data cleaning

    Missing values, duplicates, outliers (very odd values) and typos. This is the next lesson.

  • Feature engineering, scaling, and selection

    Making useful new columns, bringing numbers to a common range, and choosing which columns to keep.

  • Dimensionality reduction

    Squeezing many columns into fewer while keeping what matters. Useful when one-hot encoding has made the table very wide.

Quick check

Why does the guide recommend a pipeline for preprocessing?

Next: data cleaning.

Sources

How is this lesson?

Last updated on 16 November 2026

Learn with me

Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.