Machine Learning Foundations / Core / 02.15
Scikit-learn: the toolbox most ML starts with
A friendly tour of scikit-learn: what it is for, how to install it, and the one simple pattern (fit, then predict) that every tool in it shares.
13 min readFree
What scikit-learn is
Imagine a workshop wall covered in tools. A hammer, a saw, a tape measure, each hung in its own place, each used the same careful way. You do not forge a hammer every time you need a nail. You take one down and use it.
Scikit-learn is that wall, for machine learning. In the last lessons you learned what a model is, and how to get data ready for one. Now you need tools that actually build models, so you do not have to write the maths yourself. Scikit-learn is the toolbox most of this course points to.
Its own Getting Started guide describes it like this: 'Scikit-learn is an open source machine learning library that supports supervised and unsupervised learning. It also provides various tools for model fitting, data preprocessing, model selection, model evaluation, and many other utilities.' Open source means anyone can read the code and use it for free.
Its home page gives three more short facts. It offers 'Simple and efficient tools for predictive data analysis'. It is 'Built on NumPy, SciPy, and matplotlib', the libraries you met in the lesson on essential libraries. And it is 'Open source, commercially usable - BSD license', which means a company can use it too.
Key words
- Library
- A bundle of ready-made code that you borrow instead of writing yourself.
- Estimator
- Scikit-learn's word for anything that learns from data. It is the model, wrapped in a tidy object.
- Fit
- To train. You fit an estimator to data, the way a tailor fits a jacket to a person.
- Predict
- To use what was learned to guess the answer for new data.
- Transformer
- A tool that changes data instead of guessing answers, such as one that rescales a column.
- X and y
- X is the table of facts (the inputs). y is the answers you want to predict (the target).
Tip
The name is written in lowercase, scikit-learn. You install it with that name, but you import it in Python as sklearn. Both names point to the same library.
What it is for, and what it is not
The home page sorts its tools into six jobs. Here are the first four, each with the job in its own words.
Classification
'Identifying which category an object belongs to.' The page lists spam detection and image recognition as uses.
Regression
'Predicting a continuous-valued attribute associated with an object.' It lists drug response and stock prices. In plain words, regression predicts a number.
Clustering
'Automatic grouping of similar objects into sets.' It lists customer segmentation as a use. No answers are given up front.
The other two jobs are dimensionality reduction (using fewer columns) and model selection ('Comparing, validating and choosing parameters and models'). A sixth job is preprocessing, which the page calls 'Feature extraction and normalization'. You met that one in the last lesson.
Now the honest part: what it does not do. Scikit-learn is for the classic kind of machine learning, with tables of data and well-known methods. It is not the tool for deep learning. Its FAQ says: 'deep learning and reinforcement learning are currently out of scope for what scikit-learn seeks to achieve.' The FAQ adds that deep learning needs GPUs, which are chips built for heavy arithmetic. Another page in the guide agrees: 'scikit-learn offers no GPU support.'
That is not a flaw. A hammer is not a bad tool because it cannot saw. For tables of numbers, which is most business data, scikit-learn is usually the right place to start. Wikipedia even reports a 2022 Kaggle survey of nearly 24,000 people in which it was 'the most widely used machine learning framework'. Kaggle is a site where people compete on data problems.
Installing it
The install page lists several ways to get scikit-learn. About the first one, installing the latest official release, it says: 'This is the best approach for most users.' On a Mac or Windows PC with Python, that means pip, Python's own installer.
The docs recommend a virtual environment first. That is a private folder of Python packages for one project, so that projects do not trip over each other. The install page says it is 'optional but strongly recommended'. This is their recipe for macOS, from the installing page.
python -m venv sklearn-env
source sklearn-env/bin/activate
pip install -U scikit-learnThe -U flag means upgrade: if the library is already there, update it. That block is a set of commands for your own terminal, so there is nothing to show as output here. The Windows command to switch the environment on is different, and the install page lists it.
To check that it worked, the same page suggests a command that prints the version and where the library lives.
python -m pip show scikit-learnOne pattern for everything
Here is the best thing about scikit-learn. Every learning tool in it works the same way. Learn the pattern once, and you can use hundreds of tools.
Think of a coffee machine with one button. You put in beans (fit). Then you get cups (predict). A different machine may use different beans, but the buttons are the same. Wikipedia lists this under features: a 'consistent way of running machine learning models (estimator.fit() and estimator.predict())'.
Estimators: fit, then predict
The Getting Started guide says scikit-learn 'provides dozens of built-in machine learning algorithms and models, called estimators. Each estimator can be fitted to some data using its fit method.' So the steps are: make the estimator, fit it, then predict.
Next is the guide's own first example, with its small data. It is shown here as the guide gives it, not run on this machine, because scikit-learn is not installed here.
# From the scikit-learn Getting Started guide, shown without its output (not run here)
from sklearn.ensemble import RandomForestClassifier
clf = RandomForestClassifier(random_state=0)
X = [[ 1, 2, 3], # 2 samples, 3 features
[11, 12, 13]]
y = [0, 1] # classes of each sample
clf.fit(X, y)
clf.predict([[4, 5, 6], [14, 15, 16]]) # predict classes of new dataRead it slowly. First, one line makes the estimator, here a random forest classifier. Then X holds the facts and y holds the answers. After clf.fit(X, y), the estimator has learned. Last, clf.predict gives answers for rows it has never seen.
The guide explains X and y. The size of X 'is typically (n_samples, n_features)', which means samples 'are represented as rows and features are represented as columns'. A sample is one row, such as one house. A feature is one column, such as its size. For y, the guide says it holds target values, and 'For unsupervised learning tasks, y does not need to be specified.' So clustering needs no answers.
The same pattern, in plain Python
To see how simple the pattern is, we can build our own estimator with no library at all. Ours is a silly one. It always predicts the average of the answers it saw. It is called MeanRegressor, because regression predicts numbers. The data is four flats: their sizes, and their weekly rents.
class MeanRegressor:
"""Predicts the average of the training answers, for every new row."""
def fit(self, X, y):
self.mean_ = sum(y) / len(y)
return self
def predict(self, X):
return [self.mean_ for _ in X]
X = [[50], [70], [90], [120]] # four flats: size in square metres
y = [150, 210, 260, 360] # their rents, in pounds a week
model = MeanRegressor()
model.fit(X, y)
print(model.mean_)
print(model.predict([[60], [100]]))Output
245.0 [245.0, 245.0]
Fit worked out one number, 245.0, the average rent. Predict then gave that number for any flat. It is a poor model, since a flat of 60 square metres and one of 100 get the same guess. Still, the shape is exactly scikit-learn's. The glossary says 'The fit method is provided on every estimator.' Notice that fit returns self, the fitted estimator, so calls can be chained, as the glossary advises.
- 1
Make it
Create the estimator. Choices, such as how many neighbours to look at, go in the brackets.
- 2
Fit it
Call fit with X and y. It learns from the examples.
- 3
Predict
Call predict with new rows of X. It gives back guesses.
- 4
Check
Compare the guesses with the real answers on data it did not train on. Fit alone does not prove a model works.
Transformers: fit, then transform
Some tools do not guess answers. They change the data, such as rescaling a column, as you saw in the lessons on preprocessing. Scikit-learn calls them transformers. The guide says they 'follow the same API as the estimator objects', but 'don't have a predict method but rather a transform method'. The glossary defines a transformer as 'An estimator supporting transform and/or fit_transform'.
Why fit a tool that only changes data? Because it must learn something first. To centre a column, it needs the column's average. Fit learns the average. Transform applies it. Here is the guide's example, shown unrun.
# From the scikit-learn Getting Started guide, shown without its output (not run here)
from sklearn.preprocessing import StandardScaler
X = [[0, 15],
[1, -10]]
StandardScaler().fit(X).transform(X)There is also a shortcut. The glossary says fit_transform 'fits the estimator and returns the transformed training data.' It does fit and transform in one step. Our own small transformer shows the idea: it learns each column's average, then subtracts it.
class Centre:
"""A transformer: learns the average of each column, then subtracts it."""
def fit(self, X, y=None):
n = len(X)
self.means_ = [sum(row[i] for row in X) / n for i in range(len(X[0]))]
return self
def transform(self, X):
return [[v - m for v, m in zip(row, self.means_)] for row in X]
def fit_transform(self, X, y=None):
return self.fit(X, y).transform(X)
X = [[50, 1], [70, 2], [90, 2], [120, 3]] # size, bedrooms
print(Centre().fit_transform(X))Output
[[-32.5, -1.0], [-12.5, 0.0], [7.5, 0.0], [37.5, 1.0]]
The average size is 82.5, so the first flat (50) becomes -32.5. The average bedrooms is 2, so 1 becomes -1. One rule matters here. Fit a transformer on the training data only, then use transform on the test data. You will meet the reason again in a moment.
Splitting data to test honestly
A student who studies the exact questions of a test will score well on that test. That does not prove they learned the subject. A model is the same. The guide puts it plainly: 'Fitting a model to some data does not entail that it will predict well on unseen data.'
So you hold some data back. The tool for this is train_test_split, which the guide calls a helper 'that splits a dataset into train and test sets'. You fit on the train part, and score on the test part, which the model has never seen. This is the guide's Iris example, shown unrun. Iris is a small, famous data set of flowers.
# From the scikit-learn Getting Started guide (not run here, no output shown)
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)It gives back four pieces: the facts and answers for training, and the facts and answers for testing. The random_state is a fixed number, so the shuffle is the same each time and others can repeat your result.
Pipelines: chaining steps
Real projects have steps in a row: tidy the data, then learn. A pipeline joins them into one object. The guide says: 'Transformers and estimators (predictors) can be combined together into a single unifying object: a Pipeline. The pipeline offers the same API as a regular estimator: it can be fitted and used for prediction with fit and predict.'
Think of an assembly line. Raw material goes in at one end, and each station does its job in order. You press one button at the start. The guide adds a big benefit: 'using a pipeline will also prevent you from data leakage', which means test data quietly sneaking into training. Here is the guide's example, shown unrun.
# From the scikit-learn Getting Started guide (not run here, no output shown)
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
pipe = make_pipeline(
StandardScaler(),
LogisticRegression()
)
pipe.fit(X_train, y_train)Quick check
A transformer, in scikit-learn, has which method that an estimator which predicts has instead of predict?
A map of the modules
Scikit-learn is organised into modules, which are folders of related tools. The API reference lists each one as sklearn.something. You do not need all of them today. Here are the ones this course will use most. Each name below is a module in the API reference.
preprocessing and model_selection
preprocessing holds tools like StandardScaler for getting data ready. model_selection holds train_test_split and the cross-validation tools for testing fairly.
linear_model, neighbors, tree and ensemble
These hold learning methods. They include LogisticRegression, the nearest-neighbours idea, decision trees and the random forest. You have already built a nearest-neighbours model by hand.
cluster and metrics
cluster groups data without answers, and includes KMeans. metrics holds the scores, such as accuracy_score, that tell you how well a model did.
Where to find help
Good documentation is part of a good tool, and scikit-learn has a lot of it. Four places are worth knowing. All are linked from the docs site.
The user guide
Long, readable chapters, one for each kind of tool. Its contents begin with 1. Supervised learning, then 2. Unsupervised learning. The Getting Started guide says to use it 'for details on all the tools that we provide'.
The examples gallery
Short, runnable programs with pictures. The Getting Started guide points you to 'numerous examples that illustrate the use of scikit-learn in many different contexts'. Copy one, change a number and see what moves.
The glossary
It lists the words and conventions used everywhere. Its first paragraph says it aims to describe the concepts 'and either detail their corresponding API or link to other relevant parts of the documentation'. Stuck on a word such as estimator or fit_transform? Look it up here.
The fourth place is the API reference, 'the class and function reference of scikit-learn'. It is the page to open when you know the tool and need its exact settings. Its own opening line says to read the user guide for the bigger picture, because 'the raw specifications of classes and functions may not be enough to give full guidelines on their use'.
Choosing the right estimator
With dozens of estimators, which do you pick? The docs have a page for that, called 'Choosing the right estimator'. It opens with a fair warning: 'Often the hardest part of solving a machine learning problem can be finding the right estimator for the job. Different estimators are better suited for different types of data and different problems.'
The page is a flowchart, like a game of 'which way now?'. It asks about your data and your goal: do you have enough samples, are you predicting a category or a quantity, do you have labelled data. It calls itself 'a bit of a rough guide', and its orange 'Try next' arrows mean: if this estimator does not do the job, follow the arrow and try the next one.
Quick check
Which of these is NOT something scikit-learn is designed for?
Quick check
In scikit-learn, what does an estimator's fit method do?
Next: choosing between models.
Sources
- scikit-learn: Home
- scikit-learn: Getting Started
- scikit-learn: Installing scikit-learn
- scikit-learn: Glossary of Common Terms and API Elements
- scikit-learn: API Reference
- scikit-learn: User Guide
- scikit-learn: Choosing the right estimator
- scikit-learn: Frequently Asked Questions
- scikit-learn: Neural network models (supervised)
- Wikipedia: scikit-learn
Last updated on 16 November 2026
Learn with me
Every lesson is free. I add new ones as I learn. Have a question, or want help with your own work? Book a short call.