Supervised Learning
Learning from labeled examples
A learning style where the model is shown each input together with its correct answer (label) and learns a function that predicts the label for new, unseen inputs.
Supervised learning is the most common and most intuitive flavor of machine learning. The model is shown thousands or even millions of (input, correct answer) pairs and tries to extract a function from them. After training, when it sees a new input it has never met, it uses that function to produce a prediction.
Two main families based on the kind of output. In classification the output is one label from a finite set: "spam or not", "dog, cat, or bird", "malignant or benign". In regression the output is a continuous number: the price of a house, tomorrow's temperature, the click-through rate of an ad.
Who provides the labels? Usually humans. This is the costliest part of supervised learning: collecting clean, consistent, sufficiently diverse labeled data. Big datasets like ImageNet are the result of years of work by thousands of human annotators.
Apprenticing under a chef. The chef hands you a plate and says "this is risotto", another and says "this is carbonara". After hundreds of plates, when you see a new one — without being told — you correctly say "risotto". You don't memorize the first plate, but with enough variety you internalize the essence of the concept. The labels came from the chef; you absorbed the pattern.
A bank wants to catch card fraud. It collects every transaction from the past five years: amount, time, location, merchant category, and the cardholder's habits. Beside each transaction it writes the actual outcome: "fraud" or "legitimate". Those labels come from chargebacks, customer disputes, and manual review teams.
After training, when the model sees an 8,500 TRY transaction at 03:14 from Brazil — knowing the cardholder usually does small purchases in Istanbul — it predicts "fraud" with high confidence. A human reviewer is alerted, the card is blocked. No engineer hand-coded a rule; everything was learned from the labeled history.
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
# Labeled data: X = features, y = class label
X, y = load_iris(return_X_y=True)
# Split into train and test (critical for generalization)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42
)
# Train the model with labeled data
model = RandomForestClassifier()
model.fit(X_train, y_train)
# Predict on examples it has never seen
accuracy = model.score(X_test, y_test)
print(f"Test accuracy: {accuracy:.2%}")- Historical data contains both inputs and correct outputs (labeled data)
- What you want to predict is a clear, measurable target
- You have enough varied and balanced examples
- Explainability matters and you can pick simpler models
- Labeling is impossible or too expensive — consider unsupervised or semi-supervised approaches
- Classes are very imbalanced (e.g. one in a million is fraud) — needs specialized techniques
- Data drifts fast and labels go stale — the model has to be retrained constantly
Leaky features
Information unavailable at prediction time leaks into training data. Classic case: predicting 'will the customer take a loan' while including 'loan start date' as a feature. Perfect at training, useless in production.
Imbalanced classes
If the positive class is 0.1% of the data, a model that predicts 'negative' for everything scores 99.9% accuracy and is useless. Fixes: under/oversampling, class weights, evaluate with F1 or ROC-AUC.
Label noise
Human labelers make mistakes or disagree. If the same image is labeled 'dog' / 'wolf' by two annotators, the model has a hard time learning consistently. Label quality dominates model quality.