Linear Regression Explained: Predicting Numbers with a Straight Line

Every evening a café near your college decides how much milk to order for tomorrow. Too little and they turn customers away; too much and it spoils. If they notice that hotter days bring more cold-coffee orders, they are already thinking like linear regression: predict a number from other numbers by finding the best straight-line relationship.

The idea in one equation

With one feature, the model is the line you drew in school:

predicted_orders = intercept + slope × temperature
                 =    20     +  4.5  × temperature

The slope (coefficient) says: each extra degree adds about 4.5 orders. The intercept is the baseline. With more features it simply extends:

orders = b0 + b1×temperature + b2×is_weekend + b3×exam_week + ...

Each coefficient tells you how the prediction changes when that feature goes up by one, holding the others constant. That readability is why linear models are still loved in banking, insurance and policy work, where you must explain every decision.

How the model learns: minimising error

For every past day, the line makes a prediction and is off by some amount (the residual). Training finds the coefficients that make the total error as small as possible. The usual measure is Mean Squared Error (MSE): square each error (so negatives do not cancel positives and big misses count extra), then average.

For linear regression there is a direct formula for the best coefficients (ordinary least squares). For bigger models, the general method is gradient descent: start with random coefficients, measure the error, nudge each coefficient in the direction that reduces it, and repeat. Picture walking downhill in fog, one careful step at a time — the same idea trains neural networks with billions of parameters.

Train one in Python

import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error, r2_score

rng = np.random.default_rng(7)
n = 300
temperature = rng.uniform(18, 42, n)
is_weekend  = rng.integers(0, 2, n)
exam_week   = rng.integers(0, 2, n)
orders = 20 + 4.5 * temperature + 35 * is_weekend - 25 * exam_week + rng.normal(0, 12, n)

X = np.column_stack([temperature, is_weekend, exam_week])
X_train, X_test, y_train, y_test = train_test_split(X, orders, test_size=0.2, random_state=42)

model = LinearRegression().fit(X_train, y_train)
print("intercept:", round(model.intercept_, 1))
for name, coef in zip(["temperature", "is_weekend", "exam_week"], model.coef_):
    print(f"{name:12s} {coef:+.1f}")

pred = model.predict(X_test)
print("MAE:", round(mean_absolute_error(y_test, pred), 1), "orders")
print("R²: ", round(r2_score(y_test, pred), 3))
print("Saturday at 35°C, no exams:", round(model.predict([[35, 1, 0]])[0]))

Because we generated the data ourselves, you can check the model recovered the true pattern (about +4.5 per degree, +35 on weekends, −25 in exam week). On real data you do not know the truth — which is exactly why the test set matters.

Reading the results

• MAE (Mean Absolute Error) — average size of the error, in the target's units. "We are off by about 10 orders on average" is something a café owner understands.
• RMSE — like MAE but punishes big misses more.
• R² — share of the variation the model explains: 1.0 is perfect, 0 is no better than always predicting the average.

When linear regression struggles

• Curved relationships — orders may rise with temperature and then fall in extreme heat. Add transformed features (temperature²) or use a tree-based model.
• Outliers — squared errors let one crazy day pull the whole line.
• Correlated features (multicollinearity) — two features that move together make individual coefficients unstable and hard to interpret.
• Too many features for the data — use regularisation: Ridge shrinks coefficients, Lasso can push useless ones to exactly zero.