ML Foundations · Lesson 1 of 6
How Machine Learning Works: Data, Features, Training and Prediction
Machine learning is a recipe: examples go in, a model comes out, and the model makes predictions on new data. Learn every step of that recipe — features, labels, train/test split, training, overfitting — with one running example.
Your college wants to know, early in the semester, which students might fail a subject so mentors can help them in time. A teacher with 20 years of experience can often guess. Machine learning tries to learn that same judgement from past data — for every student, every semester, without getting tired.
The vocabulary in one table
| Word | Meaning | In our example |
|---|---|---|
| Example / row | One thing we learn from | One student in one past semester |
| Features (X) | The inputs the model looks at | Attendance %, internal marks, assignments submitted, hours on the LMS |
| Label / target (y) | The answer we want to predict | Passed or failed the final exam |
| Model | The learned pattern that maps X to y | "Low attendance + low internals → high risk" |
| Training | Adjusting the model to fit known examples | Learning from the last 3 years of students |
| Prediction / inference | Using the model on new data | Flagging this semester's at-risk students |
The ML workflow, step by step
- Define the problem. What exactly do we predict, and what decision will it support? ("Predict fail risk by week 6, so mentors can call students in week 7.")
- Collect and clean the data. Past records with both features and outcomes.
- Prepare features. Turn raw data into useful numbers (next lesson).
- Split the data into training and test sets.
- Train one or more models on the training set.
- Evaluate on the test set — data the model has never seen.
- Deploy and monitor. Use it, and keep checking it still works as students and syllabi change.
Why we split the data
Imagine a student who gets the exact question paper in advance and memorises the answers. Full marks — but has she learnt the subject? A model can do the same: memorise its training examples perfectly and fail on new ones. So we hide part of the data (often 20%) as a test set and only check performance there.
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y) # stratify keeps the pass/fail ratio the same in bothOverfitting and underfitting
- Underfitting — the model is too simple to capture the pattern. Poor on training and test data. (Predicting "everyone passes".)
- Overfitting — the model memorised noise. Excellent on training data, much worse on test data.
- Good fit — similar, good performance on both.
The quick diagnosis: compare training score with test score. Big gap → overfitting (simplify the model, get more data, regularise). Both low → underfitting (better features, a more flexible model).
The three kinds of ML problems you will meet first
- Regression — predict a number: house price, delivery time, next month's sales.
- Classification — predict a category: pass/fail, spam/not spam, fraud/genuine.
- Clustering — group similar items without labels: customer segments.
Your first complete model in 10 lines
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
X, y = load_breast_cancer(return_X_y=True) # a real medical dataset built into scikit-learn
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
model = DecisionTreeClassifier(random_state=42) # no depth limit: free to memorise
model.fit(X_train, y_train)
print("train accuracy:", model.score(X_train, y_train)) # 1.0 — it memorised the training set
print("test accuracy: ", model.score(X_test, y_test)) # noticeably lower — that gap is overfitting
model = DecisionTreeClassifier(max_depth=3, random_state=42).fit(X_train, y_train)
print("depth 3 → train:", model.score(X_train, y_train), "test:", model.score(X_test, y_test))Run it and look at the two pairs of numbers. The unlimited tree is perfect on data it has seen and worse on data it has not; limiting depth narrows the gap. You have just observed overfitting and fixed it — the most important idea in practical ML.
Never tune your model by looking at test-set results again and again. You slowly "learn" the test set and your final number becomes too optimistic. Use a validation set or cross-validation for tuning (covered in the evaluation lesson) and touch the test set once at the end.
Try it yourself: Run the code above in Google Colab. Change max_depth to 1, 2, 5, 10 and note train vs test accuracy for each. Which depth gives the best test score? Draw the two lines on paper — that curve is the overfitting picture every ML interview asks about.
Common interview questions
Try answering each one aloud before you open it.
1What is the difference between features and labels?
Features are the input variables the model uses to make a prediction, such as attendance and internal marks. The label is the output we want to predict, such as pass or fail.
2Why do we split data into training and test sets?
To measure how well the model generalises to unseen data. Evaluating on the training data rewards memorisation; the held-out test set simulates real new data.
3What is overfitting and how can you reduce it?
Overfitting is when a model fits noise in the training data and performs much worse on new data. Reduce it with more data, simpler models, regularisation, limiting depth or features, and validating with cross-validation.
Practise this
Build something with it
FAQs
How much data do I need to train a model?
It depends on the problem and model. Simple models on clean tabular data can work with a few hundred rows; complex patterns and deep learning need far more. More relevant, clean data almost always helps.
What is random_state?
A seed that makes random operations — like the train/test split — repeatable, so you and others get the same results when rerunning the code.
References & further reading
Finished this lesson?
Mark it complete. Create a free account to save it on every device.