What Machine Learning Is

A program that derives its rules from examples

In ordinary programming a human writes the rules: "if the message contains the word X and the sender is not in the contact list, it is spam." Machine learning inverts that. You hand the program a large pile of examples — thousands of messages labelled spam or not spam — and it works out for itself the rules that separate those examples best. The code stops being a description of the logic and becomes a description of the training procedure.

How this differs from the code you write every day

• An ordinary program. The human states the rules, the machine applies them. The behaviour is deterministic and can be read end to end in the source.
• ML. The human supplies the data and a loss function, the model fits the parameters. The behaviour lives in the weights, not in the code.
• What this means for debugging. You do not fix a line of code, you fix the data, the features or the way the problem is framed. A debugger is close to useless here; metrics are close to everything.

When ML is the right call

ML earns its place where the rules are too numerous or too hard to state: image recognition (try describing a cat as a set of conditions), language understanding, demand forecasting, recommendations, fraud detection. The signature of a good candidate is this: you can tell a correct answer from a wrong one, but you cannot spell out the algorithm that produces it. If the rule fits into three if statements, write three if statements. That is cheaper, it is faster, it is covered by tests, and it will not quietly degrade on its own six months from now.

The minimal working loop

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)          # training: fitting parameters to the data
print(model.score(X_test, y_test))   # share of correct answers on unseen data

Those four lines contain the entire cycle: split the data, train, measure on something the model has never seen. Everything else in this course is detail around these four lines — how to assemble X, how to choose a model, how to measure honestly, and what to do when the number disappoints you. Pay attention to random_state: without it the split changes on every run, and you will not be able to compare two experiments against each other.

A model does not understand and does not think

A model finds statistical dependencies and applies them. Two practical limits follow. The first: model quality is capped by data quality — on dirty labels a good model is impossible in principle, and no algorithm compensates for that. The second: on data unlike the training set the model behaves unpredictably and confidently at the same time, because it has no way to say "I have never seen anything like this." That is precisely why in production a model is always accompanied by monitoring of the input distributions, not only by error logs.

Cheat sheet

• The rules are not written by a human; they come out of optimisation over data.
• ML is justified when the answer is checkable but the algorithm cannot be formalised.
• Model quality is limited by data quality, not by the choice of algorithm.
• Fix random_state, otherwise experiments are not comparable.