Data, Features and the Target Variable

X, y and everything in between

Every supervised learning problem comes down to two objects: the feature matrix X and the target vector y. A row of X is one object (a flat, a message, a customer, a transaction), a column is one feature. An element of y is the correct answer for the matching row. If you cannot draw that table for your own problem, the problem is not framed yet, and it is too early to sit down with a model.

Data

Data is the set of examples the model learns from. To predict apartment prices this is a table of sold apartments where both the characteristics and the actual transaction price are known. Three properties matter. Volume: hundreds of rows are enough for a linear model, while gradient boosting and neural networks want thousands to tens of thousands. Cleanliness: duplicates, labelling errors and broken values hurt quality more than the choice of algorithm does. Representativeness: the data must be collected the same way it will arrive in production. A model trained on apartments in one district does badly in another district not because there is too little data, but because the sample is shifted.

Features

Features are the characteristics of an object the model reasons from: floor area, number of rooms, floor level, district, condition. The choice of features decides more than the choice of model. Floor area predicts price; wallpaper colour does not. Features come as numeric, categorical, binary, temporal and textual, and each type needs its own preparation — a later lesson is devoted to exactly that.

import pandas as pd
df = pd.read_csv("flats.csv")
y = df["price"]
X = df.drop(columns=["price", "deal_closed_at", "deal_id"])
print(X.shape, X.dtypes.value_counts())
print(X.isna().mean().sort_values(ascending=False).head())
print(y.describe())

These four lines of diagnostics are worth running on every new dataset: the size, the column types, the share of missing values per feature and the distribution of the target. A column with 90% missing values is almost always useless; a column of type object where you expected a number almost always means that somewhere in the data there is a string such as "no data".

Model

A model is a parameterised function that training adjusts so that its output on X comes as close to y as possible. Once trained, it is applied to new rows: give it the features of an apartment that was not in the data and you get a price prediction. The two stages, fit and predict, are separated strictly: everything computed from data — means, category encodings, scaling ranges — is computed on the training part only.

Target leakage is the most expensive mistake at this stage

Leakage happens when X contains a feature that in reality becomes known after y, or is derived from it. The classic case: leaving the deal_closed_at field in a "will the deal close" problem, or the service disconnection date in a churn problem. The metric will look magnificent and the model will fall apart in production, because at prediction time that feature simply does not exist yet. The check is simple: for every column, ask whether it will be filled in at the moment the model has to produce an answer.

Cheat sheet

• X holds objects in rows and features in columns; y is the answer of the same length.
• A representative sample matters more than a large one.
• Every statistic is computed on train and applied to test.
• A feature unavailable at prediction time is leakage.