How data Python differs from textbook Python

Python for AI from scratch · Lesson 1 / 22

Two different Pythons

In textbooks Python looks like this: a variable, a loop, a function, an exercise about factorials. In real work with data and models you write completely different code. It is shorter, it consists almost entirely of calls into other people's libraries, and its main difficulty is not the algorithm. The difficulty is that the data inside turns out to be nothing like what you assumed.

A typical working script is thirty lines: read a file, coerce a couple of columns to the right type, filter, group, save the result. Not a single loop of your own. But there are three places where everything can quietly go sideways: the file encoding, the column dtype, and the fact that the filter handed you a copy. You edit that copy and then wonder why the original never changed.

What this changes in practice

Correctness beats writing speed. A script that finished in a second and produced the wrong total is worse than a script that crashed.
Readability beats brevity. You will not understand a one-liner built from five nested comprehensions a month from now, and you will almost certainly come back to the analysis.
Intermediate output is part of the code. After every non-trivial step you look at the shape of the data instead of trusting that it went through.
Code lives longer than it looks. The one-off reporting script usually gets run again next quarter, and you are the person running it.

The minimal check ritual

Build the habit of running three lines every time you load data. They cost five seconds and they catch most of the silly mistakes before those mistakes reach a number in a report:

print(df.shape)     # how many rows and columns actually arrived
print(df.dtypes)    # what type each column has
print(df.head(3))   # what the first rows look like

If you get 0 rows, the path or the filter is wrong. If a numeric column has dtype object, there is text hiding inside it somewhere: a comma instead of a dot, a stray space, a dash for a missing value. If there are more columns than you expected, the file contains an extra separator.

Checking on a small known example

The most useful habit in this course: before you run a computation on the full dataset, build a tiny example where you already know the answer by hand. Five rows where the mean is 10 and there are two groups. If your code returns 10 and two groups on that example, you can run it on real data. If it does not, you caught the bug in a minute instead of a week later, in front of an audience.

The same rule saves you when you work with an AI assistant. The model writes syntactically correct code almost every time. Logically correct code arrives considerably less often. Checking against a known example is the only cheap way to tell one from the other.

Insight. An error that crashes your script is cheaper than an error that does not. A crash you see immediately; a quietly wrong number you see only when somebody challenges it.

Common mistake. Writing code with no print statements at all and running it straight on the full file. You then debug for twenty minutes instead of two, because you have no idea at which step the data stopped being what you expected.

Pro tip. Keep a sample.csv of 20 rows in the project, one where you know every answer by heart. Run each change against it before the full run. It is cheaper than any test suite and it works from day one.

Cheat sheet

Production data code is glue between libraries, not algorithms.
After every load, look at shape, dtypes, head.
A small example with a known answer catches more bugs than reading the code.
Your main reader is future you, so write for that person.

1. What usually breaks a data analysis script?

2. Why do you need a small example with a known answer?

3. A numeric column has dtype object after reading a CSV. What does that tell you?

🔒 Answer the question correctly to move on to the next lesson.