Pixels, Channels and Colour Spaces

What actually enters the model

An image in memory is an array of numbers shaped "height x width x channels". For an ordinary photo that is HxWx3, type uint8, values 0-255 per channel. A 1920x1080 frame takes about 6 MB uncompressed and 200-400 KB as JPEG; half of all engineering decisions about storage and transfer are built on that gap. Understanding the layout of the array matters for practical reasons, not theoretical ones: almost every "the model outputs garbage" at the start of a project comes down to a mixed-up order of axes, channels or value range.

The four things that break most often

Channel order. One popular library reads a file as BGR, another as RGB. The mistake raises no exception: the picture simply looks bluish, and accuracy drops by 5-15 percentage points on colour-sensitive classes.
Axis order. HxWxC and CxHxW live side by side in the same pipeline. Mix them up and you get either an explicit shape error or, worse, a silent interpretation of rows as channels.
Value range. The model expects 0-1 or normalised values and you feed it 0-255. The result is confident and wrong predictions.
EXIF orientation. Phone photos are stored in landscape plus a rotation flag. A library that ignores the flag hands the model a frame rotated by 90 degrees - while the annotation was done in a viewer that respected the flag.

Colour spaces and when they help

RGB   - the input of nearly any trained network, the default
Grayscale - 1 channel; fine when colour carries no information
              (X-ray, IR, text) - saves 2/3 of the input data
HSV   - hue / saturation / value kept apart; useful for colour
        thresholds that survive changes in brightness
LAB   - lightness separated from colour; handy for correction
YCbCr - the internal format of JPEG and video codecs

The practical point of HSV: if you separate yellow road markings from asphalt with a simple threshold, in RGB that threshold drifts with every passing cloud, while in HSV mostly the V channel changes, and you do not use V in the threshold. For a neural network the choice of space barely matters: it will learn the transform itself as long as the transform is linear. The difference shows up in heuristics and preprocessing.

Bit depth and special channels

Eight bits per channel is not the whole universe. Medical and industrial cameras deliver 12-16 bits: dark regions of an X-ray hold detail that simply disappears when you coarsen it to 8 bits. If you convert a 16-bit image with a windowing transform, the window becomes part of the model - fix it and repeat it at inference. Non-colour channels are a separate story: depth from a ToF camera, thermal, near IR. You can feed them as an extra channel, but the weights of a pretrained network expect three channels, and the fourth one has to be initialised by hand.

Insight. A colour space is a tool for rules, not for the network. The network will learn any linear transform; human thresholds written in HSV live longer than the ones written in RGB.

Common mistake. Debugging a model without ever looking at the tensor that goes into it. Save a batch back to files after the whole preprocessing chain - half the bugs are visible in ten seconds.

Pro tip. Keep one image-reading module for the entire project: it opens the file, applies the EXIF rotation, converts to RGB and to the required range. Any other loading path in the code is a source of divergence between training and production.

Cheat sheet

An image is HxWxC, uint8 0-255; mismatches in axes and channels are silent.
BGR versus RGB and EXIF rotation are the two most common sources of quiet quality loss.
HSV and LAB help with thresholds and correction; RGB is enough for a network.
12-16 bits carry detail in the shadows; fix the windowing transform as part of the pipeline.

1. Why is a mixed-up channel order especially nasty?

2. In which situation is HSV more convenient than RGB?

3. What is the danger of coarsening a 16-bit image to 8 bits?

Task — checked by AI

Take 10 of your own images from different sources: a phone, a screenshot, a camera frame, a scan. For each one write down the resolution, the number of channels, the data type, whether an EXIF rotation is present and the file size. Load them two different ways (for example with two libraries) and compare: do the channel order, the orientation and the value range match? Describe the discrepancies you found and which of them would have silently ruined training.

← Back

🔒 Answer the question correctly to move on to the next lesson.