Different questions to an image — different tasks
CV is not one task but a spectrum. It is important to understand the difference between the main types, because the tool choice and complexity depend on it. They all answer different questions to an image, with growing precision and complexity. Let us cover the main ones.
Three main types
- Classification — "what is it?": one label for the whole image. "The photo is a cat". The simplest task.
- Detection — "what and where?": boxes around objects with labels. "Here are three cars and two people, each in its box". Harder.
- Segmentation — "which exact pixels?": a precise mask of the objects shape, not a box. "Here is the exact object contour pixel by pixel". Even more precise.
Growing precision = growing complexity
Note the pattern: from classification to segmentation both the answer precision (label to boxes to pixel mask) and the complexity (usually more data and computation) grow. This matters for the choice: a more precise task is more demanding. Maximum precision is not always needed — often a simpler task suffices, which is cheaper and needs less data.
Take the minimally sufficient task
A practical choice principle: determine which question you ask the image, and take the minimally sufficient task type. Need to just categorize a photo — classification (simpler, cheaper). Need to find and count objects — detection. Need the exact shape/boundary — segmentation (but harder). A common mistake is taking a task more complex than needed (segmentation where classification would suffice): more data, computation, complexity without benefit. There are other tasks too (OCR, faces, pose, tracking), but these three are the foundation of understanding CV.
Rule: classification — "what" (a label), detection — "what and where" (boxes), segmentation — "which pixels" (a mask). Growing precision = growing complexity. Take the minimally sufficient task for your question.
🧠 How does object detection differ from classification?