What Is Computer Vision? Tasks, Models and Uses

Computer vision is the field of AI that lets computers understand images and video. A camera gives a computer a grid of numbers, and computer vision turns those numbers into answers: what is in the picture, where it is, and what it is doing. Modern systems learn these skills from large sets of labelled images using deep neural networks, mainly convolutional neural networks (CNNs) and, more recently, vision transformers.

This guide explains how computer vision works, the main tasks, the models behind them, real business uses, the limits to plan for and how to start a project.

#Key takeaways

• An image is just numbers. A colour image of 224 by 224 pixels is 224 × 224 × 3 = 150,528 values.
• Three core tasks: classification (what), detection (what and where) and segmentation (which pixels).
• Deep learning took over around 2012, when a CNN won the ImageNet challenge by a wide margin.
• Vision transformers (2020) treat image patches like tokens and now compete with CNNs.
• Data quality decides results. Labelled, representative examples matter more than the model name.

#How computer vision works

1. Capture. A camera, scanner or video stream provides pixels.
2. Preprocess. Resize, normalise colours, and sometimes augment (flip, crop, change brightness) to make training data more varied.
3. Extract features. Early layers of a network find edges and textures. Deeper layers combine them into shapes and then whole objects.
4. Predict. The final layers output a label, boxes or a per-pixel map.
5. Post-process. Filter low-confidence results, remove duplicate boxes and apply business rules.

Older computer vision used hand-written rules and features. Deep learning learns the features directly from examples, which is why it needs plenty of data.

#The main computer vision tasks

Classification is the base task. Detection adds location, and segmentation adds pixel-level detail.

#Models: from CNNs to vision transformers

Convolutional neural networks (CNNs) slide small filters over an image to detect patterns, building from edges to objects. In 2012, AlexNet, a deep CNN, won the ImageNet classification contest (1,000 categories) with a top-5 error of about 15.3%, far below the roughly 26% of the best earlier approach. CNNs dominated the following years, and by 2016 and 2017 the best entries beat typical human accuracy on that benchmark.

Vision transformers (ViT), introduced in 2020, split an image into patches (for example 16 by 16 pixels), treat each patch as a token and process them with self-attention, the same mechanism behind language models. With enough training data they match or beat CNNs on classification. See how large language models work for the attention idea.

Real-time detectors such as the YOLO family trade a little accuracy for speed, which suits video, robotics and edge devices.

Multimodal models now take images and text together, so you can ask a model questions about a picture. Embeddings, covered in embeddings explained, also power image search by similarity.

#Where computer vision is used

• Manufacturing: spotting defects and checking assembly on a production line.
• Healthcare: assisting radiologists with scans, always with human review.
• Retail: shelf monitoring, visual search and virtual try-on.
• Documents: reading invoices, IDs and forms. See document processing automation.
• Transport and safety: driver assistance, traffic analytics and site safety monitoring.
• Agriculture: crop and livestock monitoring from drones and cameras.
• Security: access control and anomaly alerts, with privacy and consent in mind.

#Limits and risks

• Data bias. A model trained on narrow data fails on people, lighting or environments it never saw.
• Distribution shift. A new camera, season or product design can quietly drop accuracy.
• Adversarial inputs. Small, deliberate changes to an image can fool a model.
• Errors in the wild. Real conditions (blur, glare, occlusion) are harder than benchmark images.
• Privacy and law. Images of people raise consent, retention and regulatory questions.
• Cost. Training and running models on video can be expensive.

#How to measure a vision model

Accuracy alone is rarely enough. Use metrics that match the task: precision and recall for classification, intersection over union (IoU) and mean average precision (mAP) for detection and segmentation. Always test on images that look like production, not only on a clean validation set. See model evaluation metrics and overfitting and bias-variance.

#How to start a computer vision project

1. Define one narrow task and what success looks like in business terms.
2. Collect representative images, including hard and rare cases.
3. Label carefully. Consistent labels beat a large noisy set.
4. Start from a pretrained model and fine-tune it, rather than training from scratch.
5. Evaluate on held-out production-like data.
6. Plan deployment: cloud API, on-device or edge, and the latency you need.
7. Monitor in production and retrain as conditions change. See the MLOps guide.

Common tools include OpenCV for image handling, PyTorch for models and ready-made detectors such as the YOLO family. A foundation in machine learning helps.

#Need a vision feature built?

We build computer vision and AI features, from a scoped prototype to a monitored production system. Get a scope estimate for your AI project or see how we approach AI agent development.

#FAQ

#What is computer vision in simple words?

It is teaching computers to understand pictures and video, so they can recognise objects, read text, find locations and follow movement.

#What are the main tasks in computer vision?

Image classification, object detection, semantic and instance segmentation, pose estimation, OCR and tracking.

#What is the difference between computer vision and image processing?

Image processing changes or enhances images (resizing, filtering, sharpening). Computer vision extracts meaning from them (what is there and what it means). Vision systems often use image processing as a first step.

#What is a CNN?

A convolutional neural network is a deep learning model that uses small sliding filters to find patterns in images, building from edges to complex objects.

#What is a vision transformer?

A model that splits an image into patches, treats each as a token and uses self-attention to relate them. It was introduced in 2020 and can match or beat CNNs with enough data.

#How much data do I need?

It depends on the task and whether you fine-tune a pretrained model. Fine-tuning needs far less data than training from scratch. Start with a small, well-labelled set, measure, then add data where the model fails.

#Is computer vision accurate enough for critical uses?

It can be very accurate on well-defined tasks, but it fails in ways people do not expect. For high-stakes uses such as medicine, keep a human in the loop and test on real-world data.