Detect colors and track objects with OpenCV HSV

Before deep learning, the fastest way to follow an object was to follow its colour. It is still the right tool when the object is distinctive — a red ball, a green marker cap, a bright sticker — and you want something that runs at full frame rate on a laptop with no model files. This guide builds a complete colour tracker and explains the one trick that makes it work: switching from BGR to HSV.

Install

python -m pip install opencv-python numpy

If import cv2 fails afterwards, see No module named cv2.

Why not just compare BGR values?

OpenCV loads images as BGR (blue, green, red), each 0–255. A "red" object does not have one BGR value. In bright light it might be (40, 40, 230), in shadow (10, 10, 110). The channels all move together when lighting changes, so a rule like "red above 200" fails whenever a cloud passes.

HSVA colour model with three channels: Hue (which colour), Saturation (how vivid), Value (how bright) separates the parts:

Note the quirk: OpenCV stores hue as 0–179, not 0–359, because it halves the angle to fit in one byte. Red is near 0 (and 179), green near 60, blue near 120.

Now "red" becomes "hue near 0, reasonably saturated, not too dark", a rule that survives lighting changes far better.

import cv2
import numpy as np

print(cv2.cvtColor(np.uint8([[[0, 0, 255]]]), cv2.COLOR_BGR2HSV))
# [[[  0 255 255]]]   pure red -> hue 0
print(cv2.cvtColor(np.uint8([[[255, 0, 0]]]), cv2.COLOR_BGR2HSV))
# [[[120 255 255]]]   pure blue -> hue 120

Step 1 — Make a mask

A mask is a black-and-white image: white where a pixel falls inside your colour range, black elsewhere.

import cv2
import numpy as np

frame = cv2.imread("ball.jpg")
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)

lower = np.array([100, 120, 70])    # blue-ish: H, S, V minimums
upper = np.array([130, 255, 255])   # H, S, V maximums

mask = cv2.inRange(hsv, lower, upper)
cv2.imshow("mask", mask)
cv2.waitKey(0)

Set the saturation and value minimums above zero. Without them, grey walls and dark shadows (which have a hue value but no real colour) leak into the mask.

Step 2 — Red wraps around

Hue is circular, and red sits at both ends: 0–10 and 170–179. One range cannot cover it, so use two and combine them:

lower1, upper1 = np.array([0, 120, 70]),   np.array([10, 255, 255])
lower2, upper2 = np.array([170, 120, 70]), np.array([179, 255, 255])

mask = cv2.inRange(hsv, lower1, upper1) | cv2.inRange(hsv, lower2, upper2)

Forgetting this is the classic reason a red tracker "works but only on orange-ish red".

Step 3 — Clean the mask

Raw masks are speckled with stray white pixels and have holes inside the object. Morphological operations fix both:

kernel = np.ones((5, 5), np.uint8)
mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)    # removes small white specks
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)   # fills small holes

Optionally blur the frame (cv2.GaussianBlur(frame, (7, 7), 0)) before converting to HSV to reduce sensor noise.

Step 4 — Find the object

ContoursOutlines of connected white regions in a binary image turn the mask into objects you can measure:

contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

if contours:
    biggest = max(contours, key=cv2.contourArea)
    if cv2.contourArea(biggest) > 500:          # ignore tiny blobs
        x, y, w, h = cv2.boundingRect(biggest)
        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)

        m = cv2.moments(biggest)
        cx = int(m["m10"] / m["m00"])
        cy = int(m["m01"] / m["m00"])
        cv2.circle(frame, (cx, cy), 6, (0, 255, 255), -1)

cv2.moments gives the centre of the shape. The area threshold (500 pixels here) is what separates "the ball" from noise; tune it to your camera distance.

Step 5 — Run it live with a trail

import cv2
import numpy as np
from collections import deque

LOWER1, UPPER1 = np.array([0, 120, 70]),   np.array([10, 255, 255])
LOWER2, UPPER2 = np.array([170, 120, 70]), np.array([179, 255, 255])
KERNEL = np.ones((5, 5), np.uint8)

trail = deque(maxlen=32)
cap = cv2.VideoCapture(0)

while True:
    ok, frame = cap.read()
    if not ok:
        break

    frame = cv2.flip(frame, 1)                       # mirror view
    blurred = cv2.GaussianBlur(frame, (7, 7), 0)
    hsv = cv2.cvtColor(blurred, cv2.COLOR_BGR2HSV)

    mask = cv2.inRange(hsv, LOWER1, UPPER1) | cv2.inRange(hsv, LOWER2, UPPER2)
    mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, KERNEL)
    mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, KERNEL)

    contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    center = None
    if contours:
        biggest = max(contours, key=cv2.contourArea)
        if cv2.contourArea(biggest) > 500:
            m = cv2.moments(biggest)
            center = (int(m["m10"] / m["m00"]), int(m["m01"] / m["m00"]))
            (x, y), radius = cv2.minEnclosingCircle(biggest)
            cv2.circle(frame, (int(x), int(y)), int(radius), (0, 255, 0), 2)

    trail.appendleft(center)
    for i in range(1, len(trail)):
        if trail[i - 1] is None or trail[i] is None:
            continue
        thickness = max(1, int(6 * (1 - i / len(trail))))
        cv2.line(frame, trail[i - 1], trail[i], (0, 165, 255), thickness)

    cv2.imshow("tracker", frame)
    cv2.imshow("mask", mask)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

cap.release()
cv2.destroyAllWindows()

Keep the mask window open while you tune. When tracking misbehaves, the mask shows why in a second.

Find the right range for your object

Guessing numbers is slow. Add sliders:

def nothing(_):
    pass

cv2.namedWindow("tune")
for name, start, top in [("H min", 0, 179), ("H max", 179, 179),
                         ("S min", 0, 255), ("S max", 255, 255),
                         ("V min", 0, 255), ("V max", 255, 255)]:
    cv2.createTrackbar(name, "tune", start, top, nothing)

# inside your loop:
lower = np.array([cv2.getTrackbarPos(n, "tune") for n in ("H min", "S min", "V min")])
upper = np.array([cv2.getTrackbarPos(n, "tune") for n in ("H max", "S max", "V max")])
mask = cv2.inRange(hsv, lower, upper)

Drag the sliders until only your object is white, then copy the six numbers into your constants.

When colour tracking fails

Colour is brittle compared with learned detectors: it knows nothing about what an object is. If you need to find faces, hands or people regardless of colour, use a model — for example MediaPipe face detection or hand tracking.

FAQ

Why use HSV instead of RGB for colour detection?

HSV separates colour (hue) from brightness (value), so a colour range stays valid when lighting changes. In RGB all three channels change together with brightness, making simple thresholds unreliable.

Why is the hue range 0 to 179 in OpenCV?

OpenCV halves the usual 0–359 degree hue so it fits in an 8-bit value. Red is near 0 and 179, green near 60 and blue near 120.

How do I detect red in OpenCV?

Red wraps around the hue circle, so create two masks, one for hues 0–10 and one for 170–179, and combine them with a bitwise OR.

How do I get the position of the detected object?

Find contours on the cleaned mask, pick the largest, and use cv2.moments to compute its centre, or cv2.boundingRect for a box.

Next reading on Sythra Articles

• Draw in the air with MediaPipe finger tracking
• Face detection: MediaPipe vs Haar cascades
• Extract text from an image with Python OCR