---
title: "Draw in the air with MediaPipe finger tracking"
description: "Build an air-drawing canvas in Python. Track your index fingertip with MediaPipe, switch draw, hover and clear by gesture, and overlay it on live video."
url: https://articles.sythra.ai/articles/draw-in-air-mediapipe-opencv
slug: draw-in-air-mediapipe-opencv
author: "vaibhavkothari"
author_url: https://articles.sythra.ai/writers/vaibhavkothari
date_published: 2026-11-12T21:11:00.153Z
date_modified: 2026-11-12T21:11:00.153Z
topics: ["Python", "Mediapipe", "Opencv", "Computer Vision"]
reading_time_minutes: 6
publisher: "Sythra"
publisher_url: https://sythra.ai
access: free
language: en
---

# Draw in the air with MediaPipe finger tracking

> Build an air-drawing canvas in Python. Track your index fingertip with MediaPipe, switch draw, hover and clear by gesture, and overlay it on live video.

Source: https://articles.sythra.ai/articles/draw-in-air-mediapipe-opencv · Author: vaibhavkothari · Published: 2026-11-12 · Reading time: 6 min · Topics: Python, Mediapipe, Opencv, Computer Vision

Point your index finger at the webcam and paint on thin air. This project is a good test of everything hand tracking teaches: reading landmarks, turning finger positions into *gestures*, and keeping a persistent drawing separate from the live video. By the end you will have a canvas with draw, pause and erase modes.

## What we are building

- **Index finger up alone** → draw
- **Index + middle fingers up** → "hover" (move without drawing)
- **Fist or all fingers up** → clear the canvas
- Pressing `c` also clears it, `q` quits

## Install

```bash
python -m pip install opencv-python mediapipe numpy
```

MediaPipe usually needs Python 3.8–3.12. If installation fails on a newer Python, create a [virtual environment](https://articles.sythra.ai/articles/venv-vs-conda-vs-uv) with a supported version.

## The key idea: two layers

If you draw directly onto the video frame, the line is erased on the next frame because a new frame arrives. So keep a separate persistent **canvas** and combine it with each frame for display:

```
camera frame (new every frame)  ─┐
                                 ├──► merged image shown on screen
canvas (kept between frames)    ─┘
```

## Hand landmarks you need

MediaPipe returns 21 landmarks per hand with coordinates normalised to 0–1. The useful ones:

| Landmark | Index |
|---|---|
| Index fingertip | 8 |
| Index middle joint (PIP) | 6 |
| Middle fingertip | 12 |
| Middle PIP | 10 |
| Ring fingertip / PIP | 16 / 14 |
| Pinky fingertip / PIP | 20 / 18 |

A finger is "up" when its **tip is above its PIP joint** — meaning a *smaller* `y`, because image y grows downward.

## Step 1 — Setup and finger detection

```python
import cv2
import mediapipe as mp
import numpy as np

mp_hands = mp.solutions.hands
mp_draw = mp.solutions.drawing_utils

hands = mp_hands.Hands(
    max_num_hands=1,
    min_detection_confidence=0.7,
    min_tracking_confidence=0.6,
)

TIPS = {"index": 8, "middle": 12, "ring": 16, "pinky": 20}
PIPS = {"index": 6, "middle": 10, "ring": 14, "pinky": 18}


def fingers_up(landmarks):
    """Return a dict like {'index': True, 'middle': False, ...}."""
    return {
        name: landmarks[TIPS[name]].y < landmarks[PIPS[name]].y
        for name in TIPS
    }
```

This ignores the thumb on purpose — thumb direction depends on which hand it is and is not needed here.

## Step 2 — The main loop

```python
cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1280)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 720)

canvas = None
prev_point = None
color = (0, 140, 255)        # BGR orange
thickness = 8

while True:
    ok, frame = cap.read()
    if not ok:
        break

    frame = cv2.flip(frame, 1)                     # mirror so movement feels natural
    h, w = frame.shape[:2]
    if canvas is None:
        canvas = np.zeros((h, w, 3), dtype=np.uint8)

    rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)   # MediaPipe expects RGB
    result = hands.process(rgb)

    mode = "no hand"
    if result.multi_hand_landmarks:
        hand = result.multi_hand_landmarks[0]
        lm = hand.landmark
        up = fingers_up(lm)

        tip = (int(lm[8].x * w), int(lm[8].y * h))

        if all(up.values()) or not any(up.values()):
            mode = "clear"
            canvas[:] = 0
            prev_point = None
        elif up["index"] and not up["middle"]:
            mode = "draw"
            if prev_point is not None:
                cv2.line(canvas, prev_point, tip, color, thickness)
            prev_point = tip
        else:
            mode = "hover"
            prev_point = None

        cv2.circle(frame, tip, 10, (255, 255, 255), 2)
        mp_draw.draw_landmarks(frame, hand, mp_hands.HAND_CONNECTIONS)
    else:
        prev_point = None

    # merge canvas onto the frame
    gray = cv2.cvtColor(canvas, cv2.COLOR_BGR2GRAY)
    _, inv = cv2.threshold(gray, 1, 255, cv2.THRESH_BINARY_INV)
    inv = cv2.cvtColor(inv, cv2.COLOR_GRAY2BGR)
    frame = cv2.bitwise_and(frame, inv)       # cut a hole where the drawing is
    frame = cv2.bitwise_or(frame, canvas)     # paint the drawing in

    cv2.putText(frame, f"mode: {mode}", (20, 40),
                cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 2)
    cv2.imshow("Air canvas", frame)

    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break
    if key == ord("c"):
        canvas[:] = 0

cap.release()
cv2.destroyAllWindows()
```

### Why `prev_point` matters

A line is drawn between the previous fingertip position and the current one. Setting `prev_point = None` whenever you are not drawing breaks the stroke, so lifting your finger does not leave a straight line jumping across the canvas.

### Why the bitwise merge?

Adding the canvas to the frame would make the drawing translucent against bright backgrounds. The mask approach first blanks the frame wherever the canvas has ink, then paints the ink in, giving solid colour.

## Step 3 — Make strokes smooth

Raw fingertip positions jitter by a few pixels. Blend each new point with the previous one:

```python
SMOOTH = 0.6   # 0 = no smoothing, closer to 1 = heavy smoothing

def smooth(prev, new, amount=SMOOTH):
    if prev is None:
        return new
    return (
        int(prev[0] * amount + new[0] * (1 - amount)),
        int(prev[1] * amount + new[1] * (1 - amount)),
    )

# in draw mode:
tip = smooth(prev_point, tip)
```

This is exponential smoothing, the same trick used in the [virtual mouse guide](https://articles.sythra.ai/articles/virtual-mouse-opencv-mediapipe). Too much smoothing makes the line lag behind your finger, so tune between 0.3 and 0.7.

## Step 4 — Add a colour palette

Use hover mode as a "selection" gesture: when you hover over a swatch at the top of the screen, switch colour.

```python
SWATCHES = [
    ((20, 70),  (0, 140, 255)),    # orange
    ((90, 140), (0, 200, 0)),      # green
    ((160, 210), (255, 100, 0)),   # blue
]

def draw_palette(frame):
    for (x1, x2), col in SWATCHES:
        cv2.rectangle(frame, (x1, 60), (x2, 110), col, -1)

def pick_color(tip, current):
    if 60 <= tip[1] <= 110:
        for (x1, x2), col in SWATCHES:
            if x1 <= tip[0] <= x2:
                return col
    return current
```

Call `draw_palette(frame)` before showing the frame, and in `hover` mode set `color = pick_color(tip, color)`. Because drawing only happens in draw mode, you can reach the swatches without leaving a mark.

## Troubleshooting

| Problem | Fix |
|---|---|
| Lines jump across the canvas | Reset `prev_point = None` when not drawing |
| Drawing flickers or lags | Lower resolution to 640×480; reduce smoothing |
| Gestures misfire | Raise `min_detection_confidence`; require the gesture for 3 frames before switching mode |
| Left hand behaves oddly | The tip-above-PIP rule works for either hand when your palm faces the camera |
| `cv2.imshow` window is black | Camera index wrong or used by another app; try `VideoCapture(1)` |

## Ideas to extend it

- Add an eraser: a gesture where the fist draws in black with a thick brush
- Save the drawing with `cv2.imwrite("drawing.png", canvas)` on a key press
- Recognise what you drew by sending the canvas to a classifier or an LLM with vision

The same landmark-to-gesture approach drives the [gesture volume controller](https://articles.sythra.ai/articles/hand-gesture-volume-control-python) and the [finger counter](https://articles.sythra.ai/articles/count-fingers-mediapipe-python).

## FAQ

### How does air drawing with MediaPipe work?

MediaPipe finds 21 hand landmarks in each frame. The code reads the index fingertip position, draws a line from its previous position onto a separate canvas, and overlays that canvas on the video.

### How do I tell if a finger is up in MediaPipe?

Compare the fingertip landmark's y value with the joint below it. Because image y increases downward, the finger is up when the tip's y is smaller than the PIP joint's y.

### Why does my drawing disappear on the next frame?

You are drawing on the video frame itself, which is replaced each loop. Draw on a persistent canvas array and merge it with each new frame.

### Can I draw with both hands?

Yes. Set `max_num_hands=2` and keep a separate previous-point variable for each hand.

## Next reading on Sythra Articles

- [Build a virtual mouse with OpenCV and MediaPipe](https://articles.sythra.ai/articles/virtual-mouse-opencv-mediapipe)
- [Count fingers with MediaPipe](https://articles.sythra.ai/articles/count-fingers-mediapipe-python)
- [Detect colors and track objects with OpenCV HSV](https://articles.sythra.ai/articles/opencv-color-detection-hsv-tracking)

## Glossary (terms defined in this article)
- **landmarks** (medium) — Numbered key points on the hand — knuckles, joints and fingertips
- **exponential smoothing** (medium) — Blending the old value with the new one so movement looks calm