Gesture Recognition using OpenCV + Python

This python script can be used to analyse hand gestures by contour detection and convex hull of palm region using OpenCV, a library used for computer vision processes.


The video below shows the working of the code:


1. Capture frames and convert to grayscale

  • Our ROI is the the hand region, so we capture the images of the hand and convert them to grayscale.
  • Why grayscale ?
    We convert an image from RGB to grayscale and then to binary in order to find the ROI i.e. the portion of the image we are further interested for image processing. By doing this our decision becomes binary: “yes the pixel is of interest” or “no the pixel is not of interest”.

 2. Blur image

  • I’ve used Gaussian Blurring on the original image. We blur the image for smoothing and to reduce noise and details from the image. We are not interested in the details of the image but in the shape of the object to track.
  • By blurring, we create smooth transition from one color to another and reduce the edge content. We use thresholding for image segmentation, to create binary images from grayscale images.

3. Thresholding

  • In very basic terms, thresholding is like a Low Pass Filter by allowing only particular color ranges to be highlighted as white while the other colors are suppressed by showing them as black.
  • I’ve used Otsu’s Binarization method. In this method, OpenCV automatically calculates/approximates the threshold value of a bimodal image from its image histogram. But for optimal results, we may need a clear background in front of the webcam which sometimes may not be possible.

4. Draw contours

5. Find convex hull and convexity defects

  • We now find the convex points and the defect points. The convex points are generally, the tip of the fingers. But there are other convex point too. So, we find convexity defects, which is the deepest point of deviation on the contour. By this we can find the number of fingers extended and then we can perform different functions according to the number of fingers extended.

You can reach me out over email, I’ll be happy to help 🙂

Write a Comment



    • # get hand data from the rectangle sub window on the screen
      crop_img = img[100:300, 100:300]

      here make ” crop_img = img[100:500, 100:800] “

  1. You mean the dimensions of the square where I place my hand ? I've specified the dimensions myself, you can change as per your need. The problem is that I am using Otsu's Binarization method for thresholding the palm region so I need minimum interference because of the surrounding colors. Therefore, I've used small square large enough to contain a hand gesture.

  2. Hi Vipul
    I'd like to make contact with you about gesture recognition. I am looking to pay a developer (you?) to code a hand gesture recognition python script for opencv and raspberry pi. I currently have opencv 3 and python 2.7 installed on a pi 2. will your code work? Thanks. Andre (

  3. thank you for your cod and explaining . but I have always this error
    Traceback (most recent call last):
    File "", line 16, in
    ValueError: too many values to unpack
    can you help me

  4. Hello vipul,
    Im working on my project finger recognition with raspberry pi and python.
    I already try your given code. And its work without error.
    But, nothing appear. Camera not functioning. Video not opening. No detection.
    Or i need press a key or what?
    Please help me sir.

  5. Hi
    sometime the code works fine but most of the time it is giving erroranious outnut also its not forming contours properly.I think its due to the background .Please suggest a solution

  6. Hi,
    The thresholding technique which I've used here (Otsu's Binarization method) may give some erroneous results for non uniform background. You should try it in front of some clean uniform background.

  7. Hello Vipul,
    I have been trying the same on Raspberry Pi. I changed the code a little bit but I am getting an error that says out of resources(other than memory) or incorrect buffer length for resolution.. Kindly help me out.

    Thank you

  8. iam on project to recognise people that smoking and using a phone during fueling their vehicle. so is it possible for me to recognise the ciggarates and handphone using this opencv and phython
    thank you very much.

  9. hi vipul,
    I need to know some facts regarding the project CURSOR CONTROL USING HAND GESTURE RECOGNITION
    -which window library should i use to control mouse movements.
    -can u suggest some ways for background substraction so only the hand can be captured.

  10. hey m getting the following error… please help

    line 16, in
    (version, _, _) = cv2.__version__.split('.')
    ValueError: too many values to unpack