Gesture Recognition using OpenCV + Python

This python script can be used to analyse hand gestures by contour detection and convex hull of palm region using OpenCV, a library used for computer vision processes.

code: https://github.com/vipul-sharma20/gesture-opencv

The video below shows the working of the code:

How?

1. Capture frames and convert to grayscale

  • Our ROI is the the hand region, so we capture the images of the hand and convert them to grayscale.
  • Why grayscale ?
    We convert an image from RGB to grayscale and then to binary in order to find the ROI i.e. the portion of the image we are further interested for image processing. By doing this our decision becomes binary: “yes the pixel is of interest” or “no the pixel is not of interest”.

 2. Blur image

  • I’ve used Gaussian Blurring on the original image. We blur the image for smoothing and to reduce noise and details from the image. We are not interested in the details of the image but in the shape of the object to track.
  • By blurring, we create smooth transition from one color to another and reduce the edge content. We use thresholding for image segmentation, to create binary images from grayscale images.

3. Thresholding

  • In very basic terms, thresholding is like a Low Pass Filter by allowing only particular color ranges to be highlighted as white while the other colors are suppressed by showing them as black.
  • I’ve used Otsu’s Binarization method. In this method, OpenCV automatically calculates/approximates the threshold value of a bimodal image from its image histogram. But for optimal results, we may need a clear background in front of the webcam which sometimes may not be possible.

4. Draw contours

5. Find convex hull and convexity defects

  • We now find the convex points and the defect points. The convex points are generally, the tip of the fingers. But there are other convex point too. So, we find convexity defects, which is the deepest point of deviation on the contour. By this we can find the number of fingers extended and then we can perform different functions according to the number of fingers extended.

You can reach me out over email, I’ll be happy to help 🙂