Face Detection using OpenCV

Facial recognition is a way of recognizing a human face through technology. A facial recognition system uses facial features from a photograph or video to confirm it is a face.

  • To compare the information with a database of known faces to find a match.
  • DeepFace is the technology used by Facebook to recognize a face

video-face-recognition-opecv

HAAR cascades:

A Haar Cascade is a classifier that is used to detect the object for which it has been trained. The Haar cascade is trained by superimposing the positive image over a set of images. This type of training is generally done on a server and on various stages.

Better results are obtained by using high-quality images and increasing the number of stages for which the classifier is trained for. Cascade makes it easier to build a model. We are using the pre-defined Haar cascade which is available on https://github.com/opencv/opencv/tree/master/data/haarcascades

You can create own cascade files.

The important method used:

  • CascadeClassifier : Loads the cascade XML file and it is present in OpenCV
  • detectMultiScale: detects the one/more faces present in the image
  • cvtColor: short for convertColor, converts the color of an image.
  • rectangle : Draws a rectangle over given coordinates
  • VideoCapture: captures the video from the webcam if the value is 0 (internal webcam), if value 1 then external camera;
  • imshow: Showcases image on a pop
  • destroyAllWindows: Kill all the popups opened by the current program

Face detection based on Image with OpenCV

Pre-requisites:

Steps to build the program:

  • Import the CV2 module to the program
    import cv2​
  • Load the cascade for face
    #Load the cascade
    face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')​
  • Read the image from the local system
    #Read the input image
    img = cv2.imread("pavan-face-recognition-opencv.png")​
  • Convert the Image to Grayscale because image recognition and haar features work with grayscale
    #Convert into GrayScale (face detection works better on Grayscale)
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)​
  • Detect the face/faces if one or more present
    #Detect Faces
    faces = face_cascade.detectMultiScale(gray, 1.1, 4)​
  • Draw box/rectangle over the detected faces.
    #Draw rectangle around each faces
    for (x, y, w, h) in faces:
        cv2.rectangle(img, (x, y), (x + w, y + h), (255, 0, 0), 2)​
  • Show the image in a popup window
    #Display the output
    cv2.imshow('Some title', img)​
  • Provide option input to close the window.
    # press any key to close the popup
    cv2.waitKey()​

The complete code for the face detection on an image.

import cv2
#Load the cascade
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
#Read the input image
img = cv2.imread("pavan-face-recognition-opencv.png")
#Convert into GrayScale (face detection works better on Grayscale)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
#Detect Faces
faces = face_cascade.detectMultiScale(gray, 1.1, 4)
#Draw rectangle around each faces
for (x, y, w, h) in faces:
    cv2.rectangle(img, (x, y), (x + w, y + h), (255, 0, 0), 2)
#Display the output
cv2.imshow('Some title', img)
# press any key to close the popup
cv2.waitKey()

face-recognized-opencv-image

If there is more than one image, still the above program holds good.
multiple-faces-detection-opencv

If you notice the above image, there is a box on the shirt of the right side person. So Open CV can give wrong answers as well. The result purely depends on the cascade file we use.

Face and eyes detection in OpenCV based on Image

In the above program, we have created a program to find the face. When it comes to the real-time scenarios you are not only going to find the face but you will use eys and other features of the face.

In the below code I have made changes to the above program to find the eys as well.

We have found the face based on the total image; and yes also will be present inside the total image but when we search for the eyes in the total image then our search will take more time; to avoid this we use the kindergarten concept that Eyes are present in the face.

So instead of searching for the eyes in total images, we are going to search for eyes only inside the face.

Changes to the above face detection program to find eyes as well :
  • Load the cascade for the eyes: https://github.com/opencv/opencv/blob/master/data/haarcascades/haarcascade_eye.xml
  • Use the area of the face only to detect the eyes
        # consider only area when we have face
        roi_gray = gray[y:y+h, x:x+w]
        roi_color = img[y:y+h, x:x+w]​
  • Detect the eyes
    # detect eys cascade
    eyes = eye_cascade.detectMultiScale(roi_gray, 1.1, 3)​
  • Draw rectangle around eyes
    for (ex, ey, ew, eh) in eyes:
            #draw rectangle around the eyes
            cv2.rectangle(roi_color, (ex, ey), (ex+ew, ey+eh), (0, 255, 0), 2)​

Find the eyes on the image using OpenCV

import cv2
#Load the cascade
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
eye_cascade = cv2.CascadeClassifier('haarcascade_eye.xml')
#Read the input image
img = cv2.imread("pavan-face-recognition-opencv.png")
#Convert into GrayScale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
#Detect Faces
faces = face_cascade.detectMultiScale(gray, 1.1, 4)
#Draw rectangle around each faces
for (x, y, w, h) in faces:
    cv2.rectangle(img, (x, y), (x + w, y + h), (255, 0, 0), 2)
    
    # consider only area when we have face
    roi_gray = gray[y:y+h, x:x+w]
    roi_color = img[y:y+h, x:x+w]
    #detect eyes
    eyes = eye_cascade.detectMultiScale(roi_gray, 1.1, 3)
    for (ex, ey, ew, eh) in eyes:
        #draw rectangle around the eyes
        cv2.rectangle(roi_color, (ex, ey), (ex+ew, ey+eh), (0, 255, 0), 2)
#Display the output
cv2.imshow('Some title', img)
cv2.waitKey()



eyes-detection-opencv

Just for fun:

What happens if I have four eyes on the face.
pavan-face-recognition-opencv-four-eyes

Output: it was able to find three eyes.
three-eyes-detection-opencv

Face detection is based on Live video

If you are before 1995, then you could have seen the movies were playing using the film strip. The film strip will be pulled at 24 images per second speed so that the images make a video.
film-strip-opencv

Another example is the flipbook when flipbook pages are move at a speed we see a video.

Steps to detect the face and eyes based on the video:

By this time, you right have understood that video is made up of multiple images. So we will be applying face detection on each image present in the video.

  • Import OpenCV
  • Define the Cascades
    # Loading the cascades
    face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
    eye_cascade = cv2.CascadeClassifier('haarcascade_eye.xml')​
  • Create a method to find faces
    # Defining a function that will do the detections
    def detect_faces(frame):​

    • Convert the images to grayscale
      gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    • Detect the face on the image
      faces = face_cascade.detectMultiScale(gray, 1.3, 5)​​
    • Draw rectangle over the face
      for (x, y, w, h) in faces:
              cv2.rectangle(frame, (x, y), (x+w, y+h), (255, 0, 0), 2)​

      • Consider the area which has a face for finding the eyes rather than a complete image.
         roi_gray = gray[y:y+h, x:x+w]
         roi_color = frame[y:y+h, x:x+w]​
      • Detect the eyes
        eyes = eye_cascade.detectMultiScale(roi_gray, 1.1, 3)​
      • Draw rectangle around then eyes
         for (ex, ey, ew, eh) in eyes:
                    cv2.rectangle(roi_color, (ex, ey), (ex+ew, ey+eh), (0, 255, 0), 2)​
    • Return the image.
      return frame​
  • Capture the video from the Web camera and store it
    # Doing some Face Recognition with the webcam
    video_capture = cv2.VideoCapture(0)​
  • Loop till the loop is broke
  • Read each Frame and Pass it to the method we have created above
  • Show the image returned in a popup window
  • Break the loop if the user presses q key in keyboard
    while True:
        _, frame = video_capture.read()
        canvas = detect_faces(frame)
        cv2.imshow('Video', canvas)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break​
  • Then end the program by closing the popup window

The complete program to find faces in Web camera

# Importing the libraries
import cv2

# Loading the cascades
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
eye_cascade = cv2.CascadeClassifier('haarcascade_eye.xml')

# Defining a function that will do the detections
def detect_faces(frame):
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    faces = face_cascade.detectMultiScale(gray, 1.3, 5)
    for (x, y, w, h) in faces:
        cv2.rectangle(frame, (x, y), (x+w, y+h), (255, 0, 0), 2)
        roi_gray = gray[y:y+h, x:x+w]
        roi_color = frame[y:y+h, x:x+w]
        eyes = eye_cascade.detectMultiScale(roi_gray, 1.1, 3)
        for (ex, ey, ew, eh) in eyes:
            cv2.rectangle(roi_color, (ex, ey), (ex+ew, ey+eh), (0, 255, 0), 2)
    return frame

# Doing some Face Recognition with the webcam
video_capture = cv2.VideoCapture(0)
while True:
    _, frame = video_capture.read()
    canvas = detect_faces(frame)
    cv2.imshow('Video', canvas)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
video_capture.release()
cv2.destroyAllWindows()

The output

What happens when we close one side of the face.
no-face-with-one-eye-opencv

Working with video from the local system

We can load the video files which are present in our file system to apply face detection on the video file.

Only the second half code is changed.

  • Read the video just by providing the filename to the VideoCapture
    # Doing some Face Recognition with the webcam
    video_capture = cv2.VideoCapture("Video.mkv")​
  • Loop till the video is opened (break if the video is closed by external forces)
    while video_capture.isOpened():
  • Find the faces, as long as video present; break the loop if there is no more frame available (or video reached the end)
    ret, frame = video_capture.read()
        if ret == True:
            canvas = detect_faces(frame)
            cv2.imshow('Video', canvas)
            if cv2.waitKey(1) & 0xFF == ord('q'):
                break
        else:
            break
    video_capture.release()
    cv2.destroyAllWindows()​
# Doing some Face Recognition with the webcam
video_capture = cv2.VideoCapture("Video.mkv")
while video_capture.isOpened():
    ret, frame = video_capture.read()
    if ret == True:
        canvas = detect_faces(frame)
        cv2.imshow('Video', canvas)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
    else:
        break
video_capture.release()
cv2.destroyAllWindows()
# Importing the libraries
import cv2

# Loading the cascades
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
eye_cascade = cv2.CascadeClassifier('haarcascade_eye.xml')

# Defining a function that will do the detections
def detect_faces(frame):
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    faces = face_cascade.detectMultiScale(gray, 1.3, 5)
    for (x, y, w, h) in faces:
        cv2.rectangle(frame, (x, y), (x+w, y+h), (255, 0, 0), 2)
        roi_gray = gray[y:y+h, x:x+w]
        roi_color = frame[y:y+h, x:x+w]
        eyes = eye_cascade.detectMultiScale(roi_gray, 1.1, 3)
        for (ex, ey, ew, eh) in eyes:
            cv2.rectangle(roi_color, (ex, ey), (ex+ew, ey+eh), (0, 255, 0), 2)
    return frame

# Doing some Face Recognition with the webcam
video_capture = cv2.VideoCapture("Video.mkv")
while video_capture.isOpened():
    ret, frame = video_capture.read()
    if ret == True:
        canvas = detect_faces(frame)
        cv2.imshow('Video', canvas)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
    else:
        print("No more frames available")
        break
video_capture.release()
cv2.destroyAllWindows()

video-from-file-steam-opencv

Video from URL with OpenCV for Face detection

Video from URL does not have much difference with file system video, only thing is instead of local path you have to give the URL in VideoCapture().

Video is present at https://chercher.tech/files/video-from-website.mkv

# Importing the libraries
import cv2

# Loading the cascades
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
eye_cascade = cv2.CascadeClassifier('haarcascade_eye.xml')

# Defining a function that will do the detections
def detect_faces(frame):
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    faces = face_cascade.detectMultiScale(gray, 1.3, 5)
    for (x, y, w, h) in faces:
        cv2.rectangle(frame, (x, y), (x+w, y+h), (255, 0, 0), 2)
        roi_gray = gray[y:y+h, x:x+w]
        roi_color = frame[y:y+h, x:x+w]
        eyes = eye_cascade.detectMultiScale(roi_gray, 1.1, 3)
        for (ex, ey, ew, eh) in eyes:
            cv2.rectangle(roi_color, (ex, ey), (ex+ew, ey+eh), (0, 255, 0), 2)
    return frame

# Doing some Face Recognition with the webcam
video_capture = cv2.VideoCapture('https://chercher.tech/files/video-from-website.mkv')
while video_capture.isOpened():
    ret, frame = video_capture.read()
    if ret == True:
        canvas = detect_faces(frame)
        cv2.imshow('Video', canvas)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
    else:
        print("No more frames available")
        break
video_capture.release()
cv2.destroyAllWindows()

video-from-url-opencv

About Author :

I am Pavankumar, Having 8.5 years of experience currently working in Video/Live Analytics project.

Comment / Suggestion Section
Point our Mistakes and Post Your Suggestions