Detecting Eyes With Python & OpenCv

Lately, I have been working on an image processing application for nokia’s n900. One part of the idea is to have automatic red eye detection and removal, to make everything easier for the user. My previous post was about getting the latest opencv version to work with python bindings on n900. I suggest checking it out, in case you are also interested on mobile development. A big thanks for all this goes to http://nashruddin.com/OpenCV_Eye_Detection

Starting out

First, lets start from the top. Import opencv and create a loader for the data.

import cv

def Load():

image = cv.LoadImage(“images/dallas.jpg”)
faceCascade = cv.Load(“../haarcascades/haarcascade_frontalface_alt.xml”)
eyeCascade = cv.Load(“../haarcascades/haarcascade_eye.xml”)
return (image, faceCascade, eyeCascade)

So, here we load the image, a haarcascade for the face recognition and a haarcascade for eye recognition. Then return all these for later use. After this, lets do the display and test everything works fine.

def Display(image):

cv.NameWindow(“Red Eye Test”)
cv.ShowImage(“Red Eye Test”, image)
cv.WaitKey(0)
cv.DestroyWindow(“Red Eye Test”)

Okay, so the Display takes image as a parameter (which has to be loaded before to cv). Then we just create a display window and show the image until the user presses any key. Now you should see an image of me when you just call these two files.

image, faceCascade, eyeCascade = Load()
Display(image)

Into the REAL business

Okay, so now it is time to go to the actual business! First, we should start with creating the detection function.

def DetectRedEyes(image, faceCascade, eyeCascade):

min_size = (20,20)
image_scale = 2
haar_scale = 1.2
min_neighbors = 2
haar_flags = 0

So far nothing very special has happened. In the beginning we just define the minimum size of the detected area (these relate to faces, so if the face is smaller than 20×20, it shouldn’t be detected), how small to scale the image, etc.

# Allocate the temporary images
gray = cv.CreateImage((image.width, image.height), 8, 1)
smallImage = cv.CreateImage((cv.Round(image.width / image_scale),
cv.Round (image.height / image_scale)), 8 ,1)

# Convert color input image to grayscale
cv.CvtColor(image, gray, cv.CV_BGR2GRAY)

# Scale input image for faster processing
cv.Resize(gray, smallImage, cv.CV_INTER_LINEAR)

# Equalize the histogram
cv.EqualizeHist(smallImage, smallImage)

In the first row here we create a gray image and a small image to make processing less resource intensive (the computer still is pretty good at recognizing stuff from smaller images!).

# Detect the faces
faces = cv.HaarDetectObjects(smallImage, faceCascade, cv.CreateMemStorage(0),
haar_scale, min_neighbors, haar_flags, min_size)

Now, here we detect the faces from the picture. We use HaarDetectObjects to find the faces from the smaller image we created beforehand. For more information about the HaarDetectObjects(), check http://opencv.willowgarage.com/documentation/python/object_detection.html#haardetectobjects

# If faces are found
if faces:

for ((x, y, w, h), n) in faces:
# the input to cv.HaarDetectObjects was resized, so scale the
# bounding box of each face and convert it to two CvPoints
pt1 = (int(x * image_scale), int(y * image_scale))
pt2 = (int((x + w) * image_scale), int((y + h) * image_scale))
cv.Rectangle(image, pt1, pt2, cv.RGB(255, 0, 0), 3, 8, 0)

Here We just detect the faces and draw rectangles around the found faces.

# Estimate the eyes position
# First, set the image region of interest
# The last division removes the lower part of the face to lower probability for false recognition
cv.SetImageROI(image, (pt1[0],
pt1[1],
pt2[0] – pt1[0],
int((pt2[1] – pt1[1]) * 0.6)))

Okay, so after finding the faces, we create a region of interest, from where the eyes should be found. The area is actually the same as the rectangle drawn, EXCEPT for the lower part of the image, where 1/3 of the face is left out (normally this means the mouth area). This is just to make the recognition a little bit faster and to lessen the amount of faulty eyes found. Underneath is a screenshot of what the image will look like after setting the region of interest.

Now, all that is left, is to repeat the same kind of operation that was done to the whole picture to find faces, but this time to find eyes from the region of interest.

# Detect the eyes
eyes = cv.HaarDetectObjects(image, eyeCascade,
cv.CreateMemStorage(0),
haar_scale, min_neighbors,
haar_flags, (20,15))

# If eyes were found
if eyes:

# For each eye found
for eye in eyes:

# Draw a rectangle around the eye
cv.Rectangle(image,
(eye[0][0],
eye[0][1]),
(eye[0][0] + eye[0][2],
eye[0][1] + eye[0][3]),
cv.RGB(255, 0, 0), 1, 8, 0)

# Finally, reset the image region of interest (otherwise this won’t
# be drawn correctly
cv.ResetImageROI(image)

This is the final and important part. If you do not call the ResetImageROI, the only thing that will be shown, is the small cropped (region of interest) area of the picture with red squares drawn around the found faces. So, it is important to remember to call this ResetImageROI(). All that is now left, is to return the image.

return image

Now, we should just add all these function calls to main and things should go just fine.

if __name__ == “__main__”:

image, faceCascade, eyeCascade = Load()
image = DetectRedEyes(image, faceCascade, eyeCascade)
Display(image)

This is what you should get as a result:

So, this concludes the part of detecting the eyes. In the next part I will add the possibility for removing the found red eyes (if there are any, in my example picture there are none).

7 thoughts on “Detecting Eyes With Python & OpenCv

  1. Pingback: Opencv 2.2 python examples « « Memory Dump Memory Dump

  2. Do you have the full working source code that would actually work on python 2.7? I am actually using openCV 2.2. Thank You in anticipation.

  3. Hi, i assembled your code with a few modification to get input from the webcam, and here is it:

    ##############################################

    import cv
    import time
    import Image

    cv.NamedWindow(“camera”, 1)
    capture = cv.CreateCameraCapture(0)

    width = None
    height = None
    width = 320
    height = 240

    if width is None:
    width = int(cv.GetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_WIDTH))
    else:
    cv.SetCaptureProperty(capture,cv.CV_CAP_PROP_FRAME_WIDTH,width)

    if height is None:
    height = int(cv.GetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_HEIGHT))
    else:
    cv.SetCaptureProperty(capture,cv.CV_CAP_PROP_FRAME_HEIGHT,height)

    result = cv.CreateImage((width,height),cv.IPL_DEPTH_8U,3)

    def Load():

    return (faceCascade, eyeCascade)

    def Display(image):
    cv.NamedWindow(“Red Eye Test”)
    cv.ShowImage(“Red Eye Test”, image)
    cv.WaitKey(0)
    cv.DestroyWindow(“Red Eye Test”)

    def DetectRedEyes(image, faceCascade, eyeCascade):
    min_size = (20,20)
    image_scale = 2
    haar_scale = 1.2
    min_neighbors = 2
    haar_flags = 0

    # Allocate the temporary images
    gray = cv.CreateImage((image.width, image.height), 8, 1)
    smallImage = cv.CreateImage((cv.Round(image.width / image_scale),cv.Round (image.height / image_scale)), 8 ,1)

    # Convert color input image to grayscale
    cv.CvtColor(image, gray, cv.CV_BGR2GRAY)

    # Scale input image for faster processing
    cv.Resize(gray, smallImage, cv.CV_INTER_LINEAR)

    # Equalize the histogram
    cv.EqualizeHist(smallImage, smallImage)

    # Detect the faces
    faces = cv.HaarDetectObjects(smallImage, faceCascade, cv.CreateMemStorage(0),
    haar_scale, min_neighbors, haar_flags, min_size)

    # If faces are found
    if faces:
    for ((x, y, w, h), n) in faces:
    # the input to cv.HaarDetectObjects was resized, so scale the
    # bounding box of each face and convert it to two CvPoints
    pt1 = (int(x * image_scale), int(y * image_scale))
    pt2 = (int((x + w) * image_scale), int((y + h) * image_scale))
    cv.Rectangle(image, pt1, pt2, cv.RGB(255, 0, 0), 3, 8, 0)
    face_region = cv.GetSubRect(image,(x,int(y + (h/4)),w,int(h/2)))

    cv.SetImageROI(image, (pt1[0],
    pt1[1],
    pt2[0] – pt1[0],
    int((pt2[1] – pt1[1]) * 0.7)))
    eyes = cv.HaarDetectObjects(image, eyeCascade,
    cv.CreateMemStorage(0),
    haar_scale, min_neighbors,
    haar_flags, (15,15))

    if eyes:
    # For each eye found
    for eye in eyes:
    # Draw a rectangle around the eye
    cv.Rectangle(image,
    (eye[0][0],
    eye[0][1]),
    (eye[0][0] + eye[0][2],
    eye[0][1] + eye[0][3]),
    cv.RGB(255, 0, 0), 1, 8, 0)

    cv.ResetImageROI(image)
    return image

    faceCascade = cv.Load(“haarcascade_frontalface_default.xml”)
    eyeCascade = cv.Load(“haarcascade_eye.xml”)

    while True:
    img = cv.QueryFrame(capture)

    image = DetectRedEyes(img, faceCascade, eyeCascade)
    cv.ShowImage(“camera”, image)
    k = cv.WaitKey(1);
    if k == ‘f’:
    break

    #########################################################

    But what i get is just noise in the output window. Once i did get the red boxes around my eyes, but afterwards i’ve been getting only noise. Could you tell me what the problem might be? I mean i saw my eyes getting detected, and then the next time i run it, i get just noise! Like on TV

    • Traceback (most recent call last):
      File “ashis.py”, line 98, in
      image = DetectRedEyes(img, faceCascade, eyeCascade)
      File “ashis.py”, line 43, in DetectRedEyes
      gray = cv.CreateImage((image.width, image.height), 8, 1)
      AttributeError: ‘NoneType’ object has no attribute ‘width’

      I can’t use your code, why?

  4. พี่ครับสอนผมหน่อยดิ

  5. Pingback: Detecting Eyes With Python & OpenCv | Iyke & Ji's little corner

  6. awesome article . thanks japskua🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s