Removing Red Eyes with OpenCV and Python

As I wrote in my blog yesterday, I started working on automatic red eye removal with python and OpenCV. Today, I finally managed to get it working (despite of the small amount of fever, which lead to some very interesting solutions :-P). The system works on the few pictures I have tested. Of course, it is not perfect, but should be at least somewhat acceptable.

The First part of the program is from the yesterday’s blog (eye detection), that needs to be run in order to get the information required for the red eye removal (yeah, you really need to have eyes recognized for this to work!), I have included the DetectRedEyes here, because I did some modification to it.

Detecting the Eyes

I’m not going to comment this much. Just to note, that I have removed drawing the rectangles around the eyes and replaced that with adding the information to an eyesList, which should contain the coordinates for the eyes in the big picture. I will skip to the changed eyes part…

def DetectEyes(image, faceCascade, eyeCascade):

“””

“””

“””

# Detect the eyes

eyes = cv.HaarDetectObjects(image, eyeCascade,
cv.CreateMemStorage(0),
haar_scale, min_neighbors,
haar_flags, (20,15))

eyesList = []

# If eyes were found
if eyes:

# For each eye found,
for eye in eyes:

# We save the locations on the BIG picture
# and append them to the eyesList

eyeX1 = eye[0][0]
eyeY1 = eye[0][1]
eyeX2 = eye[0][0] + eye[0][2]
eyeY2 = eye[0][1] + eye[0][3]

correctPosition = (eyeX1 + point1,
eyeY1 + point2,
eyeX2 + point1,
eyeY2 + point2)

eyesList.append(correctPosition)

# Finally, reset the image region of
# interest (otherwise this won’t be drawn correctly

cv.ResetImageROI(image)

return image, eyesList

Removing Red Eyes

def RemoveRedEyes(image, eyes, threshold=2.1):

# Check for existence of the eyes list
if eyes:

# Go through the eyes list [one eye at a time]
for position in eyes:

# Rename the positions, this is just for code readability
eyeX1 = position[0]
eyeY1 = position[1]
eyeX2 = position[2]
eyeY2 = position[3]

# Set the image Region of interest to be the eye area [this reduces processing time]
cv.SetImageROI(image, (eyeX1, eyeY1, eyeX2 – eyeX1, eyeY2 – eyeY1))

Okay, so first to the function. RemoveRedEyes(image, eyes, threshold=2.1). The image is the loaded up image in OpenCV, and the eyes is the list of the eyes in the DetectEyes(). Then We start going through the eyes list. The eyeX1, eyeY1, etc… are just used for making the code easier to read and understand. In the last row, we set the image region of interest to be the area of the eye in question. This reduces the processing time notably, because there is now only small amount of the image to be processed (not the complete 5+ Megapixels).

# Gets the image pixel i,j
# Then the channel value B,G,R order n with pixel[n]
# Get the size of the image (this time image means only the region of # interest, It has been “cropped” at the current time to be smaller)

sizeX, sizeY = cv.GetSize(image)
i = 0
j = 0

# Go through all the pixels in the image
while i < sizeX:
j = 0

while j < sizeY:

# This gets the pixel in question
pixel = cv.Get2D(image, i, j)

# Calculate the intensity compared to blue and green average
redIntensity = pixel[2] / ((pixel[1] + pixel[0]) / 2)

# If the red intensity is greater than 2.0, lower the value
if redIntensity >= threshold:

newRedValue = (pixel[1] + pixel[0]) / 2
# Insert the new red value for the pixel to the image
cv.Set2D(image, i, j,
cv.RGB(newRedValue, pixel[1], pixel[0]))

j += 1

i += 1

# Reset the image region of interest back to full image
cv.ResetImageROI(image)

# Return the resulting image
return image

Now, the hard part was here. After all, understanding that is the key to the SUCESSION! First, we get the size of the image with cv.GetSize(image). Now we use this information to go through all the pixels of the selected region.

pixel = cv.Get2D(image, i, j)

redIntensity = pixel[2] / ((pixel[1] + pixel[0]) / 2)

Here we load the (i,j) coordinate pixel and then calculate the intensity of the red in the picture. The pixel values in (at least current version of) openCV go here in order Blue, Green, Red. So pixel[0] means blue value, pixel[1] green value and pixel[2] means the red value.
Here we calculate the red intensity by dividing the red value by the average of blue and green. If the red is a lot greater than the other values, we certainly have more red. (And because we are talking of the eyes here, this shouldn’t be normal – at least I don’t know any red eyed people).

If redIntensity >= threshold:
newRedValue = (pixel[1] + pixel[0]) / 2
cv.Set2D(image, I, j,
cv.RGB(newRedValue, pixel[1], pixel[0]))

Here, if the red intensity of greater than the threshold given at the beginning (default = 2.0), we should do something. The new red value is given according to the average of the blue and green values, so it is scaled down to the same level. In the last row we just insert the new red value to the pixel in question to the big image.

Now, just run the whole thing:

if __name__ == “__main__”:

image, cascade, eyeCascade = Load()
image, eyes = DetectEyes(image, cascade, eyeCascade)
image2, cascade, eyeCascade = Load()
RemoveRedEyes(image2, eyes)
cv.ShowImage(“Changed image”, image2)
Display(image)

Some Results

So, here are some results of the changes I have managed to get. In this first picture, we can see that the baby has a little bit of red eyes left, because the threshold level is so high. If we would lower it a little bit, say to 1.5, the result would be following:

However, this will cause trouble with other pictures. For example the picture of me, which I used as an example yesterday has too much red color (well, it does not have red eyes, but it could have!). When we change the threshold here down to 1.5, the whole region of interest area gets a little bit messed up.

But Again, if we return back to the 2.1 (just my hunch), we don’t get this error.

Okay, so have fun testing this out. The source code with all the image files and haarcascades is included here.

http://dl.dropbox.com/u/4692161/red_eye_removal.tar.gz

8 thoughts on “Removing Red Eyes with OpenCV and Python

  1. Hey! Great stuff! Must. learn. HAAR. soon.😀

  2. Really great job. Could you attach your source code to this blog website so that I can readily download it or send it to my email box? It seems that I cannot access the dropbox. I don’t know why. Thanks!

  3. Sadly the wordpress site does not allow me to upload any non-media files. This is the reason why I had to upload it to my dropbox account. I sent you the source to your email. Enjoy😉

  4. Hi that great work, When I try convert your images it worked (baby and man), but when I convert some my images it return error:
    in DetectEyes
    return image, eyesList
    UnboundLocalError: local variable ‘eyesList’ referenced before assignment
    Could you help with this issue?

    • It sounds like there is no list made of the eyes. This probably means, that the algorithm did not find any eyes in the picture (it is not 100% perfect after all). Or perhaps it has something to do with the file type of the images? (I don’t think raw files will be recognized).

  5. I notice yet another issue upon photo.jpg file:
    Traceback (most recent call last):
    File “./test_rd.py”, line 216, in
    RemoveRedEyes(image2, eyes)
    File “./test_rd.py”, line 195, in RemoveRedEyes
    redIntensity = pixel[2] / ((pixel[1] + pixel[0]) / 2)

    If you could look at my problem about images it would be great. I a little edit your python file to take 2 arguments (-i as input file a and -o as output file):
    I add my files to http://www.go24.cz/test/test_rd.html
    I would like to know whether problem is in my images or in python file.
    Thank you for help

  6. Pingback: Фильтр "Красных глаз"

  7. Pingback: Image Manipulation: Red Eye Effect removal tools, techniques & solutions « Jai’s Weblog – Tech, Security & Fun…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s