Removing Red Eyes with OpenCV and Python

As I wrote in my blog yesterday, I started working on automatic red eye removal with python and OpenCV. Today, I finally managed to get it working (despite of the small amount of fever, which lead to some very interesting solutions :-P). The system works on the few pictures I have tested. Of course, it is not perfect, but should be at least somewhat acceptable.

The First part of the program is from the yesterday’s blog (eye detection), that needs to be run in order to get the information required for the red eye removal (yeah, you really need to have eyes recognized for this to work!), I have included the DetectRedEyes here, because I did some modification to it.

Detecting the Eyes

I’m not going to comment this much. Just to note, that I have removed drawing the rectangles around the eyes and replaced that with adding the information to an eyesList, which should contain the coordinates for the eyes in the big picture. I will skip to the changed eyes part…

def DetectEyes(image, faceCascade, eyeCascade):

“””

“””

“””

# Detect the eyes

eyes = cv.HaarDetectObjects(image, eyeCascade,
cv.CreateMemStorage(0),
haar_scale, min_neighbors,
haar_flags, (20,15))

eyesList = []

# If eyes were found
if eyes:

# For each eye found,
for eye in eyes:

# We save the locations on the BIG picture
# and append them to the eyesList

eyeX1 = eye[0][0]
eyeY1 = eye[0][1]
eyeX2 = eye[0][0] + eye[0][2]
eyeY2 = eye[0][1] + eye[0][3]

correctPosition = (eyeX1 + point1,
eyeY1 + point2,
eyeX2 + point1,
eyeY2 + point2)

eyesList.append(correctPosition)

# Finally, reset the image region of
# interest (otherwise this won’t be drawn correctly

cv.ResetImageROI(image)

return image, eyesList

Removing Red Eyes

def RemoveRedEyes(image, eyes, threshold=2.1):

# Check for existence of the eyes list
if eyes:

# Go through the eyes list [one eye at a time]
for position in eyes:

# Rename the positions, this is just for code readability
eyeX1 = position[0]
eyeY1 = position[1]
eyeX2 = position[2]
eyeY2 = position[3]

# Set the image Region of interest to be the eye area [this reduces processing time]
cv.SetImageROI(image, (eyeX1, eyeY1, eyeX2 – eyeX1, eyeY2 – eyeY1))

Okay, so first to the function. RemoveRedEyes(image, eyes, threshold=2.1). The image is the loaded up image in OpenCV, and the eyes is the list of the eyes in the DetectEyes(). Then We start going through the eyes list. The eyeX1, eyeY1, etc… are just used for making the code easier to read and understand. In the last row, we set the image region of interest to be the area of the eye in question. This reduces the processing time notably, because there is now only small amount of the image to be processed (not the complete 5+ Megapixels).

# Gets the image pixel i,j
# Then the channel value B,G,R order n with pixel[n]
# Get the size of the image (this time image means only the region of # interest, It has been “cropped” at the current time to be smaller)

sizeX, sizeY = cv.GetSize(image)
i = 0
j = 0

# Go through all the pixels in the image
while i < sizeX:
j = 0

while j < sizeY:

# This gets the pixel in question
pixel = cv.Get2D(image, i, j)

# Calculate the intensity compared to blue and green average
redIntensity = pixel[2] / ((pixel[1] + pixel[0]) / 2)

# If the red intensity is greater than 2.0, lower the value
if redIntensity >= threshold:

newRedValue = (pixel[1] + pixel[0]) / 2
# Insert the new red value for the pixel to the image
cv.Set2D(image, i, j,
cv.RGB(newRedValue, pixel[1], pixel[0]))

j += 1

i += 1

# Reset the image region of interest back to full image
cv.ResetImageROI(image)

# Return the resulting image
return image

Now, the hard part was here. After all, understanding that is the key to the SUCESSION! First, we get the size of the image with cv.GetSize(image). Now we use this information to go through all the pixels of the selected region.

pixel = cv.Get2D(image, i, j)

redIntensity = pixel[2] / ((pixel[1] + pixel[0]) / 2)

Here we load the (i,j) coordinate pixel and then calculate the intensity of the red in the picture. The pixel values in (at least current version of) openCV go here in order Blue, Green, Red. So pixel[0] means blue value, pixel[1] green value and pixel[2] means the red value.
Here we calculate the red intensity by dividing the red value by the average of blue and green. If the red is a lot greater than the other values, we certainly have more red. (And because we are talking of the eyes here, this shouldn’t be normal – at least I don’t know any red eyed people).

If redIntensity >= threshold:
newRedValue = (pixel[1] + pixel[0]) / 2
cv.Set2D(image, I, j,
cv.RGB(newRedValue, pixel[1], pixel[0]))

Here, if the red intensity of greater than the threshold given at the beginning (default = 2.0), we should do something. The new red value is given according to the average of the blue and green values, so it is scaled down to the same level. In the last row we just insert the new red value to the pixel in question to the big image.

Now, just run the whole thing:

if __name__ == “__main__”:

image, cascade, eyeCascade = Load()
image, eyes = DetectEyes(image, cascade, eyeCascade)
image2, cascade, eyeCascade = Load()
RemoveRedEyes(image2, eyes)
cv.ShowImage(“Changed image”, image2)
Display(image)

Some Results

So, here are some results of the changes I have managed to get. In this first picture, we can see that the baby has a little bit of red eyes left, because the threshold level is so high. If we would lower it a little bit, say to 1.5, the result would be following:

However, this will cause trouble with other pictures. For example the picture of me, which I used as an example yesterday has too much red color (well, it does not have red eyes, but it could have!). When we change the threshold here down to 1.5, the whole region of interest area gets a little bit messed up.

But Again, if we return back to the 2.1 (just my hunch), we don’t get this error.

Okay, so have fun testing this out. The source code with all the image files and haarcascades is included here.

http://dl.dropbox.com/u/4692161/red_eye_removal.tar.gz