Aligning faces with py opencv-dlib combo

Face alignment with Dlib and OpenCV

This is my first trial at using Jupyter notebook to write a post, hope it makes sense.

I've recently taught a class on generative models: http://hi.cs.stonybrook.edu/teaching/cdt450

In class we've manipulated face images with neural networks.

One important thing I found that helped is to align the images so the facial features overlap.
It helps the nets learn the variance in faces better, rather than waste their "representation power" on the shift between faces.

The following is some code to align face images using the excellent Dlib (python bindings) http://dlib.net. First I'm just using a standard face detector, and then using the facial fatures extractor I'm using that information for a complete alignment of the face.

After the alignment - I'm just having fun with the aligned dataset 🙂

First we include some necessary packages:

A utility to download and unzip/untar a file

Aligning faces images

This function will crop and scale the faces in the image with a facial landmark detector, to match the alignment of the faces in the celebA dataset (http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html).

To use it, download http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2 first.

---

Download the faces dataset

We will use the FDDB: http://vis-www.cs.umass.edu/fddb/

Align the faces

Use the face alignment tool

Visualize the faces dataset

Aligned

Not aligned

---

A tool to produce a GIF from lists of images, also able to apply an average sliding window between them

Face Averages

Animations:

Aligned

Not aligned

Naturally, the aligned faces have more facial features in common.

---

Variational Autoencoder

A callback to render the losses as a graph

Train the VAE

Apply VAE to find faces in inanimate objects

Objects were downloaded from Google Images search - they may be copyrighted! (Oops)

Inspired by this Medium post: https://medium.com/starts-with-a-bang/averaging-inanimate-objects-can-produce-human-faces-1a80cd1448d8

Morph between faces with the VAE

And an animation morph GIF:


Hope you enjoyed!

(All sources are license-free, use them at will! But don't blame me if something breaks. Also please don't use my name to endorse your project)

Share