Creating Funny Caricatures With Automatic Face Morphing

December 3, 2018 4 minute read

Any political attribution to this post is purely coincidence.

A sweet accident

This is my first post here and by coincidence it starts with an accident. A friend of mine told me he was making a video for promoting Vei a Vei, a neighborhood association for helping people who temporarily require food support and other basic necessities. He was using some semi-commercial software to mimic the morphing effect popularized by the marvelous video-clip Black or White. However, because he had to pay some bills for unlocking some functional features, he was doing it quite manually in the end. I wanted to help him, so after a quick search on the Internet, I ended up playing around with a very fancy face morphing code. I wanted to make it easier for my friend, a plug-and-play thing, or at least something to be run in the terminal prompt with a single execution instruction. The original post required manually adding some extra facial landmarks and just a single morphing frame was generated. So, while adapting the code, I made the mistake of not clearing the landmarks coordinates for every frame iteration. That caused the landmarks for one of the faces to be completely transferred to the other. That first cartoon face I saw was so funny that I thought it was a joke hidden in the original code. But it wasn’t. It was my bug! A bit of uncertainty in digital arts is like salt and pepper for a good recipe. I heard once from the great Sergi Jordà that some randomness is a good ingredient for extending interactive systems to more expressive terrains. That comes to my mind often, to let the emotions and other unexpected facts be, stay around, just flow in the space and time.

Hillary Clinton's caricature morphed with Donald Trump's facial expression.

The morphing dream

Morphing is as old as the first human who was capable of imagining something being transformed into something else. It is the alchemist dreaming of transmuting organic matter into noble metals. It is the magic that converts the wand into a bouquet of flowers. It is the smooth transition by which two objects become the another; two connected states of two different materials, or simply the creation of a single object having an untangled dual state.

Back on Earth, digital morphing became popular due to multiple films and video clips during the 90s. Back then, techniques outlined curvatures in both the source and target images, to later match them by applying some sort of spatial transformation, like the Beier-Neely algorithm. However, the procedure to create a single clip was really tedious. Nowadays, the object recognition algorithms based on computer vision have made it possible to drastically automate the procedure, especially for human faces. For instance, Dlib is a great toolkit for automatically detecting face landmarks in a way that it was hard to believe was feasible just ten years ago, high-end smartphones allow users to do all kinds of face-based magic tricks, and popular apps like FaceApp create awesome make-up effects, surprising everyone by making the paintings at Rijksmuseum smile amusingly.

Foo — Antonio de la Gándara (1861-1917) - The cabaret's gentlemen and four artists of the *Chat Noir* (1984).

Let’s morph it

FaceMorph performs automatically the following steps:

Facial landmarks recognition on both source and target images (Dlib).
Triangular Delaunay segmentation.
Affine transformation between Delaunay triangles of both faces.
Alpha blending on paired triangles with a given transparency.

To create a caricature effect, the alpha blend on it must be set to either the maximum or the minimum values in order to completely make one of the faces appear, and then the affine transformation must target the landmarks of the other face.

A great choice for shaping funny caricatures is to pair faces with completely different facial dimensions between the eyes, nose or mouth. Not only grins, but also disparate poses can generate crazy distortions too.

Donald Trump's caricature morphed with Hillary Clinton's facial expression.

Hillary Clinton's caricature morphed with Velazquez's facial expression.

In some cases, however, if the poses between the two faces are too distant, the morphing effect gets too distorted or simply broken. That’s the case when the two faces are looking in opposite directions or they are in different places in the image. For the first issue, an horizontal flip can do the trick, while for the second one, a reasonable head alignment, towards the center, will work.

Caricatures of Van Gogh and Paul Goughin morphed with each other's facial expression. Note the Paul Goughin's self-portrait was originally heading to the right side of the painting.

Now we rise the score with a disturbing series of creepy faces. Note that some of them are extreme cases in which some facial parts get too distorted, but it still work pretty well.