Creating funny caricatures with automatic face morphing

4 minute read

Any political attribution to this post is purely coincidence.

A sweet accident

This is my first post here and by coincidence it starts with an accident. A friend of mine told me he was making a video for promoting Vei a Vei, a neighborhood association for helping people who temporarily require food support and other basic necessities. He was using some semi-commercial software to mimic the morphing effect popularized by the marvelous video-clip Black or White. However, because he had to pay some bills for unlocking some functional features, he was doing it quite manually in the end. I wanted to help him, so after a quick search on the Internet, I ended up playing around with a very fancy face morphing code. I wanted to make it easier for my friend, a plug-and-play thing, or at least something to be run in the terminal prompt with a single execution instruction. The original post required manually adding some extra facial landmarks and just a single morphing frame was generated. So, while adapting the code, I made the mistake of not clearing the landmarks coordinates for every frame iteration. That caused the landmarks for one of the faces to be completely transferred to the other. That first cartoon face I saw was so funny that I thought it was a joke hidden in the original code. But it wasn’t. It was my bug! A bit of uncertainty in digital arts is like salt and pepper for a good recipe. I heard once from the great Sergi Jordà that some randomness is a good ingredient for extending interactive systems to more expressive terrains. That comes to my mind often, to let the emotions and other unexpected facts be, stay around, just flow in the space and time.

A Hillary Clinton's caricature morphed with the Donal Trump's facial expression.

The morphing dream

Morphing is as old as the first human who was capable of imagining something being transformed into something else. It is the alchemist dreaming of transmuting organic matter into noble metals. It is the magic that converts the wand into a bouquet of flowers. It is the smooth transition by which two objects become the another; two connected states of two different materials, or simply the creation of a single object having an untangled dual state.

Back on Earth, digital morphing became popular due to multiple films and video clips during the 90s. Back then, techniques outlined curvatures in both the source and target images, to later match them by applying some sort of spatial transformation, like the Beier-Neely algorithm. However, the procedure to create a single clip was really tedious. Nowadays, the object recognition algorithms based on computer vision have made it possible to drastically automate the procedure, especially for human faces. For instance, Dlib is a great toolkit for automatically detecting face landmarks in a way that it was hard to believe was feasible just ten years ago, high-end smartphones allow users to do all kinds of face-based magic tricks, and popular apps like FaceApp create awesome make-up effects, surprising everyone by making the paintings at Rijksmuseum smile amusingly.

Foo
Antonio de la Gándara (1861-1917) - The cabaret's gentlemen and four artists of the Chat Noir (1984).

Let’s morph it

The FaceMorph code performs automatically the following steps:

  1. Facial landmarks recognition on both the origin and target faces (Dlib).
  2. Triangular Delaunay segmentation.
  3. Affine transformation between the Delaunay triangles of both faces.
  4. Alpha blending on the paired triangles with a given transparency.

To create a caricature effect, the alpha blend on it must be set to either the maximum or the minimum values in order to make completely appear one of the faces, and then the affine transformation must target the landmarks of the other face.

A great choice for shaping funny caricatures is to pair faces with completely different facial dimensions between eyes, nose or mouth. Not only grins, but also disparate poses can generate crazy distortions too.

A Donald Trump's caricature morphed with the Hillary Clinton's facial expression.
A Hillary Clinton's caricature morphed with the Velazquez's facial expression.

In some cases however if the poses between the two faces are so distant, the morphing effect gets too much distorted or simply broken. That’s the case when the two faces are looking at opposite directions or they are located at very far away relative locations in the image. For the first issue, an horizontal flip can make the trick, while for the second one, a fair enough head alignment towards the center can work fine.

Caricatures of Van Gogh and Paul Goughin morphed with each other's facial expression. Note the Paul Goughin's self-portrait was originally heading to the right side of the paining.

Now we rise the score with a creepy series of disturbing faces. Note that some of them are limit cases in which some facial parts get too distorted, but still worked pretty good.

A couple of caricatures of Galileo Galilei morphed with the Hillary Clinton's (left) and Velazquez's (right) facial expressions.
A Velazquez's caricature morphed with the Jim Carrey's facial expression.
A Batman's (Christian Bale) caricature with the Joker's (Heath Ledger) facial expression.

That’s all folks. Keep morphing and have fun.