Creating funny caricatures with automatic face morphing

4 minute read

Any political attribution to this post is purely coincidence.

A sweet accident

This is my first post here and casually starts with an accident. A friend of mine told me he was making a video for promoting Vei a Vei, a neighborhood association for helping people who temporally require food support and other basic necessities. He was using some semi-commercial software to mimic the morphing effect popularized by the marvelous Black or White video-clip. But because he had to pay some bills for unlocking some functional features, he was doing it quite manually in the end. I wanted to help him. So after a quick search in Internet, I ended up playing around with a very fancy face morphing code. I wanted to make it easier for my friend, almost plug-and-play or at least something to be run in the terminal prompt with a single execution instruction. The original post required to add manually some extra facial landmarks and just a single morphing frame was generated. So while adapting the code, I made the mistake of not clearing the landmarks coordinates for every frame iteration. That caused the landmarks for one of the faces to be completely transfered to the other. That first cartoon face I saw was so funny that I though it was a joke hidden in the original code. But it wasn’t. It was my bug! A bit of uncertainty in digital arts are like salt and pepper for a good receipt. I heard once from the great Sergi Jordà that some randomness is a good ingredient for extending interactive systems to more expressive terrains. That comes to my mind often, to let the emotions and other unexpected facts be, stay around, just flow in the space and time.

Hillary Clinton's caricature morphed with Donal Trump's facial expression.

The morphing dream

Morphing is as old as the first human could image something to be transformed into another thing. It is the alchemist dreaming in transmuting organic matter into noble metals. It is the magic that converts the wand into a bouquet of flowers. It is the smooth transition in which two objects become one another. Two connected states of two different matters or simply the manifestation of a single matter having an untangled dual state.

Coming back to the Earth, digital morphing got popular in the 90’s in multiple films and video clips. Techniques were based on outlining curvatures in the initial and target images, to match them and then apply some spatial transformation, like the Beier Neely algorithm. But the procedure to create a single clip was really tedious. These days object recognition algorithms in computer vision have let morphing software to advance in order to automate the procedure quite a lot. Specially for human faces. For instance, Dlib is a great toolkit for detecting facial landmarks in an image, among other things. The high-end smart-phones line operate all kind of face-based magic tricks for shooting pictures. Or popular apps like faceApp created awesome makeup effects and surprised everyone making Rijksmuseum’s paintings amusingly smile.

Antonio de la Gándara (1861-1917) - The cabaret's gentlement and four artists of the Chat Noir (1984).

Let’s morph it

The FaceMorph code performs automatically these steps:

  1. Facial landmarks recognition in both faces (Dlib).
  2. Triangular Delaunay segmentation.
  3. Affine transformation between the Delaunay triangles of both faces.
  4. Alpha blending on the paired triangles with a given transparency.

To create a caricature effect, the alpha blend must be set either to the maximum or minimum in order to make appear completely one of the faces and then the affine transformation must target the other face’s landmarks set.

A great choice for shaping funny caricatures is to pair faces with completely different facial dimensions between eyes, nose or mouth. Not only grins, but also disparate poses can generate crazy distortions too.

Donald Trump's caricature morphed with Hillary Clinton's facial expression.
Hillary Clinton's caricature morphed with Velazquez's facial expression.

In some cases however if the poses between the two faces are so distant, the morphing effect gets too much distorted or simply broken. That’s the case when the two faces are looking at opposite directions or they are located at very far away relative locations in the image. For the first issue an horizontal flip can make the trick and for the second one a fair enough head alignment towards the center can work fine.

Caricatures of Van Gogh and Paul Goughin morphed with each other's facial expression. Note the Paul Goughin's self-portrait was originally heading to the right side of the paining.

Now we rise the score with a creepy series of disturbing faces. Note that some of them are limit cases in which some facial parts get too distorted, but still worked pretty good.

Caricatures of Galileo Galilei morphed with Hillary Clinton's (left) and Velazquez's (right) facial expressions.
Velazquez's caricature morphed with Jim Carrey's facial expression.
Batman's (Christian Bale) caricature with the Joker's (Heath Ledger) facial expression.

That’s all folks. Keep morphing and have fun.