Hunting Facial Pareidolias With Deep Learning
Usage rights of herein pictures have been carefully respected and, likewise, photographers’ authorship has been credited by linking each image to its source. Click on the images to disclose it.
Who has never played to discover known shapes while looking at the clouds? That moment when voilà, you recognize a form emerging from the sky and shout “Look, up there! a face!”. Pareidolia describes that happy accident when we marvelously interpret a visual, also auditory, pattern as a meaningful object.
Another source of illusory findings is aerial imaging, such as the famous Face on Mars or the Lunar Silhouettes. However, some of the most inspiring places to let your imagination fly are restrooms, probably on account of the spare relaxing time we usually enjoy while sitting in those quiet cubicles. So much so that the legend says that Albert Einstein used to seek inspiration there for long times. Personally, I find restrooms mind-stimulating, specially when walls are decorated with marvelous abstract textures.
For this perceptual phenomenon, our visual system has been trained over millennia under evolutionary pressure to recognize countless shapes. Particularly important are those which are vital for the sake of our survival, such as the silhouette of predators or colorful fruits. However, among all shapes, face recognition has an outstanding position for obvious reasons.
A particular characteristic of this neural event is that once we have recognized a particular object, we can’t avoid recognizing it anytime we see it again. We can’t unsee what we’ve seen. Moreover, recognition is evoked instantly in a click-time. Two well-known examples are the Cow and the Dalmatian illusions.
Visual memory —regardless the biological plausibility this term has— and how it is innerly managed by neural connections in our brain have also an important role in a more complex, high-level process: the imagination. In some cases, when the perception is extremely vivid, reality and imagination entangle into an hallucinating experience, in which shadows and fears can emerge.
An hallucination effect in images was showcased some time ago by DeepDream, a computer algorithm called “gradient based optimization” for Deep Neural Networks (DNNs), by which the figures previously learned by the net amazingly pop-up from the image structures.
DNNs —biologically inspired, but not a faithful mimic of the human brain nature— have the capacity of learning highly complex and abstract patterns. They result in a powerful tool for detecting objects, also faces, in a great variety of poses and not necessarily seen during the training stage. In machine learning this is call generalization.
So, why not to use deep learning to hunt facial pareidolias? Since such imaginary faces produce a strong and similar perceptual response —as evidenced by scientific studies like this or this other— then, deep learning detectors could be fooled in a similar way. This technical article delves more deeply into it.
Single Shot Scale-Invariant Face Detector (S3FD) is a remarkable neural model, which shows a superb generalization performance at finding faces with different sizes and view angles, even partially occluded. In this experiment, S3FD has been employed not to find real faces, but patterns which resemble faces. To that end, scores estimated by S3FD —which stand for the probability of being a face— have been considered strictly between 0.5 and 1.0.
The following facial pareidolias have been automatically discovered in around 25K pictures of the Little Unsplash Dataset. In their ambiguity, they witness a great captivating and emotional power. Curiously, while many false positives were encountered on the way, in the end it turned out that higher or lower scores were not particularly relevant to determine the real visual impact the pareidolia possesses. It worths also noting that some facial pareidolias are more perceptibly salient after being cropped, from its source image, in a separated frame. Others also required being rotated to stand up or being displayed at a proper viewing size.
In the “artworks-I’d-like-to-mention” section, we find the prolific artist and creator of uncountable pareidolias with common objects, Christoph Niemann. Also, the AI-driven dadaist, Vladimir Alexeev in this and other articles. The Derivative Works project, a machine learning experiment to create facial collages. The Driessens & Verstappen microscopic sandfaces. Or this other experiment in which a face-tracker looks for facial shapes within the Hello Little Fella! Flickr group.
Back to S3FD, let’s try a more specific experiment. Here, S3FD processed four pairs of the above imaginary ghost faces, the original clean shapes and the hand-outlined drawings. It turned out that no face was detected in any of the clean versions, and only two were correctly detected in the outlined ones. Besides these figures are totally different from the training set, it seems that S3FD is unable to internally complete highly abstract faces and therefore figure out things that they are simply not there.
Such completion evocation is a very human peculiarity, as shown by the famous Kanizsa illusion, formalized by Gestaltism, and later measured by psychophysics. In other words, apparently, this kind of supervised deep learning architectures are not ready for imagination, yet, in the complete sense of the word. In a near future, unsupervised predictive models, such as Contrastive Predictive Coding, large-scale transformers like Image-GPT, or introspective attention and memory networks, will have much to say on this regard.
By way of conclusion, the collection of pictures bellow comprises crops of clouds discovered by looking for a needle in a haystack over 1.96 million pictures of the Full Unsplash Dataset. Each of the ephemeral shapes is a golden nugget in a vast sea of pixels. They are evocative, intriguing, disturbing, and surely arouse disparate emotional memories to each of us. Among the faces, one can glimpse glorious warriors, mythological figures, screaming daemons and dragons, contemplating animals, or gloomy puppets. It unavoidably brings to my mind reminiscences of the Goyesque and grotesque black paintings, enigmatic Daliesque figures, dramatic Picassian expressions, or energetic and striking Saurian archetypes.
If you liked this post, you might also enjoy flickering through the Faces in Things Twitter account.
Thanks for reading and watching!
