Today, UH math professor Krešo Josić recognizes your face.  The University of Houston presents this series about the machines that make our civilization run, and the peoplewhose ingenuity created them.  

We are gifted at recognizing faces. We perceive faces in clouds; we perceive faces in mandrake roots. Like magnets faces attract our attention, and they do so from early infancy. An entire region of our brain seems devoted to telling our neighbor from our spouse, based on facial features. Imagine what would happen if we couldn't! Then you will see why it's so important to be good at telling faces apart. 

Mandragora, from Tacuinum Sanitatis (1474)

How is it possible for us to recognize familiar people from different angles, or even wearing a Halloween costume? We don't fully understand the process, but it does seem to be holistic. Our minds integrate all the features of a face into a representation that we process as a whole. Face recognition has many surprising features: Did you know that eyebrows are among the most important identifiers; or that it may be easier to recognize a caricature of a friend than an actual picture?

How do machines fare at this task? Face recognition software has improved enormously in the last 15 years. In some ways computers surpass humans in their ability to tell apart faces, and they're getting better at it every day. 

But let's look at one of the first face recognition algorithms developed by mathematicians Larry Sirovich and Michael Kirby at Brown University in the 1980s. They started by computing an average face from a set of pictures. 

25 faces

Then they used that same set of pictures to find the most common ways faces differ from their average. These differences were expressed using a set of images called eigenfaces. 

Eigenfaces

Sirovich and Kirby managed to approximate any face as a combination of a relatively small set of eigenfaces. They used linear algebra, the same branch of mathematics that lets Netflix suggest movies you may like.

The average of the 25 faces

Now neuroscientists find that our brain might be doing something similar when we recognize a friend. Suppose we encounter her on the street. Once our brain realizes that it's observing a face, cells in certain parts of the cortex seem to respond to deviations of our friend's face from the average. Certain brain cells might respond to her nose being shorter, while others indicate that her eyebrows are more peaked than those on an average face. 

We have an amazing ability to discern faces even when they're, say, composed of fruits or vegetables — as in the paintings of Arcimboldo. We effortlessly pull the details before us into an integrated representation. This representation is not only linked with the person we recognize, but it becomes that person in our mind. To us people are their visage: When you think of somebody, you first recall their face. Indeed we use faces to navigate our complex society. Just imagine a world of faceless beings! Our ability to smoothly recognize and memorize faces has evolved hand in hand with our ability to function as part of human society. And an essential step in making our smart machines part of our society will be teaching them to recognize us.

Vertumnus (Skoklosters Slott, Balsta, Sweden), and Summer (Louvre, Paris) by Arcimoboldo

I'm Krešo Josić, at the University of Houston, where we're interested in the way inventive minds work. 

(Theme music)

---

For an interesting discussion of the problem of face recognition in humans you can look at https://www.nytimes.com/2007/02/13/health/psychology/13face.html.

It is still debated whether a particular part or parts of the brain are primarily used to tell apart faces. A pointer to some references is available at https://en.wikipedia.org/wiki/Fusiform_face_area. A good review of our current (as of early 2008) understanding of face recognition in humans and how it compares to commonly used algorithms can be found in Tsao and Livingstone. “Mechanisms of face perception.” Annu Rev Neurosci (2008) vol. 31 pp. 411-37 http://dx.doi.org/10.1146/annurev.neuro.30.051606.094238.

Face recognition in humans has many counterintuitive features. You can read more about them here https://web.mit.edu/bcs/sinha/papers/20Results_2005.pdf.

There are many people at UH and in Houston working on different aspects of face recognition. I will just mention Prof. Kakadiaris at UH, but a search on Google will lead you to many others.

Additional information: 

It seems that the regions of the brain responsible for face recognition remain plastic throughout life. This could mean that we are always able to use these areas to efficiently add more faces to be processed by this circuitry. 

Prosopagnosia is the inability to recognize faces. Surprisingly, it appears that the ability to recognize objects is intact in people thus afflicted, but their ability to integrate the different features of a face into the percept of a face is impaired. On the other hand Tsao and Livingstone discuss the example of a patient who seems to be normal at recognizing faces, but has problems in recognizing objects. This suggests that a specialized and somewhat autonomous circuitry in our brain is devoted to face recognition. 

In 1994 the visage of the Virgin Mary appeared to Diane Duyser of Hollywood, FL on a grilled cheese sandwich. Mrs. Duyser was not alone in her belief, and 10 years later the sandwich fetched $28,000 when auctioned on eBay. 

There is a surprising tendency for humans to meet robots or artifical faces with revulsion if they become too realistic. This effect is called the uncanny valley https://en.wikipedia.org/wiki/Uncanny_valley.

Images from Wikipedia:

The Arcimboldo figures
https://en.wikipedia.org/wiki/Arcimboldo

Mandrake Image https://en.wikipedia.org/wiki/File:Mandragora_Tacuinum_Sanitatis.jpg