Today, let's talk about uncertainty and an 18th century Presbyterian minister. The University of Houston Mathematics Department presents this program about the machines that make our civilization run, and the people whose ingenuity created them.
You are taking a walk, and you see an animal dart between the bushes. You only catch a brief glimpse of something furry. Was it a cat or a raccoon? If it is light outside, you will most likely conclude that you've seen a cat. You may not be aware of it, but you have just performed a computation: You've combined unreliable data with previous experience to find the most likely among different possibilities.
The word "computation" invokes symbols, machines and algorithms. However, our minds constantly compute: Our actions and decisions are based on a combination of what our senses tell us and what we already know about the world. The fact that most of this information is uncertain does not seem to bother us at all.
How can we compute in the presence of uncertainty? It took many years to answer this question. The process started with the 18th century mathematician and minister Thomas Bayes. He was interested in how our beliefs about the world should evolve as we accumulate new, but uncertain evidence. Bayes suggested a solution, and the French mathematician Simone de Laplace developed it into a powerful theory.
Portrait of Thomas Bayes. [Wikipedia]
Increased computer power allowed statisticians to apply Bayes' method to a number of very hard problems. In the 1960s the statisticians Frederick Mosteller and David Wallace turned their attention to the Federalist papers, a series of 85 essays written in 1787 and 1788 by John Jay, Alexander Hamilton and James Madison. The authorship of 12 of the essays was disputed, but the differences in writing style between Hamilton and Madison could decide the issue. Strewn throughout the essays were hints that pointed towards one or the other author. For example Hamilton wrote "while" and Madison used "whilst." But such evidence was not certain: the text could have been edited by others before printing, for instance. Mosteller and Wallace used computers and Bayes' method to integrate many small bits of uncertain evidence. Their conclusion that James Madison was the author of all disputed essays has been accepted by historians.
Bayes' idea applies whenever uncertain information is combined. When you listen to your friend you do more than just hear her speak. The movement of her lips, her body language together with the words she speaks allows you to understand what she says. Remove all cues but the sound, and you will find the task more difficult. This happens when you listen to a song on the radio. Mick Jagger's promise that "I'll never be your beast of burden" can be heard as "I'll never leave your pizza burning," or a hundred other variants.
But computation and uncertainty seem to be contradictory. Isn't there always exactly one right answer in mathematics? The answer is no! Many times we can only say that one possibility is more probable than others. And Bayes' idea allows us to reach the best possible conclusions using uncertain information.
This is Krešo Josić at the University of Houston, where we are interested in the way inventive minds work.
See earlier episode discussing Bayesian statistics.
You don't need more than an elementary understanding of algebra to see what Bayes' Theorem is saying. Here is a nice explanation.
Here is an explanation in pictures, for which not even algebra is required.
Wikipedia has a nice entry on Thomas Bayes.
Here is a brief description of the disputed Federalist Papers.
For a fascinating account of the history of Bayes' theorem, and how it formed the basis of an approach that was used to crack encryption schemes used by Nazis, and is being used in medicine today, you can look at the book by Sharon Bretsch McGrayne at Amazon.
Krešimir Josić's Blog or follow kjosic on Twitter.
This episode was first aired on February 27, 2013