“Reality is that which, when you stop believing in it, doesn’t go away”.
How to Build a Universe That Doesn’t Fall Apart Two Days Later, Philip K. Dick
Last week’s cliff hanger flashed Bayes Theorem and today we are going to put it through its paces. I’m repeating the same equation here in words( | is the symbol for given, ∝ is the symbol for is proportional to):
Probability (hypothesis|data) ∝ Probability (data|hypothesis) * Probability (hypothesis)
Remember the point of the equation is to weigh a belief in light of some new data. This new degree of belief in the hypothesis is called the posterior probability, what we have as a result of the operations. The operations ask how likely is this data given our belief and just how probable is that belief anyway? The second form normalizes the numbers by dividing by the evidence so that the distributions that they represent sum to 1.
The posterior then becomes your prior degree of belief the next time new data confronts your hypothesis. Iterating the process, chaining the equation to itself this way, is one way of modeling the human reasoning process. An example pertinent to the concerns of this blog: I believe anthropomorphic climate change is really real.
The process I went through to arrive at this conclusion went something along these lines. First being open-minded I had no idea what to believe, nothing that would satisfy my most critical searches for evidence justifying a position one way or the other. I do know the earth is warming, a trend both sides mostly agree on. I understand how green house gas physics is a part of what has allowed the biosphere to flourish for billions of years and how changes in its composition are linked to changes in temperature throughout geological periods. These are some of the relevant prior understandings I am bringing to the question. I do not know what to make of the claim that human activity is having a significant effect on these gases. This is the state of maximum entropy in the lingo of Bayes. Then I study the data about the melting ice caps; say a dozen peer reviewed articles, handful of books, a couple of documentaries, and plenty of photographic evidence. The majority of the evidence is arguing that the rate of melting is accelerating because the contributions of industrial gases are a statistically significant factor. Now my honest sense of what is real finds veracity in the claim that climate change is related to human activity. Say I became 70% convinced; there is probably something to the claim that anthropomorphic climate change is really real. Now when I turn to the study of ocean acidification I bring my previously reasoned position with me. As the process of my study continued through all the types of data available and how the models built of the data are constructed and interpreted eventually the claim worked its way into that inner bucket of “this is real.”
Much money is spent and enormous efforts are applied to the public conversations around climate change to present the public with the impression that either side might be right. A concerted effort by those with much to lose has created the impression that the question is 50 / 50; maybe yes, maybe no. Big oil and their bought talking-heads in congress and on TV would like to say to our children’s generation – “I’ll flip you for it.” A more indefensible position is hard to imagine. The circumstances are complex enough to rule out any simple black and white conclusions – yes, human activity is causing all climate change or no, human activity has absolutely nothing to do with climate change. This is not a situation in which a binary choice applies as if flipping a coin. We will look at flipping a coin in just a moment to draw the contrast in as stark of terms as I am capable of.
If we are going to find any peace of mind we are going to need to learn to think straight and one of the indispensable skills that requires is fine tuning our B.S. detectors.
Now to debate my position is always welcome. Intellectual honesty and integrity lay down a simple rule: bring me data and / or an alternative hypothesis that will convince me otherwise. There is a place for poetics, rhetoric, spin and color. This is not that place. This is reasoning and it is being applied to life and death questions. As we are all in this together I think we must agree that reason is the only reliable guide we seem to have access to as owners of a finite understanding embedded in the universe we are reasoning about. Remember that bit about being able to measure also providing access to the only objectivity we can claim? It is the same type of thing here. You can assert without further evidence that a man in the clouds or a deceased uncle told you the claim was a lie, but I can hardly be slighted for dismissing you as not sufficiently serious given the stakes. You can bring out data but on this question the overwhelming majority in every relevant field is against you. Going against the objectivity of the majority is indeed your prerogative, after all, absolutes are off the table and how else will the paradigms change? Still, to assert your position is anything but one of the inhabitants of cranksville, that would simply be dishonest. What my mother would call a lie.
So far I have just pulled percentages out of the air in my examples. The actual process is hardly so arbitrary; in fact it is in the transparency of the reasoning mechanic that the great strength of Bayesian work shines. It is why science is using it in more and more of its modeling, why spy agencies have been using it for ages and those building models of the brain find it central to their work. If you want to join the fun and games I’d like to mention and thank Mr. Kruschke for his fine guide, Doing Bayesian Data Analysis.
What follows is an excerpt from a project around Bayesian thought. It is offered to convey some sense of the processes.
There are two main characteristics of probability distributions to keep in mind: the area under the curve always sums to 1 and the shapes of the curves shift to where the bulk of the probability is to be found. Using various shapes allows us to express our degrees of belief be they small, large or indifferent. This illustration uses the Beta distribution as a convenient way to express degrees of beliefs though there are many others and there is no mathematical requirement for the prior to be expressible as a function at all. As long as the curve’s area sums to one, any conceivable shape can be drawn point by point using a grid approach. Let’s return to our investigation of that strange animal in our distribution jungle, the prior. It combines with the data in the likelihood, the crucible of the equation where their interaction results in an updated belief. For the likelihood in what follows a binomial distribution is used to model a binary outcome. We will graphically explore what happens as a prior encounters data that comes in all shapes and sizes. Some data we encounter is expected while other data catches us by surprise.
A friend has given you a shiny half dollar for your birthday. He assures you that this is a very special coin and encourages you to flip it to see if it comes up heads or tails. You look it over carefully but see nothing amiss so you expect the chance of the coin coming up heads to be about 50 – 50. You give the coin a good flip and it lands tails. Three more flips all come up tails too. You notice a mischievous smirk on your friends face but throwing caution to the wind flip the coin a fourth time and now it lands heads up. Five more tosses all come up tails. “I’ll bet you $50 the next flip comes up tails” your friend offers. Taken aback you begin to ask yourself just exactly how much you believe that this is indeed a fair half dollar.
I will not keep you in suspense. The coin was purchased at a magic shop where the dealer assured your friend that it was specially manufactured with a bias for landing tail side up. Let’s see how a Bayesian model deals with this situation. A fair coin can be modeled as having a 0.5 bias, meaning it has an equal chance of coming up heads or tails. Because your friend is smart enough not to believe magicians, he tried the coin out in the store before the purchase. This being a magic store his prior belief about the fairness of the coin was completely uncommitted, far as he was concerned it might have any bias at all. In the graph below this uncommitted prior belief is modeled as a Beta(1,1) distribution, a straight line covering an area summing to 1 illustrating all outcomes are considered equally likely. The likelihood below the prior shows your friend flipped the coin ten times in the magic store and only once did it come up heads. The posterior in the lowermost panel mirrors the likelihood, the Beta(1,1) prior having no affect. In the bottom panel the bias for heads is shown as .1. The center of the distribution peak convinced your friend the coin had bias that would cause it to land heads up only one tenth of the time. The deal with the proprietor of the magic store was consummated and now here you are wondering about the innocence of the very same coin.
This is not the first coin you have ever seen, you’ve been around the block. You consider that of all the coins you have encountered they seemed to flip fair, maybe not perfectly unbiased but for the most part trustable enough to decide which football team should go first. You have a prior with most of the area around 0.5 but are willing to account for some variations. The Beta(2,2) distribution used below expresses this nicely. The first column of graphs show how your prior belief in a distribution centered about 0.5 would change to one closer to 0.2 if you also flipped the coin ten times and observed heads only once. Notice that the prior has had its influence; you are not willing to grant the bias is one tenth on just ten observations. So far in our story you have actually flipped the coin nine times, do you take the bet? Just how many tosses would it take to overcome the effect of your prior expectation so that you also arrive at the correct estimate of the bias centered at one tenth? The second column shows it would take about 180 throws. You could be playing with your new birthday present a long time.
What if your prior conviction of the fairness of the coin was even stronger? You reason that half dollars are minted by the U.S. government according to strict specifications. Every coin may not be exactly 100% unbiased but surely if there is a bit of bias it is small. A Beta(10,10) prior captures your considerations and the graphs below tell the story. Even with only one head showing up in ten tosses the resulting degree of belief in this half dollar’s bias against heads is only about 0.35. You still expect to see three or four heads in ten throws. Lucky for you, though you trust the government mints you long ago learned your friend can be sneaky. You decline the bet and go off to dinner together, paying dutch. This is a fairytale ending, a very good thing. How many more throws would it have taken to bring your posterior belief around to one tenth when starting with this stronger prior? A whopping 1,600 throws, you could still be flipping that half dollar next year when your friend came over to give you a deck of cards for your birthday…
I hope this illustrated how the data matters. Honest interpretations of it are possible because of the different understandings each of us develops over the course of our life experiences and studies. What will convince one person will not necessarily convince another and not just because they are refusing to reason with care. All this is captured in this simple example of the Bayesian explanation of reasoning but it is impossible to miss the larger implications; given sufficient evidence all people, regardless of their prior convictions, will tend towards the same inescapably probabilistic conclusions.
This sort of reasoning is a public affair. It is the Lingua Franca of social conversations that are involved with contingency planning, risk analysis and a whole host of other critical processes. The transparency of assigning probability allows us to evaluate each others positions. It is an adult form of conversation for adult issues that will directly affect the degree of suffering occurring in our world.