Why Speculation About MH370 is Evidence of Innumeracy

Modern air travel is ridiculously safe. Aircraft are not designed using prayer, or crystals, or chi, or any other pre-scientific or anti-scientific “way of knowing” that is demonstrably far less effective than publicly testing ideas by some combination of systematic observation, controlled experiment and Bayesian inference.

Pilots are not trained by looking to the Bible or the Quran or the Guru Granth Sahib as a guide, but using principles that have been worked out by publicly testing ideas by some combination of systematic observation, controlled experiment and Bayesian inference (I belabour this list because if I simply said “science” people would misunderstand, or treat it as a magical incantation that means something other than the completely mundane discipline of publicly testing ideas by some combination systematic observation, controlled experiment and Bayesian inference, which is what leads to weird claims like “Science can’t prove everything”, as if science can prove anything.)

In any case, thanks to all that work by people “who do not teach their God will rouse them/just before the bolts work loose” major airline disasters are unbelievably rare, which is to say: extremely improbable.

That means that when a disaster does happen, the cause is almost certain to be some extremely improbable confluence of events, be it multiple failures of independent systems or some unexpected interaction of systems in combination (the Ariane V explosion was of the latter kind: all the individual sub-systems worked properly, but in combination they destroyed the rocket.)

When we speculate on the possible causes for an event, we are properly limited to things that are not vastly less probable than the most common known causes. The famous medical dictum, “When you hear hoofbeats, think horses, not zebras” applies. There are a variety of causes for the sound of hoofbeats, and the most probable ones will be the cause most of the time because horses, even in modern cities with modern police forces, are just not that rare. I don’t think a year has gone by in my adult life that I didn’t encounter a police horse in a downtown area somewhere.

The relatively high probability of the most common cause in such cases sharply limits the range of speculation, because there just aren’t that many things that are comparably probable.

In the case of air disasters, however, the most common causes are incredibly low probability events. There is a huge range of things that have comparably low probability, and that means the field for speculation is very nearly unbounded, so we can wander across it almost endlessly, never getting any wiser, never getting any closer to the truth.

Speculation in such cases adds nothing. It is not like the case where there are a small number of highly probable causes. In such cases we might be able to exhaustively examine the minutia of the evidence and distinguish between them. But that is only possible because they are so few.

In the case where the most common cause is wildly improbable, it is simply not possible to pluck one hypothesis from the vast array of more-or-less equally plausible ones and study it to the point where it can be significantly raised or lowered in plausibility. For one, it is the nature of vastly improbable events to be very sensitive to detailed assumptions, so the lack of knowledge that surrounds air disasters in their early stages leaves room for different speculators to come to vastly different conclusions based on tiny differences in how they fill in the huge gaps in available information.

As such, engaging in speculation as to what happened to MH370 as if that speculation will ever carry us one whit closer to understanding what happened is strong evidence of innumeracy. The people who are doing this simply do not understand the numerical realities of Bayesian inference in such situations.

This is not to say that such speculation can’t be entertaining, and if people want to entertain each other by making up stories around indistinguishably implausible hypotheses, I’m going to consider them somewhat heartless, cruel and inhumane–because this is after all the tragic disappearance of over 200 human beings–but I won’t call them innumerate.

Still, apart from the rather goulish entertainment value, we should all understand that this is a time for mourning, and silence, and careful study of the few data we have in the hope that physically searching–which is nothing but the testing of the ideas “MH370 is at location X/Y” using systematic observation–will lead us to evidence of what actually occurred. The thing we can be practically certain of is that speculation will not.

About TJ

Scientist, engineer, inventor, writer, poet, sailor, hiker, canoeist, father.
This entry was posted in bayes, epistemology, probability, psychology, religion. Bookmark the permalink.

8 Responses to Why Speculation About MH370 is Evidence of Innumeracy

  1. Tor Hylbom says:

    I really enjoy your posts. I don’t see a “Follow” button and any way to register as a subscriber, so I could be notified of new posts as you publish them.

  2. Gareth says:

    This is the post I needed to read about a week ago. You write lucidly and sanely about the problem.

    O, what is the source of “who do not teach their God will rouse them/just before the bolts work loose”?

    • TJ says:

      Thanks! The quote is from Kipling’s The Sons of Martha, which I misquoted from memory. The correct version is:

      They do not preach that their God will rouse them a little before the nuts work loose.
      They do not preach that His Pity allows them to drop their job when they damn-well choose.
      As in the thronged and the lighted ways, so in the dark and the desert they stand,
      Wary and watchful all their days that their brethren’s ways may be long in the land.

  3. Colin says:

    Hi TJ, You’re 100% right that we should use a Bayesian approach however this depends on your priors and how you interpret the events. Having lived in Singapore and Malaysia for last 15 years I clearly remember Silkair 185 and the NTSB conclusions on the flight so my prior for Pilot suicide is quite high, it’s also increased by the massive improvements in flight safety as prior for accidents reduces. Many subsequent events such as timing of the failure, increase the posterior for this. The final resting place (though still P<1) is also consistent with someone trying to make the plane hard to find rather than a random flight path of a plane after it missed landing in Langkawi (I haven't seen one analyst suggest what would happen to a plane if it overshot the programmed instruction to fly to Langkawi, would it fly around in circles or head off to the South Indian Ocean.) I'm sure officials have much more information than they are releasing and have even better posterior P to back up their currents statements.

  4. Anita says:

    Agree. A time for mourning and silence.