The other day I was listening to an episode of the Freakonomics Radio podcast entitled, “How to Be Less Terrible at Predicting the Future.” ¹ As part of the episode, they interviewed Dr. Philip Tetlock, professor at the University of Pennsylvania, cross-appointed in Wharton and in the School of Arts and Sciences. Dr. Tetlock was and is interested in why “experts” continue to make wrong predictions and so he co-created The Good Judgement Project (GJP).²

The GJP was a sort of a long-term empirical contest to see if entrants could do a better job at predicting future events than different intelligent agencies (the tournament lasted four years). And it turned out that the top forecasters (called superforecasters) reportedly did “30% better than intelligence officers with access to actual classified information,” at predicting future geopolitical events.³

Dr. Tetlock was asked what the secret sauce was to making the most accurate possible probability estimates. That’s when my Mormon-ears perked up and my Mormon-lens started filtering. I translated the question (which I paraphrased above) to:

“What characteristics would a prophet need to posses in order to make accurate prophecies?” 

Here is a list of characteristics/strategies/trainings that helps make a forecaster a superforecaster. I’ll allow you, the reader, to make comparisons between superforecasters and our prophets and come to your own conclusions regarding the comparisons:

1. Uses teams
2. Are taught strategies on how to avoid biases.
3. Are taught strategies for improving accuracy.
4. Non-dogmatic
5. Less likely than the general population to believe in fate and destiny.
6. More likely to believe in chance.
7. Proud of their accomplishments but humble with their judgements.
8. Realize that they are always very close to forecasting something completely wrong.
9. Very careful.
10. They are not “chickens who hang around the maybe zone and never say anything more than minor shades of maybe.”
11. Take well-considered bets.
12. Tend to be more actively open minded.
13. Treat their beliefs not as sacred possessions to be guarded but rather as testable hypotheses to be discarded when the evidence mounts against them.
14. Try not to have too many ideological sacred cows.
15. Willing to move fairly quickly in response to changing circumstances.
16. They are pretty good with numbers (numerate).
17. Comfortable with the idea that they can quantify states of uncertainty along a scale from 0 to 1.0, or 0 to 100 percent.
18. More detailed (granular is the word Dr.Teltock uses) in their appraisals of uncertainty – using numbers instead of “maybe” or “good-chance.”
19. More likely to use what Danny Kahneman calls the outside view, rather than the inside view. Let me use Dr. Tetlock’s words to clarify:
    

    “So, if I asked you a question about whether a particular sub-Saharan dictator is likely to survive in power for another year, a regular forecaster might get to the job by looking up facts about that particular dictator in that particular country, whereas the superforecasters might be more likely to sit back and say, “Hmm, well, how likely are sub-Saharan dictators who’ve been in power x years likely to survive another year?” And the answer for that particular question tends to be very high. It’s in the area of 85, 95 percent, depending on the exact numbers at stake. And that means their initial judgment will be based on the base rate of similar occurrences in the world. They’ll start off with that and then they will gradually adjust in response to idiosyncratic inside-view circumstances. So, knowing nothing about the African dictator or the country even, let’s say I’ve never heard of this dictator, I’ve never heard of this country, and I just look at the base rate and I say, “hmm, looks like about 87 percent.” That would be my initial hunch estimate. Then the question is, “What do I do?” Well, then I start to learn something about the country and the dictator. And if I learn that the dictator in question is 91 years old and has advanced prostate cancer, I should adjust my probability. And if I learn that there are riots in the capital city and there are hints of military coups in the offing, I should again adjust my probability. But starting with the base-rate probability is a good way to at least ensure that you’re going to be in the plausibility ballpark initially.”

20. Not lazy
21. Curious
22. Certain level of socioeconomic status and flexibility (allows them time to research).
23. Above average in intelligence.
24. Believe one can attach numerical probabilities to specific historical events without using vague verbiage (ie: “might,” “could,” “may,” happen).

Concluding Questions for the Reader

Given the above twenty-four characteristics, do our LDS prophets (past or present) qualify as superforecasters? What superforecaster characteristics do our LDS prophets possess or lack? Leave a comment.  I am interested in your thoughts.

____________

NOTES

1 To listen to this episode of Freakonomics Radio and to read its transcript, click here.

2 Dr. Tetlock has written about the findings from the GJP in his books, Superforcasting: The Art and Science of Prediction (click here to purchase) and Expert Policitical Judgement: How Good is It? How Can We Know? (click here to purchase). I have not read any of his books. As I said, I only heard about all of this, this week. If you would like to visit the GJP’s website, click here. 

3 Spiegel, Alix. “So You Think You’re Smarter Than A CIA Agent”. NPR.org. Retrieved January 18, 2016. Click here to listen

Miguel Barker-Valdez

Miguel is a Guatemalan-American Mormon living in the Northwest with his family. He is one of the proprietors of the Rational Faiths blog.

All posts by Miguel Barker-Valdez