The other day I was listening to an episode of the Freakonomics Radio podcast entitled, “How to Be Less Terrible at Predicting the Future.” 1 As part of the episode, they interviewed Dr. Philip Tetlock, professor at the University of Pennsylvania, cross-appointed in Wharton and in the School of Arts and Sciences. Dr. Tetlock was and is interested in why “experts” continue to make wrong predictions and so he co-created The Good Judgement Project (GJP).2
The GJP was a sort of a long-term empirical contest to see if entrants could do a better job at predicting future events than different intelligent agencies (the tournament lasted four years). And it turned out that the top forecasters (called superforecasters) reportedly did “30% better than intelligence officers with access to actual classified information,” at predicting future geopolitical events.3
Dr. Tetlock was asked what the secret sauce was to making the most accurate possible probability estimates. That’s when my Mormon-ears perked up and my Mormon-lens started filtering. I translated the question (which I paraphrased above) to:
“What characteristics would a prophet need to posses in order to make accurate prophecies?”
Here is a list of characteristics/strategies/trainings that helps make a forecaster a superforecaster. I’ll allow you, the reader, to make comparisons between superforecasters and our prophets and come to your own conclusions regarding the comparisons:
- Uses teams
- Are taught strategies on how to avoid biases.
- Are taught strategies for improving accuracy.
- Less likely than the general population to believe in fate and destiny.
- More likely to believe in chance.
- Proud of their accomplishments but humble with their judgements.
- Realize that they are always very close to forecasting something completely wrong.
- Very careful.
- They are not “chickens who hang around the maybe zone and never say anything more than minor shades of maybe.”
- Take well-considered bets.
- Tend to be more actively open minded.
- Treat their beliefs not as sacred possessions to be guarded but rather as testable hypotheses to be discarded when the evidence mounts against them.
- Try not to have too many ideological sacred cows.
- Willing to move fairly quickly in response to changing circumstances.
- They are pretty good with numbers (numerate).
- Comfortable with the idea that they can quantify states of uncertainty along a scale from 0 to 1.0, or 0 to 100 percent.
- More detailed (granular is the word Dr.Teltock uses) in their appraisals of uncertainty – using numbers instead of “maybe” or “good-chance.”
- More likely to use what Danny Kahneman calls the outside view, rather than the inside view. Let me use Dr. Tetlock’s words to clarify:
“So, if I asked you a question about whether a particular sub-Saharan dictator is likely to survive in power for another year, a regular forecaster might get to the job by looking up facts about that particular dictator in that particular country, whereas the superforecasters might be more likely to sit back and say, “Hmm, well, how likely are sub-Saharan dictators who’ve been in power x years likely to survive another year?” And the answer for that particular question tends to be very high. It’s in the area of 85, 95 percent, depending on the exact numbers at stake. And that means their initial judgment will be based on the base rate of similar occurrences in the world. They’ll start off with that and then they will gradually adjust in response to idiosyncratic inside-view circumstances. So, knowing nothing about the African dictator or the country even, let’s say I’ve never heard of this dictator, I’ve never heard of this country, and I just look at the base rate and I say, “hmm, looks like about 87 percent.” That would be my initial hunch estimate. Then the question is, “What do I do?” Well, then I start to learn something about the country and the dictator. And if I learn that the dictator in question is 91 years old and has advanced prostate cancer, I should adjust my probability. And if I learn that there are riots in the capital city and there are hints of military coups in the offing, I should again adjust my probability. But starting with the base-rate probability is a good way to at least ensure that you’re going to be in the plausibility ballpark initially.”
- Not lazy
- Certain level of socioeconomic status and flexibility (allows them time to research).
- Above average in intelligence.
- Believe one can attach numerical probabilities to specific historical events without using vague verbiage (ie: “might,” “could,” “may,” happen).
Concluding Questions for the Reader
Given the above twenty-four characteristics, do our LDS prophets (past or present) qualify as superforecasters? What superforecaster characteristics do our LDS prophets possess or lack? Leave a comment. I am interested in your thoughts.
1 To listen to this episode of Freakonomics Radio and to read its transcript, click here.
2 Dr. Tetlock has written about the findings from the GJP in his books, Superforcasting: The Art and Science of Prediction (click here to purchase) and Expert Policitical Judgement: How Good is It? How Can We Know? (click here to purchase). I have not read any of his books. As I said, I only heard about all of this, this week. If you would like to visit the GJP’s website, click here.
3 Spiegel, Alix. “So You Think You’re Smarter Than A CIA Agent”. NPR.org. Retrieved January 18, 2016. Click here to listen
Your question at the end is a zinger.
The more church history I look into, the more and more you have to “excuse” the vast majority (almost all?) predictive comments by past church leaders as “not in their prophetic voice” (when Christ would come again – several times, the Civil war was punishment for violence against the church and would bring down the country, the civil rights movement was just a communist front and blacks would never receive the priesthood on earth, everyone urgently needs to get a years supply of food, the American Indians are from Lehi’s family – those just from one short podcast I listened to yesterday!)
It seems clear to me that if you follow the prophet because they are going to tell the future isn’t why you do it. You follow them if you feel the Lord tells you they are his mouthpiece and then YOU MUST follow that up and ask God if the words are his.
I also enjoyed this episode of Freakonomics, but I have to say that the question as to whether LDS prophets could be described as ‘superforecasters’ was the last thing from my mind.
I would argue that no – LDS prophets do not qualify as superforecasters.
Their views are largely based on presuppositions (AKA biases), they are very dogmatic, and they definitely do not attach numerical probabilities to their predictions.
In line with the previous comment – the trend we have seen among church leadership over the last 20+ years has been to avoid making definitive statements about the future.
Would we want LDS leaders to adopt some of these qualities of ‘superforecasters’? Sure, less dogmatism and more healthy pragmatic skepticism would be great.
But would I want to hear more prophesies to come from the pulpit at General Conference? With the current trends in leadership – definitely not.
You pretty much said what I was thinking.
What a coincidence that I just listened to this episode yesterday. I was wondering the same thing as you. Do LDS prophets in any way think or behave like superforecasters? In general, I don’t think so.
Traits they do have in line with superforecasters.
1. Uses teams
2. Are *taught* strategies on how to avoid biases.(not sure but maybe)
3. Are *taught* strategies for improving accuracy.(from their education this is true)
9. Very careful.
16. They are pretty good with numbers (numerate).(who knows, perhaps)
20. Not lazy
22. Certain level of socioeconomic status and flexibility (allows them time to research).
23. Above average in intelligence
So 8 out of 24 of the points. If we knew them personally we perhaps could expand this further but we can’t so we have at best a 33% chance the Top 15 being a superforecaster. Since they also require unanimity (group-think) rather than taking an average of the views of the top 15 I think the outlook is probably more grim than the 33% I gave earlier. They are managers. Probably loving managers with good intentions and a lifetime of wisdom but still managers.
Interesting! I always wondered what makes someone more qualified to talk to God. In religion, it usually seems to be gender.
Do you have any video of that? I’d love to find out more
The footnotes at the bottom of the blog post have the links to the Freakonomics podcast episode and other cool links. Thanks for stopping by and leaving a comment.
Seems like Joseph had more traits on this than most others, notably not being from a higher class.
The idea of “intuition” is related, and appears in medical, business, and artistic literature. What is that “feeling” a clinician has to check x on a patient that happens to be life-saving? What is that “feeling” that a jazz guitarist has that makes the improv phrase turn out perfectly? What is the “gut-feeling” business gurus have described in the book “blink”?
Scientists explain it away as experience. A lot of experience which lends itself to macro-pattern-making.
With that definition an octagenarian like president monson would leave Joseph in the dust.