The 5 traits of ‘superforecasters’
From Michael Barrett, Analyst, Extreme Value:
The events in Paris last month have had me thinking a lot about an excellent new book I recently read: Superforecasting: The Art and Science of Prediction, by Phillip Tetlock.
When terrorists brazenly launched coordinated attacks across Paris on November 13, the U.S. intelligence community had to quickly assess what this meant for Americans both here and abroad.
As investors, we likewise collect the most current data available to us, and then use it to form an opinion of what the future holds for our investments.
In our quest to become great investors, our job is to see into the future better than others, and buy today where the profits will be tomorrow. Learning to think like the extraordinary forward thinkers that Tetlock calls “superforecasters” will help in that quest.
As I’ll show you today, the good news is that anybody can improve their forecasting (and investing) skills if they desire to do so…
Every hour of every day, the U.S. intelligence community, or “IC,” (approximately 20,000 intelligence analysts employed by 16 U.S. government agencies such as the CIA and FBI) collect massive amounts of information from around the world. Then, their primary job is to judge the implications this information may have on our national security.
Despite an operating budget that reportedly exceeds $50 billion, the intelligence community has a track record marked with colossal blunders. The decision to invade Iraq in 2003 – on the premise that Saddam Hussein possessed weapons of mass destruction that were never found – is perhaps its worst failure ever.
In the wake of that debacle, the intelligence community’s research arm (IARPA, or the Intelligence Advanced Research Projects Activity) sponsored a massive forecasting tournament. The purpose was to uncover the best methods for making the kinds of challenging forecasts that intelligence analysts face every day.
Five teams, led by some of the top researchers in the world, were created to compete in the tournament. Every day, teams were required to submit predictions to identical questions posed by IARPA about world affairs. One such question discussed at length in the book was this: Will either the French or Swiss inquiries find elevated levels of polonium in the remains of Yasser Arafat’s body? Arafat, a long-time chairman of the Palestine Liberation Organization (PLO) and enemy of Israel, died suddenly after becoming ill in 2004. In 2013, a team of Swiss experts concluded there was moderate support for the proposition that he was poisoned by highly-radioactive polonium.
Tetlock says almost 500 questions were posed over the study period, and that his team, known as the Good Judgment Project, or GJP, gathered more than 1 million individual forecasts.
After the tournament’s first year, GJP had beaten all the other teams by a wide margin. After two years, its outperformance was so good that IARPA reportedly dropped the other teams to focus on GJP’s methods and results.
Tetlock didn’t build his team of “superforecasters” by recruiting the world’s most renowned minds… He simply put the word out through blogs and professional networks. About 3,200 curious people answered his request, passed the initial tests, and joined the team.
From this pool of people – working for nothing more than a $250 Amazon gift card – Tetlock identified a few hundred that he deemed “superforecasters.” These ordinary folks didn’t have security clearances, the latest intel, or access to the vast research IC analysts did… yet they consistently out-forecasted their peers in the intelligence community.
What Tetlock discovered is that the competitive edge that superforecasters possess is the way they think through problems, not necessarily superior intelligence.
Superforecasters break problems down into components, distinguish knowns from unknowns, develop rich context, and scrutinize every assumption.
Bill Flack, one of the superforecasters mentioned in the book, constantly looks for other views he can synthesize with his own. When rendering a forecast, he asks himself questions like: “Are there holes in this? Should I be looking for something else? Would I be convinced by this if I were somebody else?”
Based on the work of his GJP team, Tetlock sees forecasting as a skill that can be developed, not one you either have or don’t.
Tetlock discovered the strongest predictor of someone’s ability to become a superforecaster wasn’t their intelligence, but rather the degree to which they were committed to updating their beliefs.
In other words, superforecasters commonly viewed their initial forecast as a starting point, a “hypothesis to be tested, not a treasure to be guarded.”
This is an important point for investors to keep in mind. Remember, when you’re buying a stock, whether it’s one recommended by a Stansberry Research product or an idea you’ve developed on your own, ultimately you’re testing a thesis.
For instance, let’s say your decision to buy Stock XYZ rests on the following analysis: “I think the growth story for XYZ is under-appreciated by investors at the current price, and that it has the potential to double my money over the next three years.”
A lot can and will happen over that projected three-year holding period for Stock XYZ. Amazon could muscle its way into XYZ’s market, reducing its growth potential. The business could suddenly begin under-performing management’s expectations due to rising costs. Or an economic downturn could reduce demand for XYZ’s most popular products.
My point is this: Once you’ve become a shareholder, your work has only just begun. You must remain vigilant about what’s happening with the company, its industry, and the broader market.
Also, learn from the world’s superforecasters and be prepared to change your mind when new information carrying significant “diagnostic value” alters your original thesis.
After years of research and work with “superforecasters,” Tetlock created a mosaic of their philosophic outlook, abilities, thinking styles, and methods. These are also important traits for those who want to be superior investors:
- Be cautious, for nothing is certain;
- Be humble, for reality is infinitely complex;
- Be intellectually curious and open-minded;
- Value diverse views and synthesize them into your own;
- Believe it’s possible to get better.
Remember, forecasts are judgments based on available information and should be updated in light of changing information. Be prepared to change your mind when the facts change.
P.S. For those who might be interested, GJP is running a 2016 geopolitical challenge anyone can participate in. You can learn more about it at their website, here.