After a string of world-shaking events America’s spies failed to predict, most recently the turmoil sweeping the Arab world, a vast project is taking shape to improve forecasting. It involves thousands of volunteers and the wisdom of crowds.
It’s officially known as the Forecasting World Events Project and is sponsored by the Intelligence Advanced Research Activity (IARPA), a little-known agency run by a woman, Lisa Porter, who is occasionally described as America’s answer to the fictional Agent Q who designs cutting edge gadgets for James Bond. Much of IARPA’s work is classified, as is its budget. But the forecasting project is not classified. Invitations to participate are now on the Internet.
The idea is to raise five large competing teams of people of diverse backgrounds who will be asked to make predictions on fields that range from politics and global security to business and economics, public health, social and cultural change and science and technology. The project is expected to run for four years and stems from the recognition that expert forecasts are very often wrong.
One of the teams is being put together by University of Pennsylvania professor Philip Tetlock, whose ground-breaking 2005 book (Expert Political Judgment: How Good is It? How Can We Know?) analysed 27,450 predictions from a variety of experts and found they were no more accurate than random guesses or, as he put it, “a dart-throwing chimpanzee”.
“To test various hypotheses,” Tetlock said in an interview, “we want a large number on my team, 2,500 or so, which would make it almost ten times bigger than the number I analysed in my book.” There are no firm numbers yet on how big the other four teams will be. But Dan Gardner, the author of a just-published book that also highlights the shortcomings of expert predictions, believes the IARPA-sponsored project will be the biggest of its kind. It is expected to start in mid-2011.