What if the government could know the future?
It’s trying. Armed with billions of tweets, Google (GOOG) searches, Facebook (FB) posts, and other publicly available social-media and online data, the Office of the Director of National Intelligence is sponsoring research projects involving 14 universities in the United States, Europe, and Israel with the goal of using advanced analytics to predict significant societal events.
“Our focus is to beat the news with greater accuracy and to do it faster by combining [various sets of] data, and we are seeing that it is possible,” said Jason Matheny, program manager of the Open Source Indicators program, which is housed within the Intelligence Advanced Research Projects Activity, or IARPA, the government’s intelligence research incubator. (Think DARPA, but for intelligence.)
This is an ambitious goal. Even nerd savant Nate Silver has yet to predict the likeliness of a popular revolt toppling a government, a deadly disease outbreak, a sudden currency collapse, or war. Still, the recent explosion in data analytics ushered in by the social-media era holds huge promise for making increasingly accurate predictions about the future, says Thomas Malone, director of the Center for Collective Intelligence at Massachusetts Institute of Technology and author of The Future of Work. But, he cautions, “it’s not magic. There are a lot of things about the future that are essentially unknowable.”
Matheny, too, understands the limitations of predictive analytics. That’s part of the point. At the end of the government’s three-year research program, he says, they hope to know with more certainty just what is predictable—that is, what categories of events are relatively more knowable.
It would also help the organization know what isn’t foreseeable at all. In other words, they are tackling Donald Rumsfeld’s infamous “unknown unknowns” problem. If you know what you can predict, then you can predict it; if you know what you can’t predict, you can make other plans.
