Reinventing CIA for the Big Data Era

The Central Intelligence Agency took a significant step into the digital age with the official launch of its Directorate for Digital Innovation (DDI), a bureau devoted to giving officers around the world better IT tools to do traditional cloak-and-dagger work.  

The CIA has a new Directorate for Digital Innovation. This is the first directorate the agency has added since 1963 and the biggest change to America’s key spy service since before the moon landing. The new office will look beyond the spy-craft of today to the very big question of how to turn the vast amounts of data that the agency collects into useful insight for analysts, agents, the agency, and the nation. The goal is to turn chatter and daily digital exhaust into a window into the future. 

The new directorate, the CIA's first in a half-century, is a milestone in the agency's quest to modernize its IT systems and further operationalize its cyber capabilities. CIA Director John Brennan announced plans for the directorate, which includes the spy agency's CIO shop, in March in a message to the CIA workforce, saying "we must place our activities and operations in the digital domain at the very center of all our mission endeavors."

CIA Deputy Director Andrew Hallman has been picked to lead the new department and he spoke in a recent interview  about his agency’s biggest change in decades. 

Hallman sees the new directorate solving three big problems for CIA. 

• The first is helping agents and operatives hone their hacking and sleuthing skills. “The way we help people use digital and cyber techniques, this will raise it to a new level,” he said. Part of that is developing intelligence about those places where the United States can’t or won’t put boots on the ground, or at least, as many boots as the US would like. 
• The second problem: improving the CIA’s data management, or what Hallman calls “governance.” That sounds like fixing the filing system but Hallman says that better data governance will help CIA answer questions about precisely what it’s doing and what it isn’t doing. 
• The third and greatest challenge Hallman is looking to solve through the directorate, taking the vast volumes of digital intelligence that CIA receives from around the world and transforming it into a moving, credible picture of the future. Intelligence, in this context, becomes almost a super power.

“We have the ability to do more sense-making to provide for analysts a real ability to forecast,” said Hallman, quickly adding that forecasting, arriving at a better understanding of multiple probabilities, is fundamentally different from prediction. “We are gaining the ability to anticipate the conditions of change to determine if they are anomalies or areas to focus on. This directorate will bring focus to that anticipatory intelligence capability,” he says. “It’s often in the aggregate that we get an impression.” That’s particularly true of social media, which provides of course not only intelligence for specific operations—targeting this or that ISIS leader in Syria—but also taking the temperature of an entire population. It’s sentiment analysis on a massive, population-wide scale. “If you have time, you can develop insight into patterns,” with social media data, he says.
What does that mean? In describing how the directorate will provide anticipatory intelligence, Hallman highlighted a 2011 program sponsored by the Intelligence Advanced Research Projects Agency, or IARPA, called Open Source Indicators, or OSI. The goal was to “develop methods for continuous, automated analysis of publicly available data in order to anticipate and/or detect significant societal events, such as political crises, humanitarian crises, mass violence, riots, mass migrations, disease outbreaks, economic instability, resource shortages, and responses to natural disaster,” according to the IARPA Web site. In other words, the agency wanted to take massive amounts of publically available data from news reports, social networks and every other available open information source and paint a picture of some event that might occur of national security significance, like a revolution or an assassination, perhaps.

The program achieved some notable success. In 2012, a team of Virginia Tech researchers affiliated with OSI effectively predicted two events on the basis of open source data from social networks and elsewhere. One was that Mexico’s election of Enrique Peña Nieto to the presidency would be met with protests. Sure enough, at 11:15 p.m. on July 1, the student group Yo Soy 132 began marching in protest against election results declaring Nieto the winner. The marches set off weeks of arrests and police actions. 

The Virginia Tech team also correctly forecast a series of protests around the impeachment of Paraguay’s President Fernando Lugo. Sure enough, on June 29, 2012, 2,500 people effectively closed the bridge linking to Paraguay Ciudad del Este to the Brazilian town of Foz de Iguaçu. The team’s methods likely would have predicted other flare-ups had they not made the specific decision to limit themselves to Central and South America.

When asked if that was the sort of thing Hallman wanted to be able to do everywhere, he answered “yes.” He called OSI, “an important feature in our resources and development…a cutting edge program.” To have real relevance, the directorate will need to be able to predict change through sentiment or other means before it’s obvious to everyone else, even the participants. 

Better anticipatory analyses in the hands of agents and analysts will require a lot of new software to allow agents to directly tap into the agency’s largest data sets and pull out insights directly relevant to them and their current operations. The directorate will create new application program interfaces, or APIs, toward that end and will also pioneer entirely new mathematical approaches to data analysis and extrapolation. 

Perhaps, for Hallman the metrics for success start with changing the way the agency uses technology and data. “We will be successful when we know that digital and cyber is central to all we do. There’s a fast and rapid increase in technological ability,” in the hands of both friends and adversaries. “We need to be able to keep pace.”
FWC: http://bit.ly/1hgbvQk
DefenseOne: http://bit.ly/1G0DkIs