At the time of writing this article, the world is dealing head-on with the global COVID-19 pandemic. Governments are scrambling to keep up with the situation using real-world data and information. Cyber adversaries—some backed by nation-states—are also weaponizing the illness to target governments, citizens, and their data.
Now is a critical time to assess how governments and defence ministries access threat intelligence for cyber and real-world crisis response, and how effective those tools and processes are.
Many threat intelligence tools and services are focused on providing clients with finished intelligence. These are meant to ease the efficiency problem that many industries, including government intelligence analysts, struggle with.
However, “raw data,” also called primary data or source data, is touted to have a number of advantages for government and defence use cases. What is raw data, and when is it preferred over processed data?
What Is Raw Data Vs Finished Intelligence?
Finished intelligence, or cooked data, is raw data that has undergone processing to gain context and become actionable. The collection, processing, and analysis of raw data are the threat intelligence lifecycle’s foundational steps.
In other words, raw data is unaltered from its original source. This could look like a network’s traffic data logs, dark web discussions, or even public social media posts.
Specialized software can identify and separate entities within a data set (parsing), and organize and display those entities by category to glean meaning and avoid redundancies (normalizing). Data discovery software can also index raw data so that it’s quickly and easily searchable and filtered for relevancy.
On the flip side, finished intelligence would look like a report summarizing the context interpreted from relevant raw data points and a suggested security response.
Finished threat intelligence products and services allow organizations to skip the raw data collection and analysis steps, which tend to be more time-consuming. Those steps are instead supported by automation and machine learning capabilities, and/or third-party analyst teams.
The main goal of finished intelligence products is to operationalize the process so organizations can respond faster to active threats and invest less time and resources in gathering and contextualizing large volumes of raw data. The result is a finished intelligence report that the client can immediately act on. While expensive, finished intelligence solutions can be ideal for private sector organizations seeking a “comprehensive” security solution.
Raw Data for National Security and Crisis Response
If finished threat intelligence solutions help organizations access actionable, processed data more efficiently, why would they use raw data feeds and tooling?
Government and defense ministries are a clear example of how efficient access to normalized, raw data has a number of advantages over processed, finished intelligence solutions. Namely:
- Government and defence tend to work with their own established tooling and processes. These existing processes don’t always complement or conform to third-party tools and services that deliver finished threat intelligence, even if they offer customization and integration options. Many large enterprise and public sector entities, such as fusion centres, have existing platforms requiring raw data integration.
- Government and defence are mission-driven. This means that they aren’t necessarily broad monitoring for threat alerts and acting on finished intelligence reports like the private sector. Rather, they’re actively pursuing raw data relevant to the mission as it unfolds. Missions have specific, evolving goals requiring raw data analysis rather than the bias of finished cyber intelligence.
- Many intelligence tools are focused specifically on producing finished cybersecurity threat intelligence. Government and defence also require data feeds aggregating relevant, non-cyber threat data from social networks and deep and dark websites. These sources are critical for informing national security responses to real-life threats like terrorism, military encroachments, global trafficking, and natural disasters.
Sophisticated raw data feeds and APIs can also address efficiency issues. The Echosec Systems Platform and API aggregates raw, normalized, indexed data from a variety of social media, deep, and dark web sources to its users. The API can be integrated seamlessly, delivering searchable raw data into existing platforms.
Machine learning models also classify the Platform’s raw data into threat categories, such as radicalism or data breaches, to streamline processing. These features enable government and defence analysts to relevant find raw data more efficiently.
As the coronavirus pandemic demonstrates, cyber threats and real-world crises—from the effects of climate change to population movement, globalization, and conflict—are becoming increasingly complex and interconnected. As these events intensify in the coming decades, government and defence must stay agile with the data sources and processes that they use to support their intelligence teams.
Advancements in raw data filtering, machine learning, and delivery are helping governments and defence bridge the gap between efficiency and mission-driven context. It’s this context that will ultimately drive effective national situational awareness and response as our world enters an era of unprecedented change and uncertainty.
Are you overlooking any data sources critical to your mission?
Book a demo to find out.