5 Types of PAI Software Every Command Center Needs

According to the IDC, 463 exabytes of online data will be generated by 2025. Without explaining what an exabyte is, this is a lot of data—and much of it is publicly available. 

In recent years, intelligence teams have come to rely on public online data to inform government decision-making and other security initiatives. Open-source data is highly valuable, often providing insights not available through classified sources. 

Publicly available information (PAI) will only become more valuable as its availability grows and adversaries continue to exploit public information spaces. But with the rate of data growth, the problem is finding and leveraging relevant information fast enough to support timely, effective decisions. 

Command centers rely on PAI software to help analysts gather, process, and analyze an overwhelming amount of public data. What is PAI software, and which capabilities does every command center need?

What is PAI Software?

The US Department of Defense defines publicly available information (PAI) as:

“Information that has been published or broadcast for public consumption, is available on request to the public, is accessible on-line or otherwise to the public, is available to the public by subscription or purchase, could be seen or heard by any casual observer, is made available at a meeting open to the public, or is obtained by visiting any place or attending any event that is open to the public.”

PAI is not the same as open-source intelligence (OSINT). While OSINT includes finished intelligence derived from publicly available sources, PAI is raw data and information that can become intelligence once processed and analyzed.

PAI includes offline sources like public meetings or documents, but most PAI valuable to command centers originates online. PAI software helps analysts collect, process, and analyze this overwhelming amount of data more efficiently.

According to Lieutenant Colonel David Lands, an intelligence officer for the Virginia Air National Guard, commercial software providers offer the most efficient means to search and consolidate vast amounts of public online data. In short, intelligence teams need commercial software to separate relevant data from noise and support global intelligence efforts.

RFPs like this highlight some common requirements for PAI software: 

• Tools that make social media, surface, deep, and dark web data indexed and searchable
• Support for multiple languages and media types, like text, audio, and video
• Features that expedite data analysis, like machine learning

The Rise of PAI

According to former Defense Intelligence Agency director Samuel Wilson, 90% of intelligence comes from open sources. The public sector has quickly realized the value of PAI as widespread digital transformation and social media use provide a wealth of valuable insights.

What makes PAI valuable?
Provides insights unavailable from classified intelligence sources like SIGINT	Doesn’t just focus on key targets—also shows broader trends in areas of interest	Improves speed-to-information for breaking events	Enables easy information sharing between allies

Combined with classified intelligence sources, PAI supports applications like geopolitical assessments, counterterrorism, and crisis response. Unclassified data has the added benefit of being easily shareable between allies, as involved parties don’t need to disclose private collection methodologies.

PAI Software for your command center

Command centers have different focus areas and software requirements. But as a general guideline, intelligence teams can benefit from including the following types of PAI software in their toolkit.

Social search

Some PAI software is designed to locate relevant social data online. Social data is associated with mainstream networks like Twitter, Linkedin, and YouTube. However, social activity is diversifying across the surface, deep and dark web.

Valuable social data is also widely available on sources that analysts may have never heard of, or have trouble accessing. This includes the deep web and dark web, which hosts forums, imageboards, marketplaces, and less-regulated social networks. Decentralized networks, paste sites, messaging apps, and news can also be considered social data sources for analysts requiring PAI.

Social search software helps intelligence teams by:

• Indexing PAI that isn’t searchable on free engines like Google.
• Making PAI from networks that are cumbersome or dangerous to navigate (like the dark web) easily accessible.
• Managing attribution for the analyst. Users search within the software and don’t have to navigate the original source where they could leave a vulnerable footprint.
• Consolidating dozens of social sources to save time and minimize information gaps.

Multiregional tools

Command centers need PAI solutions for global use cases—whether they’re focused on cybersecurity, responding to a public health crisis, or assessing an adversary’s activities. 

PAI software that provides multiregional support helps intelligence teams generate more accurate insights for these applications. It also helps users avoid information gaps when they’re unfamiliar with relevant information sources in a region of interest.

For example, sources like Naver, Douban, and Voz may be of interest to analysts covering the APAC region. Social sites like Dvach and VK are relevant for intelligence missions involving Russia. Regardless of the region, command centers must ensure their PAI software has the regional data coverage required for areas of interest.

Other features, like translation and multilingual searching, are also necessary for international missions using PAI.

Geographic Information System (GIS)

According to Esri, a geographic information system (GIS) “connects data to a map, integrating location data (where things are) with all types of descriptive information (what things are like there).” 

Integrating location data with PAI has immense value for analysts, helping them understand relationships between online activities and geographies. 

Understanding public sentiment in a specific region, investigating criminal activity, or monitoring military infrastructure in a politically unstable area are examples of situations where analysts can benefit from combining location data with PAI.

PAI software can optimize the value of geographic data by providing features like:

• Location-based searching. This allows users to search for data by location name, coordinate, or geofence.
• Data mapping. This visualizes where data has originated by plotting points on a map.
• Natural language processing. This can extract locations from content like social media posts, even if a location wasn’t tagged by the author.

Data visualization

Beyond data collection, PAI software can visualize data patterns and relationships. Data visualization tools can generate immediate insights for a search, like this analytics page created by Flashpoint Physical Security Intelligence (formerly Echosec).

Artificial intelligence and machine learning

An abundance of open-source data means that command centers are often overwhelmed, lacking resources to process and analyze data efficiently. According to Gartner, 75% of enterprises will shift from piloting to operationalizing AI by 2024, driving a five-fold increase in streaming data and analytics.

AI is necessary to cope with growing data volumes if command centers hope to generate timely insights at scale. Human expertise is irreplaceable for analysis, but AI techniques like natural language processing can offload straightforward but time-consuming intelligence tasks.

For example, intelligence tools powered by machine learning models can filter out noise, provide context, visualize patterns, generate written summaries, and anticipate potential outcomes.

Three tips for effective PAI Software

If you’re reading this blog, you might be on the hunt for new PAI tools for your command center. How do you know which solutions will best support your team, whether it’s a social search tool or data visualization software?

In an information environment overloaded with data, analysts need to generate insights quickly without sacrificing accuracy. To meet this need, command centers should prioritize tools that:

1. Are simple and easy to use. Analysts are more likely to use PAI software and have shorter learning curves if it’s user-friendly. Some commercial PAI tools are powerful but take months for users to hone. Look for PAI software that has a simple UI, an intuitive workflow, and avoids click-heavy processes.
2. Provide real-time information. PAI is valuable partly because it can provide updates faster than other information sources. However, some PAI tools may have slower latency times, displaying data long after a query was made. Vendors who offer real-time or near real-time data access should be prioritized.
3. Have adequate data coverage. Information gaps can misinform decision-makers, potentially harming assets and national security interests. Intelligence professionals should procure PAI software that covers a range of mainstream, niche, and regional sources relevant to their goals.

PAI is becoming more relevant for a variety of intelligence applications. By including social search, multiregional coverage, GIS, data visualization, and AI capabilities in their toolkit, command centers can ensure their team is equipped with the sources and functionality required to optimize PAI.