OSINT TOOLS AND TECHNIQUES

FOR THREAT INTELLIGENCE AND RISK MANAGEMENT

Introduction

The role of the modern security professional is becoming more and more complex, and it’s no surprise considering the influx of unexpected places where threats are beginning to surface. In order to gain the upper hand, your strategy must include a diverse means of gathering intelligence, both for a predictive and reactive approach. In an era where content is being created at an exponential rate - 90% of the world’s data was created in the last 2 years alone - the future of security must be intelligence-led.

New call-to-action

A major source of intelligence that cannot be overlooked is the vast amount of data being produced by consumers, hackers, newsmakers, and bloggers every single day. Globally, almost every person and organization is communicating across multiple platforms and networks, as well as handling personal and corporate needs virtually - such as shopping, travel planning, and data management. Finding like-minded communities and audiences online is the goal; however, wherever you have people congregating, especially if there is potential for monetary gain, the risk of malignant behaviour increases. Enter: Open source threat intelligence.

Read more: 
The Role of Intelligence-Led Security in the Modern Tech Stack
5 Reasons Why Every Organization Needs an OSINT Team

What is OSINT?

iStock-digital security team working overlay onlineOpen source intelligence, or OSINT, refers to the process of gathering information from public, legal data sources to serve a specific function. Some open sources might include social media, blogs, news, and the dark web. 

The concept of OSINT very basically works like this:

Public information exists data is gathered information is analyzed for intelligence

The purpose of seeking information from public data varies on the type of insights you wish to gather. Many industries and professionals look to open sources to uncover workplace security threats, protect executives, prevent loss, manage assets, gauge brand sentiment, and monitor conversations for creating marketing strategies. Intelligence professionals use certain types of OSINT for investigations, prosecution, evidence gathering, and events monitoring. 

Note: it's very important that your data provider is compliant with all privacy laws learn more here.

Raw Data vs. Finished Intelligence

Finished intelligence, or cooked data, is raw data that has undergone processing to gain context and become actionable. The collection, processing, and analysis of raw data are the threat intelligence lifecycle’s foundational steps.

OSINT tools data ingestion process

In other words, raw data is unaltered from its original source. This could look like a network’s traffic data logs, dark web discussions, or even public social media posts. 

Specialized software can identify and separate entities within a data set (parsing), and organize and display those entities by category to glean meaning and avoid redundancies (normalizing). Data discovery software can also index raw data so that it’s quickly and easily searchable and filtered for relevancy.

On the flip side, finished intelligence would look like a report summarizing the context interpreted from relevant raw data points and suggested security response.

Finished threat intelligence products and services allow organizations to skip the raw data collection and analysis steps, which tend to be more time-consuming. Those steps are instead supported by automation and machine learning capabilities, and/or third-party analyst teams. 

The main goal of finished intelligence products is to operationalize the process so organizations can respond faster to active threats and invest less time and resources in gathering and contextualizing large volumes of raw data. The result is a finished intelligence report that the client can immediately act on. While expensive, finished intelligence solutions can be ideal for private sector organizations seeking a “comprehensive” security solution.

CHECKLIST: CHOOSING THE RIGHT SOLUTION

Access to these data sources is often free, but the true value lies in what can be analyzed and extracted from the data.  Organizations using OSINT for threat intelligence require the ability to detect key information quickly and efficiently. They can do so by using a threat intelligence platform.

The vast amount of online data to sift through is overwhelming and with the complex ways today’s online threat actors are conducting themselves, the vulnerabilities to your organization continue to become more elusive. This data, when gathered and monitored effectively, can be extremely valuable for predicting, analyzing, and reviewing incidents at every stage of their occurrence. But where to begin? 

Read more:
What is Raw Data and Why National Security Depends on it
How to Build a Complete OSINT Strategy in 5 Steps
How Provider Compliance Protects Your Organization (and the Public)
How Echosec Systems Supports the Intelligence Cycle

Where to Look

Where you look for information depends on what you want to find. Performing a Google search is a simple form of OSINT, but when you are responsible for the safety and security of a particular person, place, or asset, you need to be casting a keen eye over multiple sources. Criminal behavior tends to be hidden, and it is unlikely a surface web search will take you there. 

At Echosec, we have access to a broad range of sources between the open web and social media all the way into the deep and dark web. For a complete list of sources, contact us.

Read More: 
Boost Your API Coverage: Diversifying Threat Intelligence Feeds
Not Just for Marketers: Influencer Data for Intelligence Analysts

Threats

The emergence of intelligence-led security is a direct result of the varied and growing range of threats that are being plotted, planned, discussed, and executed online. As our physical and digital realities are becoming more and more interlaced, individuals and organizations are creating more informational weaknesses and thereby more opportunities for an ever widening range of cyber attacks and other threats to occur. These include: 
  • Hacking
  • Information leaks
  • Cybercrime
  • Fraud 
  • Dark web activity
  • Shoplifting and theft
  • Active threats

Open source threat intelligence can be an invaluable addition to your protocol when handling internal processes such as:

  • PR and company reputation management
  • Workplace and facilities safety issues
  • Live events monitoring
  • Executive protection and risk management
  • Natural disasters and incident response

Read more:
Where Digital Meets Physical: Integrating Your Threat Intelligence Strategy
Cybersecurity to Passport Fraud: How OSINT Supports Airport Security
Cyber Security Breach: What Happens in the Fallout?

5 Current Cyber Attack Techniques and How to Stay Threat-Informed
OSINT Tools and Techniques for Business Risk 
Intelligence

OSINT for Corporate Security

iStock-corporate blur building people going to work executive glassGlobal enterprises are operating in the age of digital transformation. This has plenty of benefits for companies, helping improve customer experience, productivity, and resource management. But along with these benefits, wider technology adoption also means increasing opportunities for compromise.

This stands true for almost any industry with an online presence—including finance, retail, and transportation, which make up some of the world’s most cyber-targeted industries. Digital transformation also affects physical security and cyber-enabled threats as criminals adopt anonymized online communication channels. What do these risks look like?

Cyber Threats

  • Data breaches targeting corporate and customer information
  • Phishing, business email compromise (BEC), and other forms of impersonation
  • Malware and ransomware attacks
  • Credential stuffing
  • SIM swapping
  • Distributed denial of services (DDoS) attacks
  • Zero-day exploits

Cyber-Enabled Threats

  • Credit card fraud
  • Money laundering
  • Counterfeiting
  • Theft and gift card fraud
  • Workplace harassment
  • Insider threats

Physical Security Threats

  • VIP-targeted doxxing and harassment
  • Travel risk management
  • Event monitoring
  • Crises like terrorism and natural disasters

OSINT tools and techniques support corporate security teams in identifying and responding to these risks. Social media networks provide real-time updates from on-the-ground threats near executives and other physical assets like offices, employees, and corporate events. Paste sites, forums, and marketplaces across the deep and dark web often publish the earliest indicators of corporate data breaches and executive-targeted doxxing. And anonymized discussions on these covert sites help security teams identify fraud, insider threats, and cyber attack strategies directly from the source.

Combined with other risk management feeds and tools, OSINT provides corporate security with more comprehensive context and earlier risk indicators so they can respond faster and avoid blind spots. 

But many organizations face challenges in responding to risk quickly and effectively, especially as more enterprise teams—from marketing to IT and compliance—require OSINT. According to a 2021 report by Forrester Research, 42% of corporate decision-makers are currently improvising when it comes to risk management. Almost 70% claim that risk information is siloed across their departments and only 29% are confident in their risk management technologies. 

So what do corporate security teams need out of their OSINT solutions to address these gaps?

  • Wide data coverage. There are thousands of different online sources out there, from social media platforms to the deep and dark web, where relevant risk data is hiding. Many risk management tools focus only on one data source type—such as social media or the dark web—to help security teams find relevant risk information. A more ideal solution combines a variety of these sources within one platform so teams don’t have to juggle more tools than are necessary. This can just lead to information gaps and slower responses.
  • Simplicity/usability. Not everyone who needs access to online risk data has a technical background. OSINT solutions should be accessible to anyone in an organization without the click-heavy processes and complex interfaces that are typical of IT-based risk management software. Personnel should be able to easily and quickly separate the most pertinent data and view it in a digestible format.
  • Collaboration features. For risks where cross-department visibility is necessary, OSINT solutions should offer permission settings and collaboration features that allow teams to view each other’s activities or tackle a security threat together when there is overlap.
  • Integrations. Many global organizations already have a suite of risk management tools. OSINT solutions should be able to easily integrate with third-party solutions, whether they include a UI or funnel data directly into existing systems.


Speed-to-Information.
OSINT tools that prioritize usability, wide data coverage, and real-time access allow corporate security teams to access critical risk data faster. This gives organizations a much better chance of avoiding or mitigating threats from all angles.

Read More:
Open Source Intelligence for Retail Security
or download this Retail Case Study 
5 Situations Where OSINT Tools are Critical for Retail Security

Why Cyber Investigations are the New Focus for Law Enforcement

OSINT in the Public Sector

Analysts using OSINT toolsIn the face of national security threats, governments need to stay prepared and make prompt, informed decisions to protect assets and potentially save human life. Open-source intelligence has become valuable for driving these decisions. 

OSINT helps support a variety of pubic sector use cases, including:

1. Counter-terrorism and extremism. Foreign jihadist groups like the Islamic State and Al-Qaeda are no longer solely responsible for the threat of terrorism and extremism. Domestic extremist movements based on conspiracy theories, right-wing ideology, and discriminatory worldviews now also pose serious national security threats. Public online spaces are leveraged similarly for both extremist types, playing a huge role in spreading propaganda, recruitment, financing, and sometimes planning. This data helps governments understand how extremist groups operate so they can then predict public safety risks and protect citizens and assets from domestic and global terrorism.

2. Addressing mis- and disinformation. National security threats have expanded to include online influence campaigns, which can compromise democratic processes and lead to real-world security risks. Disinformation (which is engineered to deliberately deceive) and misinformation (false information that is not necessarily spread with malicious intent) is widely prevalent online. Monitoring online spaces is crucial for tracking disinformation campaigns so governments can mitigate their impact and keep the public safer and more informed.

3. Cybersecurity. Breaching government data is financially and politically lucrative for lone-wolf attackers, organized hacking groups, and nation-state actors. Sophisticated technologies are available to a greater diversity of adversaries than ever before. Persistent online threats include breaches and cyber espionage targeting classified data, network attacks disrupting critical infrastructure, and botnets enabling malware attacks and information warfare. Paste sites, discussion forums, and marketplaces on the deep and dark web often provide early indicators of breaches, malware, and attack techniques. Combining this open-source data with other cybersecurity feeds helps intelligence teams more confidently predict, mitigate, and investigate cyber compromise.

4. Transportation security. National transportation networks, including airports, seaports, and highways, make up a country’s critical infrastructure. When this infrastructure is compromised, governments and security teams need to stay prepared and alerted to prevent damage to assets, data, and human life. Online data plays a crucial role in providing the intelligence required for informed transportation security planning and incident response. For intelligence teams, social media networks and deep and dark web content can:

    • Provide the earliest alerts for location-based threats near airports, seaports, and other transportation hubs
    • Inform security teams about tactics used to bypass security systems or commit attacks, particularly at airports
    • Monitor for threats directly targeted at the security/public sector organizations themselves
    • Stay alerted to vulnerable data that could compromise a transportation network’s digital or physical security

TRANSPORTATION SECURITY WITH ECHOSEC SYSTEMS

5. Addressing national and global crises. When a national crisis occurs, governments must make timely, informed decisions to protect their data, assets, and citizens. As we’ve seen with the COVID-19 pandemic, adversaries co-opt real-world events in their strategies. Whether it’s a natural disaster, public health crisis, or terrorist attack, intelligence teams need to know how and where the crisis is occurring and how to allocate response resources. Online spaces are often the earliest sources of information to provide this context—for example, social media users often post public updates and images from the scene of a crisis. Aligning this data with other feeds can help provide a faster and more informed response.

Intelligence professionals require specialized software to collect this information and generate actionable intelligence. Commercial OSINT tools help intelligence teams gather open-source data more efficiently and in-line with mission requirements. Because intelligence teams often work with their own interfaces and tooling, they also require direct, raw data access and integrations that can be plugged into their existing systems.

According to the US Intelligence National Strategy (2019), the intelligence community is increasingly challenged by growing volumes of online data available for collection, processing, analysis, and triage. The western world is also facing a data analyst shortage coupled with a growing demand for military AI. As a result, data scientists in the public sector tend to handle more complex tasks, developing tooling and data sets to support lower-level analysts on intuitive platforms.

Intelligence teams are also challenged by a lack of access to some emerging online sources. For example, fringe networks (like alt-tech platforms, deep and dark web imageboards and paste sites, etc.) do not offer their own API or are unavailable through commercial API providers. To gather data from these sources, analysts are often required to create dummy accounts, make group requests, and navigate networks manually. This requires a significant amount of HUMINT resources that could be allocated to other areas of the intelligence cycle.

To address these challenges and satisfy intelligence requirements, OSINT solutions must:

Read More:
Connecting the Dots│ What Public Sector Analysts Need from OSINT Software
Threat Intelligence Tools and the Importance of Usability
5 Ways Governments  Can Apply Open Source Intelligence

OSINT Tools

Analyst using OSINT toolsThere are many OSINT tools on the market, both free and with costs associated. The truth is, not one intelligence tool is 100% effective as a standalone strategy. Rather, combining a selection of niche solutions to use in tandem is the best practice. Remember that the best OSINT tools will have a geographical element, giving a digital window to narrow down the data by specific locations. Refine your strategy and choose tools to develop a tech stack devoted to the specific needs of your organization. 

Social media and discussion forum monitoring: Echosec is an open source threat intelligence and data aggregation platform that helps companies extract key information and gain situational awareness from publicly available information sources. Security teams use Echosec for predictive intelligence and real time issues management, as well as brand monitoring and post-incident review.

Dark web and darknet intelligence: Beacon is a dark web discovery platform designed for threat intelligence. Beacon allows security teams to pull fully indexed data from deep and dark web sources such as Onion and Pastebin from their own browser (no Tor required). You can filter by the type of information you're looking for, like credit cards, drugs, email and other criteria.

Email hacks: Have I Been Pwned? is a free online resource to check if your email address has been put at risk due to a data breach.

Twitter monitoring: TweetDeck allows you to view multiple timelines in one user view. TweetDeck allows a user to create specific filters such as specific activity and geographical locations. 

Internet archives: Wayback Machine is an internet archive tool, like a library, of historical data. This tool allows the user to search the history of archived websites, metadata, text contents, and TV news captions.

Read more:
How to Choose an Intelligence Solution: In-House, Services, or Third-Party Platforms?
Why the Best OSINT Tools Use Geofencing Technology
How is Machine Learning Used in Cybersecurity?
Connecting the Dots │ What Public Sector Analysts Need from OSINT Software
Security Technology: Improving Risk Assessments for Service Providers
Broad Monitoring vs. Targeted Threats: Choose the Right OSINT Tools

Training

Feeling like you’re missing something? There are various organizations offering open source intelligence (OSINT) training and education. Organizations like Hetherington Group can assist with the best methods for investigative knowledge and skills for gathering critical threat intelligence for your organization.

 

Conclusion

Business is happening online, and today’s security strategy needs to be informed by the masses of public data being created every day. Gathering, filtering, and analyzing this information requires the advanced capabilities of specialized tools.

Both amateur and professional criminals are using sophisticated strategies and seemingly innocuous platforms to conduct illicit business. More and more media platforms are being infiltrated and used outside their intended purposes. Evolving threats require predictive and intelligence-led security strategies. Security teams must gather intelligence from every corner that they can. Open source threat intelligence software is essential for any enterprise using public data sources to inform their decision-making.

Not only can OSINT help protect against hidden intentional attacks such as information leaks, theft and fraud, but it also has the ability to gain real-time and location-based situational awareness to help protect people at work, at events, institutions or even the shopping mall. The right threat intelligence tools will give your security management team the upper hand.

Want to learn more about how the Echosec products can assist your open source data discovery strategy?
Reach out to our team for a demo

DEEP DIVE RESOURCES

 

 

SOCIAL MEDIA MONITORING

READ NOW

 

 

WHAT IS THREAT INTELLIGENCE?

READ NOW

 

 

DARK WEB

READ NOW

 

 

DATA BREACHES

READ NOW

 

 

OSINT TOOLS

READ NOW

 

 

DIGITAL RISK

READ NOW