Year: 2024

INSIGHTS, RESEARCH | September 4, 2024

About to Post a Job Opening? Think Again – You May Reveal Sensitive Information Primed for Cybersecurity Attacks

By Alejandro Hernandez

People are always on the move, changing their homes and their workspaces. With increasing frequency, they move from their current jobs to new positions, seeking new challenges, new people and places, to higher salaries.

Time and hard work bring experience and expertise, and these two qualities are what companies look for; they’re looking for skilled workers every single day, on multiple job search and recruiting platforms. However, these job postings might reveal sensitive information about the company that even the most seasoned Human Resources specialists don’t notice.

Job posting websites are a goldmine of information. Inherently, recruiters have to disclose certain data points, such as the technologies used by the company, so that candidates can assess whether they should apply. On the other hand, these data points could be used by malicious actors to profile a specific company and launch more sophisticated targeted attacks against the company and its employees.

To demonstrate this concept, I did research on tens of job postings from the following websites:

Surprisingly, more than 40% of job postings reveal relatively sensitive information, such as the following, which are just a sample of the information obtained from a variety of companies:

As you can see, a variety of information is disclosed inadvertently in these job postings:

  • Exact version of the software used in the backend or by end users
  • Programming languages, frameworks and libraries used
  • Cloud Service Providers where customer data resides
  • Intranet and collaborative software used within the company
  • Antivirus and endpoint security software in use
  • Industry-specific and third-party software used
  • Databases, storage and backup, and recovery platforms used
  • Business relationships with other companies
  • Security controls implemented in the company’s SDLC

Armed with this information, one can simply connect the data dots and infer things like:

  • Whether a company uses proprietary or open-source software, implying the use of other similar proprietary/open-source applications that could be targeted in an attack.
  • Whether a company performs Threat Modeling and follows a secure SDCL, providing an attacker with a vague idea of whether the in-house-developed applications are secure or not.
  • Whether a company has business relationship with other companies, enabling an attacker to target third-party companies in order to use them as pivot to attack the targeted company.

In summary, IOActive strongly encourages recruiters not to include sensitive information other than that required by the job position – in attempting to precisely target the exact candidate for a job, the level of detail you use could be costly.

INSIGHTS, RESEARCH | August 20, 2024

Get Strategic About Cyber Risk Management

By Gunter Ollmann

With global cybercrime damage costs exceeding $11 trillion last year and moving toward an estimated $20 trillion by 2026, robust cybersecurity risk management has never been more imperative.

The interconnected nature of modern technology means that, by default, even small vulnerabilities can lead to catastrophic losses. And it’s not just about finances. Unmitigated risk raises the specter of eroded customer confidence and tainted brand reputation. In this comprehensive guide, we’ll give enterprise defenders a holistic, methodical, checklist-style approach to cybersecurity risk management. We’ll focus on practical applications, best practices, and ready-to-implement strategies designed to mitigate risks and safeguard digital assets against ever-more numerous—and increasingly capable—threats and adversaries.

What is Cybersecurity Risk Management?

This subspecialty of enterprise risk management describes a systematic approach to identifying, analyzing, evaluating, and addressing cyber threats to an organization’s assets and operations. At its core, it involves a continuous cycle of risk assessment, risk decision-making, and the implementation of risk controls intended to minimize the negative impact of cyber incidents.

A proactive cyber risk mitigation approach helps organizations protect critical digital assets and bolster business continuity, legal compliance, and customer trust. By integrating risk management with the organization’s overall strategic planning, cybersecurity teams can prioritize resources efficiently and align their efforts with the business’s risk appetite and objectives.

Why Has Cyber Risk Management Become So Critical?

Getting control over cyber risk is quickly becoming a core requirement for businesses operating in today’s digital ubiquity. The proliferation of digital information and internet connectivity have paved the way for sophisticated cyber threats that can penetrate many of our most robust defenses. With the digital footprint of businesses expanding exponentially, the potential for data breaches, ransomware attacks, and other forms of cybercrime has escalated dramatically.

These incidents can result in devastating financial losses, legal repercussions, and irreparable damage to an organization’s reputation. Furthermore, as regulatory frameworks around data protection become more stringent, failure to comply can lead to significant penalties. Given these conditions, an aggressive and comprehensive approach to managing cybersecurity risks is crucial for safeguarding an organization’s assets, ensuring operational continuity, and maintaining trust with customers and stakeholders.

Effective Cyber Risk Management: A Framework-Based Approach

Adopting a structured, framework-based approach to cybersecurity risk management lets security teams corral the complexity of digital environments with a methodical, strategic mitigation methodology. For most enterprise applications, there’s no need to reinvent the wheel. There are a myriad of established frameworks that can be modified and customized for effective use in nearly any environment.

Perhaps the best known is the National Institute of Standards and Technology (NIST) Risk Management Framework (RMF), a companion to NIST’s well-tested and widely implemented Cybersecurity Framework (CSF). The NIST RMF offers a structured and systematic approach for integrating security, privacy, and risk management processes into an organization’s system development life cycle.

Such frameworks provide a comprehensive set of guidelines that help identify and assess cyber threats and facilitate the development of effective strategies to mitigate these risks. By standardizing cybersecurity practices, organizations can ensure a consistent and disciplined application of security measures across all departments and operations.

This coherence and uniformity are crucial for effectively addressing vulnerabilities and responding to incidents promptly. Equally important, frameworks incorporate best practices and benchmarks that help guide organizations toward achieving compliance with regulatory requirements, thus minimizing legal risks and enhancing the safeguarding of customer data. In essence, a framework-based approach offers a clear roadmap for managing cyber risk in a way that’s aligned with organizational strategic objectives and industry standards.

What follows is a checklist based on the 7-step RMF process. This is just a starting point. A framework to-do list like this can and should be tweaked to aid in reducing and managing specific cyber risks in your unique enterprise environment.

1. Preparation

In this initial phase, organizations focus on establishing the context and priorities for the Risk Management Framework process. This involves identifying critical assets, defining the boundaries, and codifying a risk management strategy that aligns with the organization’s objectives and resources. This is the foundation upon which a tailored approach to managing cybersecurity risk will ultimately be built throughout the system’s lifecycle.

  • Establish the context for risk management and create a risk management strategy.
  • Define roles and responsibilities across the organization.
  • Develop a taxonomy for categorizing information and information systems.
  • Determine the legal, regulatory, and contractual obligations.
  • Prepare an inventory of system elements, including software and hardware.

2. Systems Categorization

Expanding on the categorization step (above), this phase involves identifying the types of information processed, stored, and transmitted to determine potential impact as measured against the information security CIA triad (confidentiality, integrity, and availability). Organizations can assign appropriate security categories to their systems by leveraging a categorization standard such as the Federal Information Processing Standard (FIPS) 199, ensuring that the protective measures taken are tailored to the specific needs and risks associated with the information being handled. This step is crucial as it lays the groundwork for selecting suitable security controls in the later stages of the risk management process.

  • Identify the types of information processed, stored, and transmitted by the system.
  • Assess the potential impact of loss of Confidentiality, Integrity, and Availability (CIA) associated with each type.
  • Document findings in a formal security categorization statement.

3. Selecting Appropriate Security Controls

This critical step begins the safeguarding of information systems against potential threats and vulnerabilities in earnest. Based on the categorization of the information system, organizations select a baseline of security and privacy controls (NIST Special Publication 800-53 or some equivalent controls standard is a good starting point here), corresponding to the system’s impact level. This baseline acts as the jumping-off point for the security controls, which can be tailored to address the specific risks identified throughout the risk assessment process. Customization involves adding, removing, or modifying controls to ensure a robust defense tailored to the unique requirements and challenges of the organization.

  • Select an appropriate baseline of security controls (NIST SP 800-53 or equivalent).
  • Tailor the baseline controls to address specific organizational needs and identified risks.
  • Document the selected security controls in the system security plan.
  • Develop a strategy for continuously monitoring and maintaining the effectiveness of security controls.

4. Implementing the Selected Controls

Implementing security controls involves the physical and technical application of measures chosen during the previous selection phase. This step requires careful execution to ensure all controls are integrated effectively within the environment, aligning with its architecture and operational practices. Documenting the implementation details is crucial to provide a reference for future assessments and maintenance activities.

  • Implement the security controls as documented in Step 3.
  • Document the security controls and the responsible entities in place.
  • Test thoroughly to ensure compatibility and uninterrupted functionality.
  • Prepare for security assessment by documenting the implementation details.

5. Assessing Controls Performance

Assessing security controls involves evaluating effectiveness and adherence to the security requirements outlined in the overall security plan. This phase is critical for identifying any control deficiencies or weaknesses that could leave the information system vulnerable. Independent reviewers or auditors typically conduct assessments to ensure objectivity and a comprehensive analysis.

  • Develop and implement a plan to assess the security controls.
  • Perform security control assessments as per the plan.
  • Prepare a Security Assessment Report (SAR) detailing the effectiveness of the controls.
  • Determine if additional controls are needed and append the master security plan accordingly.

6. Authorizing the Risk Management Program

The authorization phase is a vital decision-making interval where one or more senior executives evaluate the security controls’ assessment results and decide whether the remaining risks to the information systems are acceptable to the organization. Upon acceptance, authorization is granted to operate the mitigation program for a specific time period, during which its compliance and security posture are continuously monitored. This authorization is formalized through the issuance of what is known as an Authorization to Operate (ATO) in some organizations, particularly in the public sector.

  • Compile the required authorization package, including the master plan, the SAR, and the so-called Plan of Action and Milestones (POA&M).
  • Assess the residual risk against the organizational risk tolerance.
  • Document the authorization decision in an Authorization Decision Document.

7. Monitoring and Measuring Against Performance Metrics

The monitoring phase ensures that all implemented security controls remain effective and compliant over time. Continuous surveillance, reporting, and analysis can promptly address any identified vulnerabilities or changes in the operational environment. This ongoing process supports the kind of flexible, adaptive security posture necessary for dealing with evolving threats while steadfastly maintaining the integrity and availability of the information system.

  • Implement the plan for ongoing monitoring of security controls.
  • Report the system’s security state to designated leaders in the organization.
  • Perform ongoing risk assessments and response actions, updating documentation as necessary.
  • Conduct reviews and updates regularly, in accordance with the organizational timelines, or as significant changes occur.

Conclusion: Formalizing Cyber Risk Mitigation

A solid risk management framework provides a comprehensive guide for enhancing the security and resilience of information systems through a structured process of identifying, implementing, and monitoring security controls.

Sticking to a framework checklist helps ensure a successful, systematic adoption. As noted throughout, engaging stakeholders from across the organization, including IT, security, operations, and compliance, is critical to ensuring a truly comprehensive risk management program. Additionally, periodic training and awareness for team members involved in various phases of the risk management project will contribute to the resilience and security of the organization’s digital assets.

Organizations can effectively safeguard their digital assets and mitigate unacceptable risks by following the outlined steps, tailoring the program to fit specific organizational needs, involving stakeholders, conducting regular training, and adapting to the evolving cybersecurity landscape. Ultimately, this kind of formal, structured cyber risk management fosters a culture of continuous improvement and vigilance in an enterprise, contributing to the overall security posture and the success of the organization.

ADVISORIES | August 7, 2024

IOActive Security Advisory | PLANET Networking – Vulnerabilities Identified

By Daniel Martinez

Affected Product

  • IGS-4215-16T2S

Firmware Version

  • 1.305b210528

Background

IOActive had the chance to access the IGS-4215-16T2S device. IOActive identified three vulnerabilities which need attention.

Timeline

  • 2022-09-29: IOActive discovers the vulnerabilities
  • 2023-03-29: IOActive informs Planet Technology about the identified vulnerabilities
  • 2023-12-13: Planet released a new firmware version (1.305b231218) informing IOActive that the vulnerabilities are fixed
  • 2024-01-09: IOActive notifies the vulnerability to INCIBE, Spanish CERT
  • 2024-02-16: IOActive confirm that the vulnerabilities were fixed after retesting them in the new firmware version
  • 2024-03-21: INCIBE shared the CVEs assigned with IOActive
  • 2024-08-07: IOActive advisory published
  • NOTE : While publishing this disclosure, IOActive had retested version FW-IGS-4215-16T2S_v1.305b231218.bix with hash 6e4ea892dc0d203c83ff02a2cba13e83. This version had the fixes. PLANET Technology published a firmware FW-IGS-4215-16T2S_v1.305b240227.bix with the hash abe64b8a62ebf339fb404fd85c0081b. They had informed that the findings have been fixed in this version. IOActive has not reviewed this firmware.
Read The Full Advisory
ADVISORIES | July 25, 2024

IOActive Security Advisory | Fortinet FortiGate – Cross-site Scripting in SSL VPN

By Jamie Riden

Affected Products

VersionAffected
FortiOS 7.47.4.0 through 7.4.3
FortiOS 7.27.2.0 through 7.2.7
FortiOS 7.07.0.0 through 7.0.13
FortiOS 6.46.4 all versions
FortiProxy 7.47.4.0 through 7.4.3
FortiProxy 7.27.2.0 through 7.2.9
FortiProxy 7.07.0.0 through 7.0.16


Background

Fortinet, Inc. (Fortinet) is a global leader of cybersecurity solutions and services that provides protection against cyber threats. It is a company that develops and sells security products and solutions, such as firewalls, endpoint security, intrusion prevention systems, web filtering, antivirus, sandbox, and VPN.

FortiGate is a network security device that provides protection against cyber threats. The device can perform various functions, such as, firewall, intrusion prevention system, web content filtering, antivirus, sandbox and VPN and is part of the Fortinet Security Fabric, which integrates different security products and services into a unified and automated platform.


Timeline

  • 2023-11-16: IOActive discovers the vulnerability
  • 2023-11-22: IOActive informs Fortinet about the identified vulnerability
  • 2024-01-12: Fortinet acknowledges the issue
  • 2024-04-26: CVE ID pre-reserved by Fortinet
  • 2024-07-10: Advisory published by Fortinet
  • 2024-07-25: IOActive advisory published
INSIGHTS, RESEARCH |

5G vs. Wi-Fi: A Comparative Analysis of Security and Throughput Performance

By Ethan Shackelford James Kulikowski & Vince Marcovecchio

Introduction

In this blog post we compare the security and throughput performance of 5G cellular to that of WiFi. This work is part of the research IOActive published in a recent whitepaper (https://bit.ly/ioa-report-wifi-5g), which was commissioned by Dell. We used a Dell Latitude 7340 laptop as an end-user wireless device, a Panda Wireless® PAU06 as a WiFi access point, and an Ettus Research™ Universal Software Radio Peripheral (USRP™) B210 as a 5G base station to simulate a typical standalone 5G configuration and three typical WiFi network configurations (home, public, and corporate). Testing was performed between January and February 2024 during which we simulated a number of different attacks against these networks and measured performance-based results for a number of different real-world environments.

Security Tests

We researched known 5G and WiFi attacks and grouped them according to five different goals: user tracking, sensitive data interception, user impersonation, network impersonation, and denial of service. These goals easily map to the classic Confidentiality, Integrity, Availability (CIA) security triad. We then reproduced the attacks against our controlled test networks to better understand their requirements, characteristics, and impact. The results of these investigations are summarized below:


We noted that, in general, the 5G protocol was designed from the ground up to provide several assurances that the WiFi protocol does not provide. Although mitigations are available to address some of the attacks, WiFi is still unable to match the level of assurance provided by 5G.

For example, 5G protects against user tracking by using concealed identifiers. Attacks to bypass these identifiers are easily detectable and require highly skilled and well-funded attackers. In contrast, WiFi does not attempt to protect against user tracking, since MAC addresses are transmitted in plaintext with every packet. Although modern devices have tried to mitigate this risk by introducing MAC address randomization, passive user tracking is still easy to accomplish due to shortcomings in MAC address randomization and probe request analysis.

IOActive also noted that the use of layered security protocols mitigated most sensitive data interception and network impersonation attacks. Although the majority of users do not use a VPN when connecting to the Internet, most websites use TLS, which, when combined with HSTS and browser preload lists, effectively prevents an attacker from intercepting most of the sensitive data a user might access online. However, even multiple layered security protocols cannot protect against vulnerabilities in the underlying radio protocol. For example, a WiFi deauthentication attack would not be affected in any way by the use of TLS or a VPN.

Performance Tests

We conducted performance tests by measuring throughput and latency from a wireless device in a variety of environments, ranging from urban settings with high spectrum noise and many physical obstacles, to rural areas where measurements more closely reflected the attributes of the underlying radio protocol.

Of particular note, we found that a wireless device could maintain a connection to a 5G wireless base over a significant distance, even with substantial interference from buildings and other structures in an urban environment.

In a rural environment, our WiFi testing showed an exponential decay with distance, as was expected, and it was not possible to maintain a connection over the same range as with 5G. We did, however, note significantly higher speeds from WiFi connections at close proximity:


Surprisingly, we did not see significant changes in latency or error rates during our testing.

Conclusions

The following network security spectrum summarizes our findings:


This spectrum provides a high-level overview of network types, from less secure to more secure, based on the characteristics we observed and documented in our whitepaper. The use of layered security mechanisms moves any network towards the more secure end of the spectrum.

Overall, we found that a typical standalone 5G network is more resilient against attacks than a typical WiFi network and that 5G provided a more reliable connection than WiFi, even over significant distances; however, WiFi provided much higher speeds when the wireless device was in close proximity to the wireless access point.

AUTHORS:
– Ethan Shackelford, IOActive Associate Principal Security Consultant
– James Kulikowski, IOActive Senior Security Consultant
– Vince Marcovecchio, IOActive Senior Security Consultant

INSIGHTS, RESEARCH | July 23, 2024

WiFi and 5G: Security and Performance Characteristics Whitepaper

By Ethan Shackelford James Kulikowski & Vince Marcovecchio

IOActive compared the security and performance of the WiFi and 5G wireless protocols by simulating several different network types and reproducing attacks from current academic research in a Dell-commissioned study. In total, 536 hours of testing was performed between January and February 2024 comparing each technologies’ susceptibility to five categories of attack: user tracking, sensitive data interception, user impersonation, network impersonation, and denial of service.

IOActive concluded that a typical standalone 5G network is more resilient against the five categories of attack than a typical WiFi network. Attacks against a 5G network generally had higher skill, cost, and effort requirements than equivalent attacks against a WiFi network.

Our performance comparison was based on measuring throughput and latency in several different urban and rural settings. We found that although WiFi supported significantly higher speeds than 5G at close proximity, 5G provided a more reliable connection over greater distances.

AUTHORS:
– Ethan Shackelford, IOActive Associate Principal Security Consultant
– James Kulikowski, IOActive Senior Security Consultant
– Vince Marcovecchio, IOActive Senior Security Consultant

ADVISORIES | June 21, 2024

IOActive Security Advisory | MásMóvil Comtrend Router –  Multiple Vulnerabilities

By Gabriel Gonzalez

Affected Products

  1. MásMóvil Comtrend Router – Version: ES_WLD71-T1_v2.0.201820
    1. HW Version: GRG-4280us
    1. FW Version: QR51S404
    1. SW Version: MMV-C04_R10

Timeline

  • 2023-08-24: IOActive discovers vulnerability
  • 2023-09-12: IOActive begins vulnerability disclosure with affected parties
  • 2024-06-10: The corresponding CNA released the CVEs to public domain.
  • 2024-06-21: IOActive advisory published
INSIGHTS | June 18, 2024

Recent and Upcoming Security Trends in Cloud Low-Level Hardware Devices: A survey

By Sean Rivera

The rapid evolution of cloud infrastructures has introduced complex security challenges, particularly concerning all of the processing devices and peripheral components that underpin modern data centers.

Recognizing the critical need for robust and consistent cloud security standards, technology firms, developers, and cybersecurity experts established the Open Compute Project Security Appraisal Framework and Enablement (OCP S.A.F.E.) Program.

At the 2024 OCP Regional Summit in Lisbon, I was joined by my colleague Alfredo Pironti, Director of Services at IOActive, to present a deep dive into the security of cloud infrastructures, the threats facing the crucial hardware that supports them, and how organizations can prevent being compromised by adopting new threat modeling techniques and security frameworks.

IOActive has monitored the state and health of hardware security for decades. We are now observing the changes in cybercriminal tactics, threats, and vulnerabilities that could compromise key components in digital supply chains and services.

When attackers target the hardware level, they can potentially exploit the entire stack. Once granted access to the hardware foundation, cybercriminals could potentially compromise physical infrastructure, data storage, applications, developer environments, code bases, and entire systems.

If vulnerable hardware is utilized in cloud services, this could even pose threats to national security as so many CSPs are now the backbone of critical infrastructure.

Hardware and computational components have evolved to meet the needs of increasingly complex cloud infrastructures and services. However, each new, enhanced capability may also create a new avenue for attack.

Take NVMe-based SSD disks and SR-IOV-enabled cards, for example. As we discussed during our presentation, historically, board problems, design flaws, or some implementation errors posed the most risk. Now, logical access bugs, data theft, arbitrary and remote code execution vulnerabilities, side-channel attacks, denial-of-service, and supply chain attacks must also be addressed.

IOActive has uncovered a wide range of risks to today’s cloud infrastructure through hands-on experience. Many hardware-based vulnerabilities stem from incorrect implementation, such as integer flaws, out-of-bounds memory issues, and race conditions.

During testing, we observed various security problems caused by component design and operational processes. A critical insight gleaned from our research is that 25% of vulnerabilities found were introduced in the design stage, showing a need for testing services early in the process.

In our presentation, we proposed an archetypal threat model that addresses the disconnect between developers, hardware manufacturers, and service providers regarding security. A core component of our model explores the divergence between the threats that cloud service providers face, and those faced by cloud hardware providers.

As addressed by the OCP S.A.F.E. framework, achieving robust security standards throughout the entire digital supply chain can assist hardware suppliers and service providers alike in tackling today’s cybersecurity challenges.

You can find a recording of our presentation here to share our knowledge and insights on cloud security and how frameworks, including OCP S.A.F.E., benefit organizations today.

– IOActive Senior Security Consultant and Researcher, Sean Rivera

INSIGHTS, RESEARCH | May 30, 2024

The Security Imperative in Artificial Intelligence

By Gunter Ollmann

Artificial Intelligence (AI) is transforming industries and everyday life, driving innovations once relegated to the realm of science fiction into modern reality. As AI technologies grow more integral to complex systems like autonomous vehicles, healthcare diagnostics, and automated financial trading platforms, the imperative for robust security measures increases exponentially.

Securing AI is not only about safeguarding data but also about ensuring the core systems — in particular, the trained models that really put the “intelligence” in AI — function as intended without malicious interference. Historical lessons from earlier technologies offer some guidance and can be used to inform today’s strategies for securing AI systems. Here, we’ll explore the evolution, current state, and future direction of AI security, with a focus on why it’s essential to learn from the past, secure the present, and plan for a resilient future.

AI: The Newest Crown Jewel

Security in the context of AI is paramount precisely because AI systems increasingly handle sensitive data, make important, autonomous decisions, and operate with limited supervision in critical environments where safety and confidentiality are key. As AI technologies burrow further into sectors like healthcare, finance, and national security, the potential for misuse or harmful consequences due to security shortcomings rises to concerning levels. Several factors drive the criticality of AI security:

  • Data Sensitivity: AI systems process and learn from large volumes of data, including personally identifiable information, proprietary business information, and other sensitive data types. Ensuring the security of enterprise training data as it passes to and through AI models is crucial to maintaining privacy, regulatory compliance, and the integrity of intellectual property.

  • System Integrity: The integrity of AI systems themselves must be well defended in order to prevent malicious alterations or tampering that could lead to bogus outputs and incorrect decisions. In autonomous vehicles or medical diagnosis systems, for example, instructions issued by compromised AI platforms could have life-threatening consequences.

  • Operational Reliability: AI is increasingly finding its way into critical infrastructure and essential services. Therefore, ensuring these systems are secure from attacks is vital for maintaining their reliability and functionality in critical operations.

  • Matters of Trust: For AI to be widely adopted, users and stakeholders must trust that the systems are secure and will function as intended without causing unintended harm. Security breaches or failures can undermine public confidence and hinder the broader adoption of emerging AI technologies over the long haul.

  • Adversarial Activity: AI systems are uniquely susceptible to certain attacks, whereby slight manipulations in inputs — sometimes called prompt hacking — can deceive an AI system into making incorrect decisions or spewing malicious output. Understanding the capabilities of malicious actors and building robust defenses against such prompt-based attacks is crucial for the secure deployment of AI technologies.

In short, security in AI isn’t just about protecting data. It’s also about ensuring safe, reliable, and ethical use of AI technologies across all applications. These inexorably nested requirements continue to drive research and ongoing development of advanced security measures tailored to the unique challenges posed by AI.

Looking Back: Historical Security Pitfalls

We don’t have to turn the clock back very far to witness new, vigorously hyped technology solutions wreaking havoc on the global cybersecurity risk register. Consider the peer-to-peer recordkeeping database mechanism known as blockchain.  When blockchain exploded into the zeitgeist circa 2008 — alongside the equally disruptive concept of cryptocurrency — its introduction brought great excitement thanks to its potential for both decentralization of data management and the promise of enhanced data security. In short order, however, events such as the DAO hack —an exploitation of smart contract vulnerabilities that led to substantial, if temporary, financial losses — demonstrated the risk of adopting new technologies without diligent security vetting.

As a teaching moment, the DAO incident highlights several issues: the complex interplay of software immutability and coding mistakes; and the disastrous consequences of security oversights in decentralized systems. The case study teaches us that with every innovative leap, a thorough understanding of the new security landscape is crucial, especially as we integrate similar technologies into AI-enabled systems.

Historical analysis of other emerging technology failures over the years reveals other common themes, such as overreliance on untested technologies, misjudgment of the security landscape, and underestimation of cyber threats. These pitfalls are exacerbated by hype-cycle-powered rapid adoption that often outstrips current security capacity and capabilities. For AI, these themes underscore the need for a security-first approach in development phases, continuous vulnerability assessments, and the integration of robust security frameworks from the outset.

Current State of AI Security

With AI solutions now pervasive, each use case introduces unique security challenges. Be it predictive analytics in finance, real-time decision-making systems in manufacturing systems, or something else entirely,  each application requires a tailored security approach that takes into account the specific data types and operational environments involved. It’s a complex landscape where rapid technological advancements run headlong into evolving security concerns. Key features of this challenging  infosec environment include:

  • Advanced Threats: AI systems face a range of sophisticated threats, including data poisoning, which can skew an AI’s learning and reinforcement processes, leading to flawed outputs; model theft, in which proprietary intellectual property is exposed; and other adversarial actions that can manipulate AI perceptions and decisions in unexpected and harmful ways. These threats are unique to AI and demand specialized security responses that go beyond traditional cybersecurity controls.

  • Regulatory and Compliance Issues: With statutes such as GDPR in Europe, CCPA in the U.S., and similar data security and privacy mandates worldwide, technology purveyors and end users alike are under increased pressure to prioritize safe data handling and processing. On top of existing privacy rules, the Biden administration in the U.S. issued a comprehensive executive order last October establishing new standards for AI safety and security. In Europe, meanwhile, the EU’s newly adopted Artificial Intelligence Act provides granular guidelines for dealing with AI-related risk. This spate of new rules can often clash with AI-enabled applications that demand more and more access to data without much regard for its origin or sensitivity.

  • Integration Challenges: As AI becomes more integrated into critical systems across a wide swath of vertical industries, ensuring security coherence across different platforms and blended technologies remains a significant challenge. Rapid adoption and integration expose modern AI systems to traditional threats and legacy network vulnerabilities, compounding the risk landscape.

  • Explainability: As adoption grows, the matter of AI explainability  — or the ability to understand and interpret the decisions made by AI systems — becomes increasingly important. This concept is crucial in building trust, particularly in sensitive fields like healthcare where decisions can have profound impacts on human lives.Consider an AI system used to diagnose disease from medical imaging. If such a system identifies potential tumors in a scan, clinicians and patients must be able to understand the basis of these conclusions to trust in their reliability and accuracy. Without clear explanations, hesitation to accept the AI’s recommendations ensues, leading to delays in treatment or disregard of useful AI-driven insights. Explainability not only enhances trust, it also ensures AI tools can be effectively integrated into clinical workflows, providing clear guidance that healthcare professionals can evaluate alongside their own expertise.

Addressing such risks requires a deep understanding of AI operations and the development of specialized security techniques such as differential privacy, federated learning, and robust adversarial training methods. The good news here: In response to AI’s risk profile, the field of AI security research and development is on a steady growth trajectory. Over the past 18 months the industry has witnessed  increased investment aimed at developing new methods to secure AI systems, such as encryption of AI models, robustness testing, and intrusion detection tailored to AI-specific operations.

At the same time, there’s also rising awareness of AI security needs beyond the boundaries of cybersecurity organizations and infosec teams. That’s led to better education and training for application developers and users, for example, on the potential risks and best practices for securing A-powered systems.

Overall,  enterprises at large have made substantial progress in identifying and addressing AI-specific risk, but significant challenges remain, requiring ongoing vigilance, innovation, and adaptation in AI defensive strategies.

Data Classification and AI Security

One area getting a fair bit of attention in the context of safeguarding AI-capable environments is effective data classification. The ability to earmark data (public, proprietary, confidential, etc.) is essential for good AI security practice. Data classification ensures that sensitive information is handled appropriately within AI systems. Proper classification aids in compliance with regulations and prevents sensitive data from being used — intentionally or unintentionally — in training datasets that can be targets for attack and compromise.

The inadvertent inclusion of personally identifiable information (PII) in model training data, for example, is a hallmark of poor data management in an AI environment. A breach in such systems not only compromises privacy but exposes organizations to profound legal and reputational damage as well. Organizations in the business of adopting AI to further their business strategies must be ever aware of the need for stringent data management protocols and advanced data anonymization techniques before data enters the AI processing pipeline.

The Future of AI Security: Navigating New Horizons

As AI continues to evolve and tunnel its way further into every facet of human existence, securing these systems from potential threats, both current and future, becomes increasingly critical. Peering into AI’s future, it’s clear that any promising new developments in AI capabilities must be accompanied by robust strategies to safeguard systems and data against the sophisticated threats of tomorrow.

The future of AI security will depend heavily on our ability to anticipate potential security issues and tackle them proactively before they escalate. Here are some ways security practitioners can prevent future AI-related security shortcomings:

  • Continuous Learning and Adaptation: AI systems can be designed to learn from past attacks and adapt to prevent similar vulnerabilities in the future. This involves using machine learning algorithms that evolve continuously, enhancing their detection capabilities over time.

  • Enhanced Data Privacy Techniques: As data is the lifeblood of AI, employing advanced and emerging data privacy technologies such as differential privacy and homomorphic encryption will ensure that data can be used for training without exposing sensitive information.

  • Robust Security Protocols: Establishing rigorous security standards and protocols from the initial phases of AI development will be crucial. This includes implementing secure coding practices, regular security audits, and vulnerability assessments throughout the AI lifecycle.

  • Cross-Domain Collaboration: Sharing knowledge and strategies across industries and domains can lead to a more robust understanding of AI threats and mitigation strategies, fostering a community approach to AI security.

Looking Further Ahead

Beyond the immediate horizon, the field of AI security is set to witness several meaningful advancements:

  • Autonomous Security: AI systems capable of self-monitoring and self-defending against potential threats will soon become a reality. These systems will autonomously detect, analyze, and respond to threats in real time, greatly reducing the window for attacks.

  • Predictive Security Models: Leveraging big data and predictive analytics, AI can forecast potential security threats before they manifest. This proactive approach will allow organizations to implement defensive measures in advance.

  • AI in Cybersecurity Operations: AI will increasingly become both weapon and shield. AI is already being used to enhance cybersecurity operations, providing the ability to sift through massive amounts of data for threat detection and response at a speed and accuracy unmatchable by humans. The technology and its underlying methodologies will only get better with time. This ability for AI to remove the so-called “human speed bump” in incident detection and response will take on greater importance as the adversaries themselves increasingly leverage AI to generate malicious attacks that are at once faster, deeper, and potentially more damaging than ever before.

  • Decentralized AI Security Frameworks: With the rise of blockchain technology, decentralized approaches to AI security will likely develop. These frameworks can provide transparent and tamper-proof systems for managing AI operations securely.

  • Ethical AI Development: As part of securing AI, strong initiatives are gaining momentum to ensure that AI systems are developed with ethical considerations in mind will prevent biases and ensure fairness, thus enhancing security by aligning AI operations with human values.

As with any rapidly evolving technology, the journey toward a secure AI-driven future is complex and fraught with challenges. But with concerted effort and prudent innovation, it’s entirely within our grasp to anticipate and mitigate these risks effectively. As we advance, the integration of sophisticated AI security controls will not only protect against potential threats, it will foster trust and promote broader adoption of this transformative technology. The future of AI security is not just about defense but about creating a resilient, reliable foundation for the growth of AI across all sectors.

Charting a Path Forward in AI Security

Few technologies in the past generation have held the promise for world-altering innovation in the way AI has. Few would quibble with AI’s immense potential to disrupt and benefit human pursuits from healthcare to finance, from manufacturing to national security and beyond. Yes, Artificial Intelligence is revolutionary. But it’s not without cost. AI comes with its own inherent collection of vulnerabilities that require vigilant, innovative defenses tailored to their unique operational contexts.

As we’ve discussed, embracing sophisticated, proactive, ethical, collaborative AI security and privacy measures is the only way to ensure we’re not only safeguarding against potential threats but also fostering trust to promote the broader adoption of what most believe is a brilliantly transformative technology.

The journey towards a secure AI-driven future is indeed complex and fraught with obstacles. However, with concerted effort, continuous innovation, and a commitment to ethical practices, successfully navigating these impediments is well within our grasp. As AI continues to evolve, so too must our strategies for defending it. 

INSIGHTS | May 28, 2024

5 Signs You’re Ready for a Red Team

By IOActive Red Team

We often talk about security as a continuum; a journey toward greater maturity and increased capability. Along that path, the practice of red team testing serves as an important milestone, not just for the benefits it offers, but also for what participating in red teaming says about the state of security — overall posture, culture, commitment to continuous improvement — in any organization.

Red team tests remain one of the most effective ways to probe defenses and identify vulnerabilities. And unlike traditional penetration tests, red team exercises simulate sophisticated cyber attacks that mimic real-world threats, providing a comprehensive assessment of security posture. That said, red teams are most effective in organizations that have reached a certain strata of infosec sophistication, a level necessary to realize the benefits of this more advanced approach.

Some of this is table stakes for any kind of advanced security methodology in any organization of any size or stripe. You need to check some basic boxes before you even get to the red team checklist.

Cybersecurity Maturity That’s Above Baseline

The organization’s security foundation must be solid. That means having clear and effective security policies and procedures in place that are not only understood, but also reliably adhered to by all stakeholders. If the organization’s policies are still in the early stages of development — or if the team is still struggling to enforce existing policy — it’s too early for the kinds of stark assessments that a more sophisticated effort like red team exercises provide.

You need a comprehensive understanding of the IT and security environments. Basic security controls and best practices must be in place along with a strong security operations team monitoring and trained to respond effectively to security incidents . There should be a history of conducting penetration tests and security assessments supported by taking corrective actions from their results. These measures will not only make existing security stronger, they ensure that the insights gained from goal-oriented, adversarial testing will be actionable, meaningful, and impactful.

With those basic qualifiers in hand, here’s five specific things to look for in your current environment that indicate your enterprise is primed and ready for the rigors of red team testing.

1. There’s a Strong Internal Security Culture

A red team engagement is not just a technical challenge; it encompasses the human factor of cyber risk. If your organization has already established a strong internal security culture, it signals that you’re ready for the next level of adversarial attack simulation. This culture should include ongoing security awareness programs, regular training sessions, and a proactive approach to security issues among all employees.

Organizations with a robust security culture are better equipped to handle the findings of a red team exercise, as their employees are more likely to follow established protocols, report suspicious activities, and participate effectively in the incident response process.

At this stage, it’s also critical to be certain the security team fully understands the role and the value of the red team. This is not an isolated assessment; it’s a strategic initiative to test and enhance the organization’s overall security posture. IT and security personnel should be educated on the purpose and benefits of red teaming, ensuring that the subsequent exercises are not perceived as critiques but rather as opportunities for growth.

2. You’ve Conducted Regular Penetration Tests

When charting a course toward greater infosec maturity, there are many stops along the route. Pentesting is one of those waypoints that should come well before the red team. Pentests are less complex, but still eminently useful activities that should be a regular occurrence in any organization that is considering stepping up to red teaming.

Organizations can utilize pentesting to focus on specific applications, internal networks, or a particularly critical system, however the testing does not assess the security team’s ability to respond to an incident quickly nor the effectiveness of the existing monitoring and detection controls. Red team exercises take security assessments to the next level by emulating real threat actors and using the same tactics, techniques, and procedures (TTPs) seen in today’s sophisticated attacks.

Incorporating regular pentests demonstrates a mature security posture and a proactive approach to managing risk. Pentests ensure that the lower-hanging security vulnerabilities have been addressed prior to the red team’s more strategic, stealthy attacks.

3. Top Management Supports the Red Team Plan

The adoption of red team testing needs buy-in from top to bottom. When the C-suite understands and supports the exercise, it encourages a culture of security awareness across all levels of the company. Such commitment from executives ensures that the resources required for red team testing — read: time and money — are allocated appropriately.

If the executive team is still bogged down chasing current, defensive shortcomings and has not yet realized the value of proactive testing, it may be too early for red teaming. It’s crucial to engage top management in order to define exercise scope and objectives that align with the strategic goals of the enterprise.

Ultimately, when the red team exercise kicks off, only a handful of employees, including 1-2 execs, are aware of when it will occur and what the goals are. The purpose of an unannounced test helps to ensure that security personnel will treat any related security alerts as a real event and respond appropriately.

4. There’s a Comprehensive Incident Response Plan in Place

An organization’s readiness to respond to security incidents is a litmus test of its resilience. Red team testing is not just about identifying vulnerabilities but also about evaluating and enhancing incident response capabilities. Each action and TTP used during the exercise will be documented and mapped to the Mitre ATT&CK Framework to help the organization understand its strengths and weaknesses when it comes to attack detection and prevention.

An organization with a comprehensive incident response plan — one that’s regularly updated and tested — is in a strong position to derive the full benefits of a red team exercise. Conversely, if incident response plans are either non-existent or incomplete, a better plan might be to concentrate resources on developing the IR protocols and saving the red teaming for a later date. After the training is complete and a well-established plan has been vetted through tabletop exercises, then it’s time to put the plan to the test and identify gaps through red teaming.

5. You Have Budget Allocated for Advanced Security Measures

Investing in information security is more critical than ever, and red team testing remains one of the best investments an organization can make; one that yields high returns in identifying and mitigating critical, business-damaging risks. If the organization has dedicated budget for security measures — and is willing to allocate a portion of that budget for advanced methods such as threat hunting and red team testing — that in itself demonstrates a serious commitment to safeguarding the company’s digital assets.

Of course, the budget for red team testing shouldn’t come at the expense of other foundational security measures. Red teaming, like most advanced infosec methodologies, is best viewed as a complement to existing security strategy and an important part of the enterprise’s ongoing risk management process. Through red team exercises, the enterprise can validate that their security controls are effective and capable of detecting or stopping an advanced attack through actionable results.

Making the Most of Red Teaming

So, you’ve met all the criteria and are ready to join the ranks of the red teaming participants. That’s no small commitment. Now that you’re on the path toward adding this methodology to the organization’s security arsenal, you can build in some reasonable expectations for success metrics in the program. Here’s some of the ways your developing red team approach should continue to pay dividends over the long haul:

  • Bolstered Security Posture: By simulating realistic attacks, red team testing helps refine defenses, making organizations resilient against not only attacks that mimic real-world threat actors , but also against future, unknown threats.
  • Spotlight on Critical Vulnerabilities: A red team will uncover weaknesses and risks that preconceived notions and traditional testing often miss by chaining multiple vulnerabilities together to accomplish its goals. This is the best way to ensure that all aspects of security are being assessed and fortified, including the people and physical locations, not just technology
  • Improved Incident Response: There’s absolutely no better way to hone IR skills than through real-world attack scenarios. Red team activities will challenge and educate security and incident response teams, significantly improving the organization’s preparedness for actual attacks by using real TTPs and testing the teams’ ability to detect and react efficiently.

Red Team Testing: Taking the Next Step

Conducting red team testing is a critical component of a comprehensive security strategy, but it’s important to approach it at the right time and with the correct level of preparation. Organizations need to evaluate themselves honestly to make sure they and their skilled defenders are ready to withstand the rigor — and the potential revelations — red team testing will almost certainly bring.

Remember, cybersecurity is a continuous process, and red team testing, when the time is right, can be a crucial part of your company’s ongoing improvement. Gear up, get ready, and get testing.