GUEST BLOG | December 13, 2022

Interdependencies – Handshakes Between Critical Infrastructures | Ernie Hayden

As of this writing, the United States was recently threatened by a major railroad union strike. The railroads are a major element of the country’s critical infrastructure. Their shutdown could lead to multiple, cascading impacts on the delivery of goods and services, not only in the US but also in Canada and Mexico. Shipping lines could also be impacted by a railroad strike, since they will not be able to receive or offload containers and cargo to and from rail cars.

Per a CNN article, a rail strike would cost the US economy about $1 billion  in the first week, according to the Association of American Railroads. The ripple effect of a shipping stoppage of that magnitude could lead to global food shortages and more: the American Chemistry Council said that around $2.8 billion in chemical shipments would be impacted each week. A separate article by the Association of American Railroads also outlines how other transportation and logistics partners are not positioned to take up the slack left by a railroad strike to keep goods moving efficiently across the nation, should a service interruption occur. Specifically, in the first half of 2022, more than 75,000 rail shipments began their journey each day; in the event of a shutdown, these shipments would sit idle.

This scenario demonstrates the fundamental concept of cascading issues following the failure of one element of the country’s critical infrastructure. This demonstrates the interdependencies in play between transportation, manufacturing, chemical production, and other vital systems.

What is Critical Infrastructure?

The concept of infrastructure in the United States began around 1983. The infrastructure under discussion included highways, public transit, wastewater treatment, water resources, air traffic control, airports, and municipal water supplies. However, these were not given the attribute of “critical.”

From 1983 until 2013, the US government formed policies and held discussions regarding infrastructure and “critical infrastructure” occurred. There were multiple iterations that are beyond the scope of this article; however, the US government perspective of “critical infrastructure” being used today in 2022 was laid out by the Obama Administration in Presidential Policy Directive 21 and Executive Order 13636.

These policy documents identified a list of US infrastructure that should be considered “…so vital to the United States that the incapacity or destruction of such systems and assets would have a debilitating impact on security; national economic security; national public health or safety; or any combination of those matters.”[1]

The following are the 16 specifically identified sectors in the US Critical Infrastructure hierarchy:

  • Chemical Sector
  • Commercial Facilities Sector
  • Communications Sector
  • Critical Manufacturing Sector
  • Dams Sector
  • Defense Industrial Base Sector
  • Emergency Services Sector
  • Energy Sector
  • Financial Services Sector
  • Food and Agriculture Sector
  • Government Facilities Sector
  • Healthcare and Public Health Sector
  • Information Technology Sector
  • Nuclear Reactors, Materials, and Waste Sector
  • Transportation Systems Sector
  • Water and Wastewater Systems Sector

These appear in graphic format at the US Department of Homeland Security Cybersecurity & Infrastructure Security Agency (CISA) website.

Interdependencies Between Critical Infrastructure Sectors

As you may have observed in the discussion of the aforementioned railroad strike, the shutdown of the railroads is not a singular impact, instead impacting other aspects of commerce, manufacturing, and modes of transportation. These other impacts can be viewed as interdependencies.

Interdependencies and connections between infrastructure sectors or elements could mean that damage, disruption, or destruction to one infrastructure element may cause cascading effects negatively impacting the continued operation of another infrastructure element or sector.

These interdependencies are not always obvious, and can be subtle. Right now, the current trend towards greater and more accelerated infrastructure interdependency is due to the Internet. Some experts on critical infrastructure analysis have observed that the advent of the Internet has made critical infrastructure more complex and more interdependent, and thus more fragile.

In their research project Keeping America Safe: Toward More Secure Networks for Critical Sectors (2017), the MIT Center for International Studies observed that “…no one currently understands the extent to which electricity generation is coupled with other sectors, and therefore the risk of ‘catastrophic macroeconomic failure’ in the event of a cyber attack is not adequately known.”

A seminal report by Rinaldi, et al, Identifying, Understanding, and Analyzing Critical Infrastructure Interdependencies written just before the catastrophic events of September 11, 2001 — reported:

“…what happens to one infrastructure can directly and indirectly affect:

  • Other infrastructures,
    • Large geographic regions, and
    • Send ripples throughout the national and global economy…”

This paragraph makes one realize that understanding and studying infrastructure interdependencies is extremely important for our country and the broader North American continent.

Another analysis of critical infrastructure interdependencies noted that “…critical infrastructure interdependencies constitute a risk multiplier: they can themselves be a threat or hazard, affect the resilience and protection performance of critical infrastructure, and lead to cascading and escalating failures.”

Interdependencies need to be understood as part of any risk assessment. If they are not considered, the risk assessment could offer the wrong conclusions.

Case Studies of Critical Infrastructure Interdependencies

In addition to our brief discussion about the railroads at the beginning of this article, here are three additional case studies that demonstrate where interdependencies were highlighted by the failure of a single element or piece of critical infrastructure.

Admittedly, the first two cases are dated; however, they are well documented and provide a sense of cascading failures in two industries.

Orbiting Satellites and Critical Infrastructure Interdependencies

In May of 1998, the five-year-old Galaxy IV Communications Satellite failed. The satellite’s computer and backup computer failed, causing the satellite to tilt away from Earth. The owner, PanAmSat, was unable to realign the satellite. PanAmSat was finally able to reroute traffic the next day to another satellite; however, the Galaxy IV was considered permanently out of service, seven years short of its intended 12-year operational life.

What were the impacts of this event? Here are some examples:

  • Pager service (remember that?) was disabled for 80-90% of U.S. customers (about 45 million). This impacted physicians, emergency departments, police, fire, and emergency medical technicians. This failure alone demonstrated how this one item of critical infrastructure can impact a major system used by emergency service providers.
  • Credit card authorization services at around 5,400 Chevron stations and most Wal-Mart stores were shut down.
  • Public television transmissions were impacted. These included:
    • CBS
    • Warner Brothers
    • UPN
    • Reuters TV
    • Motor Racing Network
    • CNN Airport Channel
    • Chinese Television Network in Hong Kong
    • US Armed Forces Entertainment Network
  • Radio stations, including National Public Radio’s national feed
  • Private business TV stations such as Aetna, Microsoft, and 3M

Overall, this one satellite failure caused a broad impact across multiple industries relying on the satellite’s communications capabilities.

Seattle Tacoma Airport and Olympic Oil Pipeline Interdependencies

The Olympic Pipeline is a 400-mile interstate pipeline system running from the US-Canada border near Blaine, Washington to Portland, Oregon. It delivers aviation jet fuel, diesel fuel, and gasoline to product terminals at Seattle (Harbor Island), Seattle-Tacoma International Airport (“Sea-Tac”), Renton, Tacoma, Vancouver (WA), and Portland.

Sea-Tac Airport receives jet fuel via a special pipeline from Renton, Washington, using a 16-inch pipe, since Sea-Tac airport cannot accommodate offloading fuel trucks to its jet fuel tank farm.

On Sunday, May 23rd, 2004, a pinhole-sized leak opened in a ¾-inch sample line due to electric conduit rubbing against the tube. The leak immediately caused between 3,300 and 10,000 gallons of fuel to leak and catch fire (most of it was burned). The pipeline was immediately shut down, impacting delivery of 11 million gallons of fuel per day system-wide. The Olympic Pipeline lost $10,000 in business each hour.

On Tuesday, May 25th, two days after the leak and subsequent pipeline isolation, Sea-Tac refueling operations were in trouble. The airport had 2.9 million gallons of fuel on hand, and typically uses 1.2 million gallons per day to fuel the airline jets. As such, Sea-Tac operations requested airlines to not refuel their planes at the airport. Alaska Airlines – the primary tenant airline at Sea-Tac – decided to refuel its planes in cities like San Francisco and San Jose to reduce reliance on Sea-Tac’s fuel.

Alaska even considered canceling some flights beginning on Wednesday, May 26th, if the pipeline was not returned to service.

On Wednesday, May 26th, the pipeline flow was restored, and Sea-Tac returned to normal refueling services soon afterwards.

This story affecting Sea-Tac is another example wherein the failure of one component led to the failure of one system, thus leading to the failure and negative impact of multiple elements of critical infrastructure.

Ukraine War and Critical Infrastructure Interdependencies

Between the start of the Russian invasion on February 24th, 2022 and March 24th, 2022, multiple critical infrastructures were seriously damaged or destroyed, including:

  • 92 factories and warehouses
  • 378 schools
  • 138 healthcare institutions
  • 12 airports
  • 7 thermal power and hydroelectric power plants

The only functioning oil refinery was also destroyed.

That was the first month of the war. Up to the time of this writing, the number of destroyed critical infrastructure facilities is much larger still.

The following table lists the categories of critical infrastructure damaged by the end of the first month of the war.

Infrastructure Number of Items Cost in $US Millions
Roads (kilometers) 8,265 $27,546
Housing 4,431 $13,542
Civilian Airports 8 $6,816
Industrial Enterprises, Factories 92 $2,921
Healthcare Institutions 138 $2,466
Nuclear Power Plants 1 $2,416
Railway Stations and Rolling Stock n/a $2,205
Bridges 260 $1,452
Ports and Port Infrastructure 2 $622
Secondary and Higher Education Institutions 378 $601

Other impacts to critical infrastructure from the war in Ukraine include the following[2]:

  • The European construction industry has had difficulty sourcing building supplies from Russia and Ukraine. For instance, Ukraine produces steel, timber, pallets, and clay for ceramic tiles. Also, the sanctions against Russia are restricting availability of copper, iron ore, and steel.
  • Ukrainian and Russian nationals provide more than 15% of the global shipping workforce, meaning that the war is leading to a shortage of able-bodied seamen.
  • Russia and Ukraine produce approximately one-third of the world’s ammonia and potassium exports, ingredients necessary for agricultural fertilizer. This alone has caused fertilizer prices to increase by 20 to 50%.
  • Food security has been impacted across the Middle East, North Africa, and Western and Central Asia — Ukraine is often called a “global breadbasket.”

The conclusion here is that a war between two countries has a major impact on critical infrastructures around the globe. It is important that the risk analysis includes a review of potential infrastructure interdependencies.

Rogers Telecom Outage, Ontario, Canada

At around 3:45 AM Eastern Time on Friday, July 8, 2022, Rogers Communications — one of the largest Internet service providers in Canada — experienced a significant disruption that rendered its network unavailable for long periods of time over the course of approximately 24 hours.

This failure impacted over 10 million wireless subscribers and 2.25 million retail internet subscribers. The outage caused problems for payment systems, Automated Teller Machines (ATMs), and phone connections, primarily in eastern Canada.

For instance, the outage affected all financial institutions across Canada, including the Royal Bank of Canada. It caused a shutdown or interruption of the INTERAC payment system used by all Canadian banks, thus affecting debit card and funds transfer services. Even Toronto Dominion Bank and Bank of Montreal suffered some service and capability interruptions.

Air Canada, which is headquartered in Toronto, suffered a major impact to its call centers. Even Vancouver International Airport – thousands of miles west of Toronto – was affected. There, travelers could not pay for parking, use terminal ATMs, or purchase items at airport retailers. Cash was king!

Even Canadian government offices, such as the Canadian Radio-television and Telecommunications Commission, lost telephone connections.

Although this is not a deep dive into the cause of this outage, the point of focus for the reader is that the failure of a telephone system resulted in nationwide impacts to commerce, retail sales, and provision of services such as automobile parking. Hence, critical infrastructure interdependencies were demonstrated on that fateful day.

Recent News: Substations Shot in North Carolina

In in early December 2022, two substations were shot at by unknown assailants. Approximately 45,000 homes lost power and some water systems were shut down due to lack of electricity. Again, this is an example of ways critical infrastructure is interrelated.

Thoughts to Ponder

In his paper on critical infrastructure interdependencies, Dr. Steven Rinaldi observed: “Interdependencies, however, are a complex and difficult problem to analyze.” This author agrees. If you look at the cases summarized above, and even consider those examples you have witnessed where the failure of one piece of critical infrastructure has affected one or more other sectors, you will realize that some extraordinary critical thinking is necessary to best understand the interdependencies.

Would your analysis have predicted the downstream failures observed with the failure of a single satellite? A tiny pipeline leak? A small regional war in Eastern Europe? Or the failure of one phone system?

It makes one wonder.

Overall, this perspective is intended to get you thinking, especially if you’re performing risk analysis of critical infrastructures and associated systems. Thinking “out of the box” is appropriate when trying to best understand the broader impacts that can surface with the failure of a single critical infrastructure.

Ernie Hayden
MIPM, CISSP, GICSP (Gold), PSP

About the Author

Ernie Hayden is a highly experienced technical consultant and thought leader in the areas of cyber/physical security. He has authored a book, Critical Infrastructure Risk Assessment – The Definitive Threat Identification and Reduction Handbook (published by Rothstein), which was awarded the ASIS International 2021 Security Book of the Year. Ernie is the Founder/Principal of 443 Consulting, LLC, and can be reached at enhayden1321(_at_)gmail.com.


[1] 2001 USA Patriot Act, Page 401, Definition of Critical Infrastructure https://www.congress.gov/107/plaws/publ56/PLAW-107publ56.pdf<

[2] Many facts from McKinsey & Company analysis dated May 9, 2022

EDITORIAL | June 30, 2020

Warcodes: Attacking ICS through industrial barcode scanners

Several days ago I came across an interesting entry in the curious ‘ICS Future News’ blog run by Patrick Coyle. Before anyone becomes alarmed, the description of this blog is crystal clear about its contents:
“News about control system security incidents that you might see in the not too distant future. Any similarity to real people, places or things is purely imaginary.”
IOActive provides research-fueled security services, so when we analyze cutting-edge technologies the goal is to stay one step ahead of malicious actors in order to reduce current and future risk. Although the casino barcode hack is made up, it turns out IOActive reported similar vulnerabilities to SICK months ago, resulting in the following advisory: This blog post provides an introductory analysis of real attack vectors where custom barcodes could be leveraged to remotely attack ICS, including critical infrastructure.

Introduction

Barcode scanning is ubiquitous across sectors and markets. From retail companies to manufacturing, utilities to access control systems, barcodes are used every day to identify and track millions of assets at warehouses, offices, shops, supermarkets, industrial facilities, and airports. Depending on the purpose and the nature of the items being scanned, there are several types of barcode scanners available on the market: CCD, laser, and image-based to name a few. Also, based on the characteristics of the environment, there are wireless, corded, handheld, or fixed-mount scanners. In general terms, people are used to seeing this technology as part of their daily routine. The security community has also paid attention to these devices, mainly focusing on the fact that regular handheld barcode scanners are usually configured to act as HID keyboards. Under certain circumstances, it is possible to inject keystroke combinations that can compromise the host computer where the barcode scanner is connected. However, the barcode scanners in the industrial world have their own characteristics. shipping automation From the security perspective, the cyber-physical systems at manufacturing plants, airports, and retail facilities may initially look as though they are physically isolated systems. We all know that this is not entirely true, but reaching ICS networks should ideally involve several hops from other systems. In some deployments there is a direct path from the outside world to those systems: the asset being handled. Depending on the industrial process where barcode scanners are deployed, we can envision assets that contain untrusted inputs (barcodes) from malicious actors, such as luggage at airports or parcels in logistics facilities. In order to illustrate the issue, I will focus on the specific case I researched: smart airports.

Attack Surface of Baggage Handling Systems

Airports are really complex facilities where interoperability is a key factor. There is a plethora of systems and devices shared between different stakeholders. Passengers are also constrained in what they are allowed to carry. luggage scanning In 2016, ENISA published a comprehensive paper on securing smart airports. Their analysis provided a plethora of interesting details; among which was a ranking of the most critical systems for airport resilience. Most modern baggage handling systems, including self-service bag drop, rely on fixed-mount laser scanners to identify and track luggage tickets. Devices such as the SICK CLV65X are typically deployed at the automatic baggage handling systems and baggage self-drop systems operating at multiple airports.

The Baggage Connection – SICK Whitepaper

More specifically, SICK CLV62x-65x barcode scanners support ‘profile programming’ barcodes. These are custom barcodes that, once scanned, directly modify settings in a device, without involving a host computer. This functionality relies in custom CODE128 barcodes that trigger certain actions in the device and can be leveraged to change configuration settings. A simple search on YouTube results in detailed walkthroughs of some of the largest airport’s baggage handling systems, which are equipped with SICK devices. Custom CODE128 barcodes do not implement any kind of authentication, so once they are generated they will work on any SICK device that supports them. As a result, an attacker that is able to physically present a malicious profile programming barcode to a device can either render it inoperable or change its settings to facilitate further attacks. We successfully tested this attack against a SICK CLV650 device.

Technical Details

IOActive reverse engineered the logic used to generate profile programming barcodes (firmware versions 6.10 and 3.10) and verified that they are not bound to specific devices. The following method in ProfileCommandBuilder.class demonstrates the lack of authentication when building profile programming barcodes.

Analyzing CLV65x_V3_10_20100323.jar and CLV62x_65x_V6.10_STD

 

Conclusion

The attack vector described in this blog post can be exploited in various ways, across multiple industries and will be analyzed in future blog posts. Also, according to IOActive’s experience, it is likely that similar issues affect other barcode manufacturers. IOActive reported these potential issues to SICK via its PSIRT in late February 2020. SICK handled the disclosure process in a diligent manner and I would like to publicly thank SICK for its coordination and prompt response in providing its clients with proper mitigations.
RESEARCH | August 22, 2017

Exploiting Industrial Collaborative Robots

Traditional industrial robots are boring. Typically, they are autonomous or operate with limited guidance and execute repetitive, programmed tasks in manufacturing and production settings.1 They are often used to perform duties that are dangerous or unsuitable for workers; therefore, they operate in isolation from humans and other valuable machinery.

This is not the case with the latest generation collaborative robots (“cobots”) though. They function with co-workers in shared workspaces while respecting safety standards. This generation of robots works hand-in-hand with humans, assisting them, rather than just performing automated, isolated operations. Cobots can learn movements, “see” through HD cameras, or “hear” through microphones to contribute to business success.

UR5 by Universal Robots2
Baxter by Rethink Robotics3

So cobots present a much more interesting attack surface than traditional industrial robots. But are cobots only limited to industrial applications? NO, they can also be integrated into other settings!

 
The Moley Robotic Kitchen (2xUR10 Arms)4
DARPA’s ALIAS Robot (UR3 Arm)5
Last February, Cesar Cerrudo (@cesarcer) and I published a non-technical paper “Hacking Robots Before Skynet” previewing our research on several home, business, and industrial robots from multiple well-known vendors. We discovered nearly 50 critical security issues. Within the cobot sector, we audited leading robots, including Baxter/Sawyer from Rethink Robotics and UR by Universal Robots.
      Baxter/Sawyer: We found authentication issues, insecure transport in their protocols, default deployment problems, susceptibility to physical attacks, and the usage of ROS, a research framework known to be vulnerable to multiple issues. The major problems we reported appear to have been patched by the company in February 2017.
     UR: We found authentication issues in many of the control protocols, susceptibility to physical attacks, memory corruption vulnerabilities, and insecure communication transport. All of the issues remain unpatched in the latest version (3.4.2.65, May 2017).6

In accordance with IOActive’s responsible disclosure policy we contacted the vendors last January, so they have had ample time to address the vulnerabilities and inform their customers. Our goal is to make cobots more secure and prevent vulnerabilities from being exploited by attackers to cause serious harm to industries, employees, and their surroundings. I truly hope this blog entry moves the collaborative industry forward so we can safely enjoy this and future generations of robots.

In this post, I will discuss how an attacker can chain multiple vulnerabilities in a leading cobot (UR3, UR5, UR10 – Universal Robots) to remotely modify safety settings, violating applicable safety laws and, consequently, causing physical harm to the robot’s surroundings by moving it arbitrarily.

This attack serves as an example of how dangerous these systems can be if they are hacked. Manipulating safety limits and disabling emergency buttons could directly threaten human life. Imagine what could happen if an attack targeted an array of 64 cobots as is found in a Chinese industrial corporation.7

The final exploit abuses six vulnerabilities to change safety limits and disable safety planes and emergency buttons/sensors remotely over the network. The cobot arm swings wildly about, wreaking havoc. This video demonstrates the attack: https://www.youtube.com/watch?v=cNVZF7ZhE-8

Q: Can these robots really harm a person?
A: Yes, a study8 by the Control and Robotics Laboratory at the École de technologie supérieure (ÉTS) in Montreal (Canada) clearly shows that even the smaller UR5 model is powerful enough to seriously harm a person. While running at slow speeds, their force is more than sufficient to cause a skull fracture.9Q: Wait…don’t they have safety features that prevent them from harming nearby humans?
A: Yes, but they can be hacked remotely, and I will show you how in the next technical section.Q: Where are these deployed?
A: All over the world, in multiple production environments every day.10Integrators Define All Safety Settings

Universal Robots is the manufacturer of UR robots. The company that installs UR robots in a specific application is the integrator. Only an integrated and installed robot is considered a complete machine. The integrators of UR robots are responsible for ensuring that any significant hazard in the entire robot system is eliminated. This includes, but is not limited to:11

  • Conducting a risk assessment of the entire system. In many countries this is required by law
  • Interfacing other machines and additional safety devices if deemed appropriate by the risk assessment
  • Setting up the appropriate safety settings in the Polyscope software (control panel)
  • Ensuring that the user will not modify any safety measures by using a “safety password.
  • Validating that the entire system is designed and installed correctly

Universal Robots has recognized potentially significant hazards, which must be considered by the integrator, for example:

  • Penetration of skin by sharp edges and sharp points on tools or tool connectors
  • Penetration of skin by sharp edges and sharp points on obstacles near the robot track
  • Bruising due to stroke from the robot
  • Sprain or bone fracture due to strokes between a heavy payload and a hard surface
  • Mistakes due to unauthorized changes to the safety configuration parameters

Some safety-related features are purposely designed for cobot applications. These features are particularly relevant when addressing specific areas in the risk assessment conducted by the integrator, including:

  • Force and power limiting: Used to reduce clamping forces and pressures exerted by the robot in the direction of movement in case of collisions between the robot and operator.
  • Momentum limiting: Used to reduce high-transient energy and impact forces in case of collisions between robot and operator by reducing the speed of the robot.
  • Tool orientation limiting: Used to reduce the risk associated with specific areas and features of the tool and work-piece (e.g., to avoid sharp edges to be pointed towards the operator).
  • Speed limitation: Used to ensure the robot arm operates a low speed.
  • Safety boundaries: Used to restrict the workspace of the robot by forcing it to stay on the correct side of defined virtual planes and not pass through them.

Safety planes in action12


Safety I/O: When this input safety function is triggered (via emergency buttons, sensors, etc.), a low signal is sent to the inputs and causes the safety system to transition to “reduced” mode.

Safety scanner13
Safety settings are effective in preventing many potential incidents. But what could happen if malicious actors targeted these measures in order to threaten human life?
Statement from the UR User Guide

Changing Safety Configurations Remotely

“The safety configuration settings shall only be changed in compliance with the risk assessment conducted by the integrator.14 If any safety parameter is changed the complete robot system shall be considered new, meaning that the overall safety approval process, including risk assessment, shall be updated accordingly.”
 

The exploitation process to remotely change the safety configuration is as follows:

Step 1.    Confirm the remote version by exploiting an authentication issue on the UR Dashboard Server.
Step 2.    Exploit a stack-based buffer overflow in UR Modbus TCP service, and execute commands as root.
Step 3.    Modify the safety.conf file. This file overrides all safety general limits, joints limits, boundaries, and safety I/O values.
Step 4.    Force a collision in the checksum calculation, and upload the new file. We need to fake this number since integrators are likely to write a note with the current checksum value on the hardware, as this is a common best practice.
Step 5.    Restart the robot so the safety configurations are updated by the new file. This should be done silently.
Step 6.    Move the robot in an arbitrary, dangerous manner by exploiting an authentication issue on the UR control service.

By analyzing and reverse engineering the firmware image ursys-CB3.1-3.3.4-310.img, I was able to understand the robot’s entry points and the services that allow other machines on the network to interact with the operating system. For this demo I used the URSim simulator provided by the vendor with the real core binary from the robot image. I was able to create modified versions of this binary to run partially on a standard Linux machine, even though it was clearer to use the simulator for this example exploit.

Different network services are exposed in the URControl core binary, and most of the proprietary protocols do not implement strong authentication mechanisms. For example, any user on the network can issue a command to one of these services and obtain the remote version of the running process (Step 1):


Now that I have verified the remote target is running a vulnerable image, ursys-CB3.1-3.3.4-310 (UR3, UR5 or UR10), I exploit a network service to compromise the robot (Step 2).The UR Modbus TCP service (port 502) does not provide authentication of the source of a command; therefore, an adversary could corrupt the robot in a state that negatively affects the process being controlled. An attacker with IP connectivity to the robot can issue Modbus read/write requests and partially change the state of the robot or send requests to actuators to change the state of the joints being controlled.It was not possible to change any safety settings by sending Modbus write requests; however, a stack-based buffer overflow was found in the UR Modbus TCP receiver (inside the URControl core binary).A stack buffer overflows with the recv function, because it uses a user-controlled buffer size to determine how many bytes from the socket will be copied there. This is a very common issue.

Before proceeding with the exploit, let’s review the exploit mitigations in place. The robot’s Linux kernel is configured to randomize (randomize_va_space=1 => ASLR) the positions of the stack, virtual dynamic shared object page, and shared memory regions. Moreover, this core binary does not allow any of its segments to be both writable and executable due to the “No eXecute” (NX) bit.

While overflowing the destination buffer, we also overflow pointers to the function’s arguments. Before the function returns, these arguments are used in other function calls, so we have to provide these calls with a valid value/structure. Otherwise, we will never reach the end of the function and be able to control the execution flow.

edx+0x2c is dereferenced and used as an argument for the call to 0x82e0c90. The problem appears afterwards when EBX (which is calculated from our previous controlled pointer on EDX) needs to also point to a structure where a file descriptor is obtained and is closed with “close” afterwards.

To choose a static address that might comply with these two requirements, I used the following static region, since all others change due to ASLR.

I wrote some scripts to find a suitable memory address and found 0x83aa1fc to be perfect since it suits both conditions:
      0x83aa1fc+0x2c points to valid memory -> 0x831c00a (“INT32”)
      0x83aa1fc+0x1014 contains 0 (nearly all this region is zeroed)
Now that I have met both conditions, execution can continue to the end of this function, and I get EIP control because the saved register on the stack was overflowed:


At this point, I control most of the registers, so I need to place my shellcode somewhere and redirect the execution flow there. For this, I used returned-oriented programming (ROP), the challenge will be to find enough gadgets to set everything I need for clean and reliable exploitation. Automatic ROP-chain tools did not work well for this binary, so I did it manually.First, I focus on my ultimate goal: to execute a reverse shell that connects to my machine. One key point when building a remote ROP-based exploit in Linux are system calls.Depending on the quality of gadgets that useint instructions I find, I will be able to use primitives such as write or dup2 to reuse the socket that was already created to return a shell or other post-exploitation strategies.In this binary, I only found one int 0x80 instruction. This is used to invoke system calls in Linux on x86. Because this is a one-shot gadget, I can only perform a single system call: I will use the execve system call to execute a program. This int 0x80 instruction requires setting up a register with the system call number (EAX in this case 0xb) and then set a register (EBX) that points to a special structure.This structure contains an array of pointers, each of which points to the arguments of the command to be executed.Because of how this vulnerability is triggered, I cannot use null bytes (0x00) on the request buffer. This is a problem because we need to send commands and arguments and also create an array of pointers that end with a null byte. To overcome this, I send a placeholder, like chunks of 0xFF bytes, and later on I replace them with 0x00 at runtime with ROP.In pseudocode this call would be (calls a TCP Reverse Shell):
 
 
All the controlled data is in the stack, so first I will try to align the stack pointer (ESP) to my largest controlled section (STAGE 1). I divided the largest controlled sections into two stages, since they both can potentially contain many ROP gadgets. 

As seen before, at this point I control EBX and EIP. Next, I have to align ESP to any of the controlled segments so I can start doing ROP.

The following gadget (ROP1 0x8220efa) is used to adjust ESP:

 
 
This way ESP = ESP + EBX – 1 (STAGE 1 address in the stack). This aligns ESP to my STAGE 1 section. EBX should decrease ESP by 0x137 bytes, so I use the number 0xfffffec9 (4294966985) because when adding it to ESP, it wraps around to the desired value.
When the retn instruction of the gadget is executed, ROP gadgets at STAGE 1 start doing their magic. STAGE 1 of the exploit does the following:
  1. Zero out the at the end of the arguments structure. This way the processor will know that EXECVE arguments are only those three pointers.
  2. Save a pointer to our first command of cmd[] in our arguments structure.
  3. Jump to STAGE 2, because I don’t have much space here.
 

STAGE 2 of the exploit does the following:

  1. Zero out the xffxffxffxff at the end of each argument in cmd[].
  2. Save a pointer to the 2nd and 3rd argument of cmd[] in on our arguments structure.
  3. Prepare registers for EXECVE. As seen before, we needed
    1. EBX=*args[]
    2. EDX = 0
    3. EAX=0xb
  4. Call the int 0x80 gadget and execute the reverse shell.
Once the TCP reverse shell payload is executed, a connection is made back to my computer. Now I can execute commands and use sudo to execute commands as root in the robot controller.
 
Safety settings are saved in the safety.conf file (Step 3). Universal Robots implemented a CRC (STM-32) algorithm to provide integrity to this file and save the calculated checksum on disk. This algorithm does not provide any real integrity to the settings, as it is possible to generate collisions or calculate new checksum values for new settings overriding special files on the filesystem. I reversed how this calculation is being made for each safety setting and created an algorithm that replicates it. On the video demo, I did not fake the new CRC value to remain the same (located on top-right of the screen), even though this is possible and easy (Step 4).
 
Before modifying any safety setting on the robot, I setup a process that automatically starts a new instance of the robot controller after 25 seconds with the new settings. This will give me enough time to download, modify, and upload a new safety setting file. The following command sets up a new URControl process. I used Python since it allows me to close all current file descriptors of the running process when forking. Remember that I am forking from the reverse shell object, so I need to create a new process that does not inherit any of the file descriptors so they are closed when the parent URControl process dies (in order to restart and apply new safety settings).
 
 
Now I have 25 seconds to download the current file, modify it, calculate new CRC, reupload it, and kill the running URControl process (which has an older safety settings config). I can programmatically use the kill command to target the current URControl instance (Step 5).
 

Finally, I send this command to the URControl service in order to load the new installation we uploaded. I also close any popup that might appear on the UI.

 
Finally, an attacker can simply call the movej function in the URControl service to move joints remotely, with a custom speed and acceleration (Step 6). This is shown at the end of the video.

Once again, I see novel and expensive technology which is vulnerable and exploitable. A very technical bug, like a buffer overflow in one of the protocols, exposed the integrity of the entire robot system to remote attacks. We reported the complete flow of vulnerabilities to the vendors back in January, and they have yet to be patched.
What are we waiting for?
 
1 https://www.robots.com/faq/show/what-is-an-industrial-robot
2 https://www.roboticsbusinessreview.com/manufacturing/cobot-market-boom-lifts-universal-robots-fortunes-2016/
3 http://www.rethinkrobotics.com/blog/humans-and-collaborative-robots-working-together-in-grand-rapids-mi/ 
https://www.youtube.com/watch?v=G6_LCwu7dOg
4https://www.wired.com/2016/11/darpa-alias-autonomous-aircraft-aurora-sikorsky/
5 https://www.universal-robots.com/how-tos-and-faqs/faq/ur-faq/release-note-software-version-34xx/
6https://www.youtube.com/watch?v=PtncirKiBXQ&t=1s
7http://coro.etsmtl.ca/blog/?p=299
8 http://www.forensicmed.co.uk/pathology/head-injury/skull-fracture/
9https://www.universal-robots.com/case-stories/
11 https://academy.universal-robots.com
12https://academy.universal-robots.com
13Software_Manual_en_US.pdf – Universal Robots

Safety planes in action12


Safety I/O: When this input safety function is triggered (via emergency buttons, sensors, etc.), a low signal is sent to the inputs and causes the safety system to transition to “reduced” mode.

Safety Scanner 13

Safety settings are effective in preventing many potential incidents. But what could happen if malicious actors targeted these measures in order to threaten human life?

Statement from the UR User Guide

Changing Safety Configurations Remotely

“The safety configuration settings shall only be changed in compliance with the risk assessment conducted by the integrator.14 If any safety parameter is changed the complete robot system shall be considered new, meaning that the overall safety approval process, including risk assessment, shall be updated accordingly.”
 

The exploitation process to remotely change the safety configuration is as follows:

Step 1.    Confirm the remote version by exploiting an authentication issue on the UR Dashboard Server.
Step 2.    Exploit a stack-based buffer overflow in UR Modbus TCP service, and execute commands as root.
Step 3.    Modify the safety.conf file. This file overrides all safety general limits, joints limits, boundaries, and safety I/O values.
Step 4.    Force a collision in the checksum calculation, and upload the new file. We need to fake this number since integrators are likely to write a note with the current checksum value on the hardware, as this is a common best practice.
Step 5.    Restart the robot so the safety configurations are updated by the new file. This should be done silently.
Step 6.    Move the robot in an arbitrary, dangerous manner by exploiting an authentication issue on the UR control service.

By analyzing and reverse engineering the firmware image ursys-CB3.1-3.3.4-310.img, I was able to understand the robot’s entry points and the services that allow other machines on the network to interact with the operating system. For this demo I used the URSim simulator provided by the vendor with the real core binary from the robot image. I was able to create modified versions of this binary to run partially on a standard Linux machine, even though it was clearer to use the simulator for this example exploit.

Different network services are exposed in the URControl core binary, and most of the proprietary protocols do not implement strong authentication mechanisms. For example, any user on the network can issue a command to one of these services and obtain the remote version of the running process (Step 1):


Now that I have verified the remote target is running a vulnerable image, ursys-CB3.1-3.3.4-310 (UR3, UR5 or UR10), I exploit a network service to compromise the robot (Step 2).The UR Modbus TCP service (port 502) does not provide authentication of the source of a command; therefore, an adversary could corrupt the robot in a state that negatively affects the process being controlled. An attacker with IP connectivity to the robot can issue Modbus read/write requests and partially change the state of the robot or send requests to actuators to change the state of the joints being controlled.It was not possible to change any safety settings by sending Modbus write requests; however, a stack-based buffer overflow was found in the UR Modbus TCP receiver (inside the URControl core binary).A stack buffer overflows with the recv function, because it uses a user-controlled buffer size to determine how many bytes from the socket will be copied there. This is a very common issue.

Before proceeding with the exploit, let’s review the exploit mitigations in place. The robot’s Linux kernel is configured to randomize (randomize_va_space=1 => ASLR) the positions of the stack, virtual dynamic shared object page, and shared memory regions. Moreover, this core binary does not allow any of its segments to be both writable and executable due to the “No eXecute” (NX) bit.

While overflowing the destination buffer, we also overflow pointers to the function’s arguments. Before the function returns, these arguments are used in other function calls, so we have to provide these calls with a valid value/structure. Otherwise, we will never reach the end of the function and be able to control the execution flow.

edx+0x2c is dereferenced and used as an argument for the call to 0x82e0c90. The problem appears afterwards when EBX (which is calculated from our previous controlled pointer on EDX) needs to also point to a structure where a file descriptor is obtained and is closed with “close” afterwards.

To choose a static address that might comply with these two requirements, I used the following static region, since all others change due to ASLR.

I wrote some scripts to find a suitable memory address and found 0x83aa1fc to be perfect since it suits both conditions:
      0x83aa1fc+0x2c points to valid memory -> 0x831c00a (“INT32”)
      0x83aa1fc+0x1014 contains 0 (nearly all this region is zeroed)
Now that I have met both conditions, execution can continue to the end of this function, and I get EIP control because the saved register on the stack was overflowed:

At this point, I control most of the registers, so I need to place my shellcode somewhere and redirect the execution flow there. For this, I used returned-oriented programming (ROP), the challenge will be to find enough gadgets to set everything I need for clean and reliable exploitation. Automatic ROP-chain tools did not work well for this binary, so I did it manually.
 
First, I focus on my ultimate goal: to execute a reverse shell that connects to my machine. One key point when building a remote ROP-based exploit in Linux are system calls. Depending on the quality of gadgets that use int instructions I find, I will be able to use primitives such as write or dup2 to reuse the socket that was already created to return a shell or other post-exploitation strategies.

In this binary, I only found one int 0x80 instruction. This is used to invoke system calls in Linux on x86. Because this is a one-shot gadget, I can only perform a single system call: I will use the execve system call to execute a program. This int 0x80 instruction requires setting up a register with the system call number (EAX in this case 0xb) and then set a register (EBX) that points to a special structure. This structure contains an array of pointers, each of which points to the arguments of the command to be executed.
 
Because of how this vulnerability is triggered, I cannot use null bytes (0x00) on the request buffer. This is a problem because we need to send commands and arguments and also create an array of pointers that end with a null byte. To overcome this, I send a placeholder, like chunks of 0xFF bytes, and later on I replace them with 0x00 at runtime with ROP.
 
In pseudocode this call would be (calls a TCP Reverse Shell):
 
 
All the controlled data is in the stack, so first I will try to align the stack pointer (ESP) to my largest controlled section (STAGE 1). I divided the largest controlled sections into two stages, since they both can potentially contain many ROP gadgets. 

As seen before, at this point I control EBX and EIP. Next, I have to align ESP to any of the controlled segments so I can start doing ROP.
The following gadget (ROP1 0x8220efa) is used to adjust ESP:
 
 
This way ESP = ESP + EBX – 1 (STAGE 1 address in the stack). This aligns ESP to my STAGE 1 section. EBX should decrease ESP by 0x137 bytes, so I use the number 0xfffffec9 (4294966985) because when adding it to ESP, it wraps around to the desired value.
When the retn instruction of the gadget is executed, ROP gadgets at STAGE 1 start doing their magic. STAGE 1 of the exploit does the following:
  1. Zero out the at the end of the arguments structure. This way the processor will know that EXECVE arguments are only those three pointers.
  2. Save a pointer to our first command of cmd[] in our arguments structure.
  3. Jump to STAGE 2, because I don’t have much space here.
 

STAGE 2 of the exploit does the following:

  1. Zero out the xffxffxffxff at the end of each argument in cmd[].
  2. Save a pointer to the 2nd and 3rd argument of cmd[] in on our arguments structure.
  3. Prepare registers for EXECVE. As seen before, we needed
    1. EBX=*args[]
    2. EDX = 0
    3. EAX=0xb
  4. Call the int 0x80 gadget and execute the reverse shell.
Once the TCP reverse shell payload is executed, a connection is made back to my computer. Now I can execute commands and use sudo to execute commands as root in the robot controller.
 
Safety settings are saved in the safety.conf file (Step 3). Universal Robots implemented a CRC (STM-32) algorithm to provide integrity to this file and save the calculated checksum on disk. This algorithm does not provide any real integrity to the settings, as it is possible to generate collisions or calculate new checksum values for new settings overriding special files on the filesystem. I reversed how this calculation is being made for each safety setting and created an algorithm that replicates it. On the video demo, I did not fake the new CRC value to remain the same (located on top-right of the screen), even though this is possible and easy (Step 4).
 
Before modifying any safety setting on the robot, I setup a process that automatically starts a new instance of the robot controller after 25 seconds with the new settings. This will give me enough time to download, modify, and upload a new safety setting file. The following command sets up a new URControl process. I used Python since it allows me to close all current file descriptors of the running process when forking. Remember that I am forking from the reverse shell object, so I need to create a new process that does not inherit any of the file descriptors so they are closed when the parent URControl process dies (in order to restart and apply new safety settings).
 
 
Now I have 25 seconds to download the current file, modify it, calculate new CRC, reupload it, and kill the running URControl process (which has an older safety settings config). I can programmatically use the kill command to target the current URControl instance (Step 5).
 

Finally, I send this command to the URControl service in order to load the new installation we uploaded. I also close any popup that might appear on the UI.

 

Finally, an attacker can simply call the movej function in the URControl service to move joints remotely, with a custom speed and acceleration (Step 6). This is shown at the end of the video.

Once again, I see novel and expensive technology which is vulnerable and exploitable. A very technical bug, like a buffer overflow in one of the protocols, exposed the integrity of the entire robot system to remote attacks. We reported the complete flow of vulnerabilities to the vendors back in January, and they have yet to be patched.

What are we waiting for?

 
1 https://www.robots.com/faq/show/what-is-an-industrial-robot
2 https://www.roboticsbusinessreview.com/manufacturing/cobot-market-boom-lifts-universal-robots-fortunes-2016/
3 http://www.rethinkrobotics.com/blog/humans-and-collaborative-robots-working-together-in-grand-rapids-mi/ 
https://www.youtube.com/watch?v=G6_LCwu7dOg
4https://www.wired.com/2016/11/darpa-alias-autonomous-aircraft-aurora-sikorsky/
5 https://www.universal-robots.com/how-tos-and-faqs/faq/ur-faq/release-note-software-version-34xx/
6https://www.youtube.com/watch?v=PtncirKiBXQ&t=1s
7http://coro.etsmtl.ca/blog/?p=299
8 http://www.forensicmed.co.uk/pathology/head-injury/skull-fracture/
9https://www.universal-robots.com/case-stories/
11 https://academy.universal-robots.com
12https://academy.universal-robots.com
13Software_Manual_en_US.pdf – Universal Robots
RESEARCH | July 2, 2015

Hacking Wireless Ghosts Vulnerable For Years

Is the risk associated to a Remote Code Execution vulnerability in an industrial plant the same when it affects the human life? When calculating risk, certain variables and metrics are combined into equations that are rendered as static numbers, so that risk remediation efforts can be prioritized. But such calculations sometimes ignore the environmental metrics and rely exclusively on exploitability and impact. The practice of scoring vulnerabilities without auditing the potential for collateral damage could underestimate a cyber attack that affects human safety in an industrial plant and leads to catastrophic damage or loss. These deceiving scores are always attractive for attackers since lower-priority security issues are less likely to be resolved on time with a quality remediation.

In the last few years, the world has witnessed advanced cyber attacks against industrial components using complex and expensive malware engineering. Today the lack of entry points for hacking an isolated process inside an industrial plant mean that attacks require a combination of zero-day vulnerabilities and more money.

Two years ago, Carlos Mario Penagos (@binarymantis) and I (Lucas Apa) realized that the most valuable entry point for an attacker is in the air. Radio frequencies leak out of a plant’s perimeter through the high-power antennas that interconnect field devices. Communicating with the target devices from a distance is priceless because it allows an attack to be totally untraceable and frequently unstoppable.

In August 2013 at Black Hat Briefings, we reported multiple vulnerabilities in the industrial wireless products of three vendors and presented our findings. We censored vendor names from our paper to protect the customers who use these products, primarily nuclear, oil and gas, refining, petro-chemical, utility, and wastewater companies mostly based in North America, Latin America, India, and the Middle East (Bahrain, Kuwait, Oman, Qatar, Saudi Arabia, and UAE). These companies have trusted expensive but vulnerable wireless sensors to bridge the gap between the physical and digital worlds.

First, we decided to target wireless transmitters (sensors). These sensors gather the physical, real-world values used to monitor conditions, including liquid level, pressure, flow, and temperature. These values are precise enough to be trusted by all of the industrial hardware and machinery in the field. Crucial decisions are based on these numbers. We also targeted wireless gateways, which collect this information and communicate it to the backbone SCADA systems (RTU/EFM/PLC/HMI).

In June 2013, we reported eight different vulnerabilities to the ICS-CERT (Department of Homeland Security). Three months later, one of the vendors, ProSoft Technology released a patch to mitigate a single vulnerability.

After a patient year, IOActive Labs in 2014 released an advisory titled “OleumTech Wireless Sensor Network Vulnerabilities” describing four vulnerabilities that could lead to process compromise, public damage, and employee safety, potentially leading to the loss of life.

Figure 1: OleumTech Transmitters infield

The following OleumTech Products are affected:

  • All OleumTech Wireless Gateways: WIO DH2 and Base Unit (RFv1 Protocol)
  • All OleumTech Transmitters and Wireless Modules (RFv1 Protocol)
  • BreeZ v4.3.1.166

An untrusted user or group within a 40-mile range could inject false values on the wireless gateways in order to modify measurements used to make critical decisions. In the following video demonstration, an attacker makes a chemical react and explode by targeting a wireless transmitter that monitors the process temperature. This was possible because a proper failsafe mechanism had not been implemented and physical controls failed. Heavy machinery makes crucial decisions based on the false readings; this could give the attacker control over part of the process.

Figure 2: OleumTech DH2 used as the primary Wireless Gateway to collect wireless end node data.
Video:  Attack launched using a 40 USD RF transceiver and antenna

Industrial embedded systems’ vulnerabilities that can be exploited remotely without needing any internal access are inherently appealing for terrorists.

Mounting a destructive, real-world attack in these conditions is possible. These products are in commercial use in industrial plants all over the world. As if causing unexpected chemical reactions is not enough, exploiting a remote, wireless memory corruption vulnerability could shut down the sensor network of an entire facility for an undetermined period of time.

In May 2015, two years from the initial private vulnerability disclosure, OleumTech created an updated RF protocol version (RFv2) that seems to allow users to encrypt their wireless traffic with AES256. Firmware for all products was updated to support this new feature.

Still, are OleumTech customers aware of how the new AES Encryption key is generated? Which encryption key is the network using?

Figure 3: Picture from OleumTech BreeZ 5 – Default Values (AES Encryption)

Since every hardware device should be unmounted from the field location for a manual update, what is the cost?

IOActive Labs hasn’t tested these firmware updates. We hope that OleumTech’s technical team performed testing to ensure that the firmware is properly securing radio communications.

I am proud that IOActive has one of the largest professional teams of information security researchers who work with ICS-CERT (DHS) in the world. In addition to identifying critical vulnerabilities and threats for power system facilities, the IOActive team provides security testing directly for control system manufacturers and businesses that have industrial facilities – proactively detecting weaknesses and anticipating exploits in order to improve the safety and operational integrity of technologies.

Needless to say, the companies that rely on vulnerable devices could lose much more than millions of dollars if these vulnerabilities are exploited. These flaws have the potential for massive economic and sociological impact, as well as loss of human life. On the other hand, some attacks are undetectable so it is possible that some of these devices already have been exploited in the wild. We may never know. Fortunately, customers now have a stronger security model and I expect that they now are motivated enough to get involved and ask the vulnerable vendors these open questions.

RESEARCH | April 17, 2014

A Wake-up Call for SATCOM Security

During the last few months we have witnessed a series of events that will probably be seen as a tipping point in the public’s opinion about the importance of, and need for, security. The revelations of Edward Snowden have served to confirm some theories and shed light on surveillance technologies that were long restricted.
 
We live in a world where an ever-increasing stream of digital data is flowing between continents. It is clear that those who control communications traffic have an upper-hand.
 
Satellite Communications (SATCOM) plays a vital role in the global telecommunications system. Sectors that commonly rely on satellite networks include:
  • Aerospace
  • Maritime
  • Military and governments
  • Emergency services
  • Industrial (oil rigs, gas, electricity)
  • Media
It is important to mention that certain international safety regulations for ships such as GMDSS or aircraft’s ACARS rely on satellite communication links. In fact, we recently read how, thanks to the SATCOM equipment on board Malaysian Airlines MH370, Inmarsat engineers were able to determine the approximate position of where the plane crashed. 
 
IOActive is committed to improving overall security. The only way to do so is to analyze the security posture of the entire supply chain, from the silicon level to the upper layers of software. 
 
Thus, in the last quarter of 2013 I decided to research into a series of devices that, although widely deployed, had not received the attention they actually deserve. The goal was to provide an initial evaluation of the security posture of the most widely deployed Inmarsat and Iridium SATCOM terminals.  
 
In previous blog posts I’ve explained the common approach when researching complex devices that are not physically accessible. In these terms, this research is not much different than the previous research: in most cases the analysis was performed by reverse engineering the firmware statically.

 
What about the results? 
 
Insecure and undocumented protocols, backdoors, hard-coded credentials…mainly design flaws that allow remote attackers to fully compromise the affected devices using multiple attack vectors.
 
Ships, aircraft, military personnel, emergency services, media services, and industrial facilities (oil rigs, gas pipelines, water treatment plants, wind turbines, substations, etc.) could all be affected by these vulnerabilities.
 
I hope this research is seen as a wake-up call for both the vendors and users of the current generation of SATCOM technology. We will be releasing full technical details in several months, at Las Vegas, so stay tuned.
The following white paper comprehensively explains all the aspects of this research IOActive_SATCOM_Security_WhitePaper
INSIGHTS | February 17, 2014

INTERNET-of-THREATS

At IOActive Labs, I have the privilege of being part of a great team with some of the world’s best hackers. I also have access to really cool research on different technologies that uncovers security problems affecting widely used hardware and software. This gives me a solid understanding of the state of security for many different software and hardware devices, not just opinions based on theories and real life experience.
 
Currently, the term Internet-of-Things (IoT) is becoming a buzzword used in the media, announcements from hardware device manufacturers, etc. Basically, it’s used to describe an Internet with everything connected to it. It describes what we are seeing nowadays, including:
  • Laptops, tablets, smartphones, set-top boxes, media-streaming devices, and data-storage devices
  • Watches, glasses, and clothes
  • Home appliances, home switches, home alarm systems, home cameras, and light bulbs
  • Industrial devices and industrial control systems
  • Cars, buses, trains, planes, and ships
  • Medical devices and health systems
  • Traffic sensors, seismic sensors, pollution sensors, and weather sensors
     …and more; you name it, and it is or soon will be connected to the Internet.
 
While the devices and systems connected to the Internet are different, they have something in common–most of them suffer from serious security vulnerabilities. This is not a guess. It is based on IOActive Labs’ security research into many of these types of devices currently being used worldwide. Sadly, we are seeing almost the exact same vulnerabilities on these devices that have plagued software vendors over the last decade. Vulnerabilities that the most important software vendors are trying hard to eradicate. It seems that many hardware companies are following really poor security practices when adding software to their products and connecting them to the Internet. What is worse is that sometimes vendors don’t even respond to security vulnerability reports or just downplay the threat and don’t fix the vulnerabilities. Many vendors don’t even know how to properly deal with the security vulnerabilities being reported.
 
Some of common vulnerabilities IOActive Labs finds include:
  • Sensitive data sent over insecure channels
  • Improper use of encryption
    • No SSL certificate validation
    • Things like encryption keys and signing certificates easily available to anyone
  • Hardcoded credentials/backdoor accounts
  • Lack of authentication and/or authorization
  • Storage of sensitive data in clear text
  • Unauthenticated and/or unauthorized firmware updates
  • Lack of firmware integrity check during updates
  • Use of insecure custom made protocols
Also, data ambition is working against vendors and is increasing attack surfaces considerably. For example, all data collected is sent to “vendor cloud” and device commands are sent from “vendor cloud”, instead of just allowing users to connect directly to and command their devices. Hacking into “vendor cloud” = thousands of devices compromised = lots of lost money.
 
Why should we worry about all of this? Well, these devices affect our everyday life and will continue to do so more and more. We’ve only seen the tip of the iceberg when it comes to the attacks that people, companies, and governments face and how easily they can be performed. If the situation doesn’t change soon, it is just matter of time before we witness attacks with tragic consequences.
 
If a headline like “+100K Digital Toilets from XYZ1.3 Inc. Found Sending Spam and Distributing Malware” doesn’t scare you because you think it’s funny and improbable, you could be wrong. We shouldn’t wait for headlines such as “Dozens of People Injured When Home Automation Devices Hacked” before we react.
 
Something must be done! From enforcing secure practices during product development to imposing high fines when products are hacked, action must be taken to prevent the loss of money and possibly even lives.
 
Companies should strongly consider:
    • Training developers on secure development
    • Implementing security development practices to improve software security
    • Training company staff on security best practices
    • Implementing a security patch development and distribution process
    • Performing product design/architecture security reviews
    • Performing source code security audits
    • Performing product penetration tests
    • Performing company network penetration tests
    • Staying up-to-date with new security threats
    • Creating a bug bounty program to reward reported vulnerabilities and clearly defining how vulnerabilities should be reported
    • Implementing a security incident/emergency response team
It is difficult to give advice to end users given that the best solution is just not to buy or use many products because they are insecure by design. At this stage, it’s just matter of being lucky and hoping that you won’t be hacked. Maybe opportunistic vendors could come up with some novel solution such as an IPS/anti* device that will protect all of your devices from attacks. Just pray that the protection device itself is not vulnerable.
 
Sometimes end users are forced to live with insecure devices since there isn’t any way to turn them off or not to use them. These include devices provided by TV cable companies, electricity and gas companies, public services companies, governments, etc. These companies and the government should take responsibility for deploying secure products.
 
This is not BS–in a couple of days we will be releasing some of the extensive research I mentioned and on which this blog post is based.
 
I intend for this post to be a wakeup call for everyone! I’m really concerned about the current situation. In the meantime, I will use the term INTERNET-of-THREATS (not Internet-of-Things). Maybe this new buzzword will make us more conscious of the situation. If it doesn’t, then at least I have tried.
INSIGHTS | November 11, 2013

Practical and cheap cyberwar (cyber-warfare): Part I

Every day we hear about a new vulnerability or a new attack technique, but most of the time it’s difficult to imagine the real impact. The current emphasis on cyberwar (cyber-warfare if you prefer) leads to myths and nonsense being discussed. I wanted to show real life examples of large scale attacks with big impacts on critical infrastructure, people, companies, etc.
 

The idea of this post is to raise awareness. I want to show how vulnerable some industrial, oil, and gas installations currently are and how easy it is to attack them. Another goal is to pressure vendors to produce more secure devices and to speed up the patching process once vulnerabilities are reported.


The attack in this post is based on research done by my fellow pirates, Lucas Apa and Carlos Penagos. They found critical vulnerabilities in wireless industrial control devices. This research was first presented at BH USA 2013. You can find their full presentation here https://www.blackhat.com/us-13/archives.html#Apa
 
A common information leak occurs when vendors highlight how they helped Company X with their services or products. This information is very useful for supply chain attacks. If you are targeting Company X, it’s good to look at their service and product providers. It’s also useful to know what software/devices/technology they use.

 

In this case, one of the vendors that sells vulnerable wireless industrial control devices is happy to announce in a press release that Company X has acquired its wireless sensors and is using them in the Haynesville Shale fields. So, as an attacker, we now know that Company X is using vulnerable wireless sensors at the Haynesville Shale fields. Haynesville Shale fields, what’s that? Interesting, with a quick Google search you end up with:
 
 
 
How does Google know about shale well locations? It’s simple, publically-available information. You can display wells by name, organization, etc.:
 
 
 
 
 
Even interactive maps are available:
 
 
 
You can find all of Company X’s wells along with their exact location (geographical coordinates). You know almost exactly where the vulnerable wireless sensors are installed.
 
Since the wells are at a remote location, exploiting the wireless sensor vulnerabilities becomes an interesting challenge. Enter drones, UAV unmanned aerial vehicles. Commercially available drones range from a couple hundred dollars to tens of thousands dollars, depending on range, endurance, functionality, etc. You can even build your own and save some money. The idea is to put the attack payload in a drone, send it to the wells’ location, and launch the attack. This isn’t difficult to do since drones can be programmed to fly to x,y coordinates and execute the payload while flying around the target coordinates (no need to return). 
 
Depending on your budget, you can launch an attack from a nearby location or very far away. Depending on the drone’s endurance, you can be X miles away from the target. You can extend the drone’s range depending on the radio and antenna used. 
 
The types of exploits you could launch from the drone range from bricking all of the wireless devices to causing some physical harm on the shale gas installations. Manipulating device firmware or injecting fake data on radio packets could make the control systems believe things like the temperature or pressure are wrong. Just bricking the devices could result in significant lost money to Company X. The devices would need to be reconfigured/reflashed. The exploits could interfere with shale gas extraction and even halt production. The consequences of an attack could be even more catastrophic depending on how the vulnerable devices are being used.
 
Attacks could be expanded to target more than just one vendor’s device. Drones could do reconnaissance first, scan and identify devices from different vendors, and then launch attacks targeting all of the specific devices.
 
In order to highlight attack possibilities and impact consequences I extracted the following from http://www.onworld.com/news/newsoilandgas.html (the companies mentioned in this article are not necessarily vulnerable, this is just for illustrative purposes):
 
“…Pipelines & Corrosion Monitoring
Wireless flow, pressure, level, temperature and valve position monitoring are used to streamline pipeline operation and storage while increasing safety and regulatory compliance. In addition, wireless sensing solutions are targeted at the billions of dollars per year that is spent managing pipeline corrosion. While corrosion is a growing problem for the aging pipeline infrastructure it can also lead to leaks, emissions and even deadly explosions in production facilities and refineries….”
 
Leaks and deadly explosions can have sad consequences.
 
Cyber criminals, terrorists, state actors, etc. can launch big impact attacks with relatively small budgets. Their attacks could produce economical loses, physical damage, even possible explosions.
 
While isolated attacks have a small impact when put in the context of cyberwar, they can cause panic in populations, political crisis, or geopolitical problems if combined with other larger impact attacks.
Probably in a future post I will describe more of these kinds of large scale, big impact attacks.
INSIGHTS | October 17, 2013

Strike Two for the Emergency Alerting System and Vendor Openness

Back in July I posted a rant about my experiences reporting the DASDEC issues and the problems I had getting things fixed. Some months have passed and I thought it would be a good time to take a look at how the vulnerable systems have progressed since then.

Well, back then my biggest complaint was the lack of forthrightness in Monroe Electronics’ public reporting of the issues; they were treated as a marketing problem rather than a security one. The end result (at the time) was that there were more vulnerable systems available on the internet – not fewer – even though many of the deployed appliances had adopted the 2.0-2 patch.

What I didn’t know at the time was that the 2.0-2 patch wasn’t as effective as one would have hoped; in most cases bad and predictable credentials were left in place intentionally – as in I was informed that Monroe Electronics were “intentionally not removing the exposed key(s) out of concern for breaking things.”

In addition to not removing the exposed keys, it didn’t appear that anyone even tried to review or audit any other aspect of the DASDEC security before pushing the update out. If someone told you that you had a shared SSH key for root you might say… check the root password wasn’t the same for every box too right? Yeah… you’d think so wouldn’t you!

After discovering that most of the “patched” servers running 2.0-2 were still vulnerable to the exposed SSH key I decided to dig deeper in to the newly issued security patch and discovered another series of flaws which exposed more credentials (allowing unauthenticated alerts) along with a mixed bag of predictable and hardcoded keys and passwords. Oh, and that there are web accessible back-ups containing credentials.

Even new features introduced to the 2.0-2 version since I first looked at the technology appeared to contain a new batch of hardcoded (in their configuration) credentials.

Upon our last contact with CERT we were informed that ‘[t]hese findings are entering the realms of “not terribly serious” and “not something the vendor can practically do much about.”‘

Go team cyber-security!

So… on one hand we’ve had one zombie alert and a good hand-full of responsibly disclosed issues which began back in January 2013… on the other hand I’m not sure anything changed except for a few default passwords and some version numbers.

Let’s not forget that the EAS is a critical national infrastructure component designed to save lives in an emergency. Ten months on and the entire system appears more vulnerable than when we began pointing out the vulnerabilities.
INSIGHTS | June 11, 2013

Tools of the Trade – Incident Response, Part 1: Log Analysis

There was a time when I imagined I was James Bond zip lining into a compromised environment, equipped with all kinds of top-secret tools. I would wave my hands over the boxes needing investigation, use my forensics glasses to extract all malware samples, and beam them over to Miss Moneypenny (or “Q” for APT concerns) for analysis. I would produce the report from my top-notch armpit laser printer in minutes. I was a hero.
As wonderful as it sounds, this doesn’t ever happen in real life. Instead of sporting a classy tuxedo, we are usually knee deep in data… often without boots! I have recently given a few presentations(1) on Incident Response (IR). The question I am most often asked concerns the tool chain that would enable an individual or a team to perform the basic actions one would expect from an Incident Responder.

Let me be clear. There is an immense difference between walking on to a crime scene in which your organization’s intent is to bring a perpetrator to court and approximately 90% of the actual IR work you will perform. The biggest difference has to do with how you collect and handle evidence, and you can rest assured that any improperly handled evidence will be relentlessly attacked by the defense.
However, I am writing this blog post from the point of view of a blue team member. I’ll focus on the tools that are available to determine what has happened, when it happened, and (most importantly) how to make sure it does not happen again.
TL;DR : Many, if not all, of the tools you will need are available in the SANS   Investigate Forensics Toolkit (SIFT) VMWare image(2) . You can skip the rest of this post and start playing with these tools right away if you want to, but I hope I can at least give you some pointers on where to look first.
As I illustrated in the introduction, IR work is incredibly unsexy. Almost without exception, you will run into a special case that requires you to work on the command line for hours trying to figure out what has occurred. The next-next finish of many a security product installation and use process is something you will nostalgically reflect on over a burning log (pun intended).
You will typically have in your possession three important pieces of evidence:
  • Log files
  • Memory images
  • Disk images
I will address these in three separate blog posts and try to give you some ideas about the tools you can use during your investigation.
In this blog post, we will start off by looking at log files.
Every device on the network creates a gigantic amount of data. Some of it is used for monitoring purposes, but most of it is (hopefully) stored and never to be looked at again. Until a system is compromised, this data is generally useless with the exception of a few hidden golden threads. These threads will give you a good starting point for figuring out what has happened. They might also be useful in tying the whole story together.  Essentially, you must figure out what the log files want to tell you. (Cue the X-Files soundtrack.)
Sifting through log files is like looking for gold. Your initial sift will be very coarse grained. As you get rid of more and more of the dirt, the mesh of your sift will get finer and finer.
In my humble opinion, there is nothing better than having a thorough understanding of the Unix command-line tools ‘grep’, ‘sed’, and ‘awk’.  The good thing about these tools is that you will find them on almost any Unix or Linux box you can get your hands on. The bad thing is that you will either need to know or need to find out what you are looking for.
In the absence of a full-fledged SIEM infrastructure, two tools exist that I recommend you become familiar with:
  • OSSEC(3)
  • log2timeline(6)
First up, OSSEC. Not only is this a good (and open-source) host-based intrusion detection system, but it also allows you to feed it collected logs through its command-line tools!  The events will trigger OSSEC out-of-the-box alerts and give you a good overview of the most interesting events that have taken place. You can also easily develop custom rule sets based on (as an example) publicly available data from other compromises(4)
As you become more experienced with OSSEC, you can build your own set of scripts, drawing inspiration from a variety of places. In cases where ye olde *nix is not available, make sure you are also familiar with a Windows shell tool. (Powershell is one heck of an alternative!) Also, keep your Command Line Kung Fu(5) handy!
You will not be able to perform IR without the powerful yet extremely elegant log2timeline(6) tool. This blog post does not offer enough space to provide a decent overview of log2timeline. In essence, it groks data from different sources and creates a ‘Super Timeline’ that helps you to narrow in on those timeframes that are relevant to your investigation. It is one thing to dig through information on a command line, but it is a completely different ball game when you can visualize events as they have occurred.
In upcoming blog posts, I will dig into the juicy secrets that can be found in memory and disk images. A major lesson to be learned as you climb on the ladder of IR is that many of the tools you will use are produced by your peers. Sure, IR software is available and works well. However, the bulk of tools available to you have been developed by your fellow incident responders to address a particular problem. Many responders generously release their tools to the public domain for others to use. We learn from others probably more than we teach.
 

(1) BYO-IR: Build Your Own Incident Response: http://www.slideshare.net/wremes/isc2-byo-ir
(2) http://computer-forensics.sans.org/community/downloads
(3) http://www.ossec.net
(4) e.g. APT1 Indicators of Compromise @ http://intelreport.mandiant.com/
(5) http://blog.commandlinekungfu.com/
(6) http://log2timeline.net/

INSIGHTS | June 4, 2013

Industrial Device Firmware Can Reveal FTP Treasures!

Security professionals are becoming more aware of backdoors, security bugs, certificates, and similar bugs within ICS device firmware. I want to highlight another bug that is common in the firmware for critical industrial devices: the remote access provided by some vendors between their devices and ftp servers for troubleshooting or testing. In many cases this remote access could allow an attacker to compromise the device itself, the company the device belongs to, or even the entire vendor organization.
I discovered this vulnerability while tracking connectivity test functions within the firmware for an industrial device gateway. During my research I landed on a script with ftp server information that is transmitted in the clear:
The script tests connectivity and includes two subtests. The first is a simple ping to the host “8.8.8.8”. The second test uses an internal ftp server to download a test file from the vendor’s ftp server and to upload the results back to the vendor’s server. The key instructions use the wget and wput commands shown in the screen shot.
In this script, the wget function downloads a test file from the vendor’s ftp server. When the test is complete, the wput function inserts the serial number of the device into the file name before the file is uploaded back to the vendor.
This is probably done to ensure that file names identify unique devices and for traceability.
This practice actually involves two vulnerabilities:
  • Using the device serial number as part of a filename on a relatively accessible ftp server.
  • Hardcoding the ftp server credentials within the firmware.
This combination could enable an attacker to connect to the ftp server and obtain the devices serial numbers.
The risk increases for the devices I am investigating. These device serial numbers are also used by the vendor to generate default admin passwords. This knowledge and strategy could allow an attacker to build a database of admin passwords for all of this vendor’s devices.
Another security issue comes from a different script that points to the same ftp server, this time involving anonymous access.
This vendor uses anonymous access to upload statistics gathered from each device, probably for debugging purposes. These instructions are part of the function illustrated below.
As in the first script, the name of the zipped file that is sent back to the ftp server includes the serial number.
The script even prompts the user to add the company name to the file name.
An attacker with this information can easily build a database of admin passwords linked to the company that owns the device.

 

The third script, which also involves anonymous open access to the ftp server, gathers the device configuration and uploads it to the server. The only difference between this and the previous script is the naming function, which uses the configs- prefix instead of the stats- prefix.
The anonymous account can only write files to the ftp server. This prevents anonymous users from downloading configuration files. However, the server is running an old version of the ftp service, which is vulnerable to public exploits that could allow full access to the server.
A quick review shows that many common information security best practice rules are being broken:
  1. Naming conventions disclose device serial numbers and company names. In addition, these serial numbers are used to generate unique admin passwords for each device.
  2. Credentials for the vendor’s ftp server are hard coded within device firmware. This would allow anyone who can reverse engineer the firmware to access sensitive customer information such as device serial numbers.
  3. Anonymous write access to the vendor’s ftp server is enabled. This server contains sensitive customer information, which can expose device configuration data to an attacker from the public Internet. The ftp server also contains sensitive information about the vendor.
  4. Sensitive and critical data such as industrial device configuration files are transferred in clear text.
  5. A server containing sensitive customer data and running an older version of ftp that is vulnerable to a number of known exploits is accessible from the Internet.
  6. Using Clear text protocols to transfer sensitive information over internet
Based on this review we recommend some best practices to remediate the vulnerabilities described in this article:
  1. Use secure naming conventions that do not involve potentially sensitive information.
  2. Do not hard-code credentials into firmware (read previous blog post by Ruben Santamarta).
  3. Do not transfer sensitive data using clear text protocols. Use encrypted protocols to protect data transfers.
  4. Do not transfer sensitive data unless it is encrypted. Use high-level encryption to encrypt sensitive data before it is transferred.
  5. Do not expose sensitive customer information on public ftp servers that allow anonymous access.
  6. Enforce a strong patch policy. Servers and services must be patched and updated as needed to protect against known vulnerabilities.

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close