Pen Testing to Protect the Water Supply

As part of the nation’s critical infrastructure, water utility companies must implement a comprehensive cybersecurity program to protect the water supply and the millions of people who rely on it every day. To help utilities coordinate security planning with federal, state and local governments, in 2013, President Barack Obama issued Presidential Directive 21 (PD21). PD21’s goal is to “reduce vulnerabilities, minimize consequences, identify and disrupt threats, and hasten response and recover efforts related to critical infrastructure.”

Since that time, the threat has continued to grow. According to the U.S. Department of Homeland Security’s Cybersecurity & Infrastructure Security Agency, U.S. water utilities now face threats from spearfishing and ransomware attacks, unsecured remote devices, and outdated software, control system devices, and operating systems.

For example, in 2023, an Iranian-backed group attacked the Municipal Water Authority of Aliquippa, Pennsylvania. The attack targeted a “booster station” that monitors and controls water in two Pennsylvania townships. Fortunately, the utility had an alarm system that alerted operators so they could prevent disruption to the water supply.

A few months later, a Russian military unit called Sandworm attacked utilities serving several small Texas towns. The effects appear to have been minimal, but Sandworm had previously caused blackouts in Ukraine and staged other attacks that one journalist called “some of the most disruptive hacking events in history.”

Around the same time, National Security Advisor Jake Sullivan and the head of the U.S. Environmental Protection Agency, Mike Regan, alerted state governors that Chinese actors are “pre-positioning” to attack the U.S. water system if political tensions between the two nations rise.

However, before the most recent threats, our client had asked us to perform a penetration test (pen test) of their physical and cyber infrastructure and identify any vulnerabilities. They would then share what they learned with government agencies to comply with PD21. We were proud to complete the project and play a role in protecting the water that our client provides to various county governments and municipalities for distribution to millions of people.

Enter Centric: Finding Vulnerabilities Without Disrupting Operations

Our engagement began with a SCADA risk assessment. For utilities, SCADA systems are the operational technology (OT) systems that control, monitor and analyze companies’ processes and devices.

For example, at a water utility, SCADA digitizes and automates data collection for things such as flow rates, chemical levels, and more. Because SCADA collects data in real time, human or computer operators can adjust equipment settings and operations immediately rather than waiting for data to be analyzed and delivered. SCADA systems can also include other security components — helpful for utilities with dozens of remote sites, like water-processing facilities.

Because SCADA networks are extremely sensitive, we structured an internal technical vulnerability assessment that would not disrupt the water supply. In this type of passive testing, we only gather information through network traffic captures and look for vulnerabilities in the normal flow of data.

In our client’s SCADA networks, we used Wireshark, an open-source network analysis tool, to capture network traffic at different times of the day and the week. We then gathered all the data we could access. We took it offline to identify vulnerabilities, such as open but unused ports and unencrypted (cleartext) passwords, protocols, packets, or other data. The report we submitted to our client listed these vulnerabilities by criticality, helping them prioritize their repairs.

With the SCADA vulnerabilities identified, we moved into the external pen testing portion of our analysis. Modern network security devices such as firewalls and intrusion prevention and detection systems do a good job of keeping threat actors out when configured correctly. During the penetration test, our team discovered modems that were used to monitor remote water reservoirs.

The client’s original target listing did not include these modems, but our team identified them as organizational assets during their open-source intelligence gathering. The team used the modems to perform reconnaissance and gain additional information about the network. We also actively tested internal and external attacks on the utility’s internal, non-SCADA networks, acting like threat actors who have bypassed security and gained access to the internal networks.

In our pen testing, our team of ethical hackers discovered many externally facing vulnerabilities. They then gained unauthenticated access to the utility’s devices, and through the devices, they broke into the internal network itself.

In addition, we physically inspected some of the clients’ remote water-processing plants. Often, equipment at remote sites is not updated as regularly as equipment in a central office, and they may contain non-standard Windows devices or network switches. Water facilities also have other remote network connections, like Wi-Fi boosters, cellular signals, satellites, and other peripheral connectivity devices hackers can exploit.

The Result: Removing Risks and Preparing for the Future

During our engagement, we analyzed the SCADA network and actively pen tested against 50 external IPs and three class B internal networks. While our assessment helped the utility comply with PD21, the long-term benefits to the company and its customers exceed compliance. Daily port scans, monthly external network penetration tests, and monthly internal technical vulnerability assessments helped harden the utility against cyberattacks, enabling it to continue providing safe, reliable water to its customers.

Conclusion

Utilities and other critical infrastructure organizations are leading a transformation in how organizations think about cybersecurity. In the old model, cybersecurity was a cost center that did not seem to deliver real value. Now, companies are beginning to recognize it as a necessary investment in the future. No one knows how tomorrow’s threats will evolve, but vulnerability assessments and pen testing are valuable tools for hardening critical systems for the future.