I Spy With My Little Eye…

“I would like to build out the ability to track who has clicked on the link.”

When I sent that email to the Sales team, I realized I sounded like a creepy stalker. Why on earth would Security want to track everything that is being clicked? I decided more context was necessary to help dispel any nasty rumors. We had just detected a real threat, a phishing attack. We detected the attack because a member of our Sales team was vigilant and reported it to the Security team.

As I wrote a follow-up message to explain my thinking, I realized the whole company could benefit if I peeled back the curtain and revealed “how security works.” It is an area ripe for expansion and disruption. By understanding the needs of a security incident responder, we can help our customers solve new problems using PagerDuty and broaden the impact we have for our customers.

How Security Incident Response Works

Security teams generally have two primary objectives: reduce cybersecurity risk to the business and improve customer trust. A very real risk we face every day is that an attacker could execute malware on any computer at any time. A security incident response team goes through a process to understand and mitigate the impact of each threat we face. The high-level steps we follow here at PagerDuty are based on the NIST Cyber Security Framework and outlined in our Security Incident Response plan:

• Detect
• Contain
• Respond
• Recover

Now, join me as I walk you through how the PagerDuty Security team responds to threats like these.

The immediate questions we want to answer are:

• Does the attack contain a malicious payload?
• Did the malicious payload detonate anywhere?

Answering these two questions allows us to understand the initial impact of the attack and properly contain the damage.

The answer to question #1 was straightforward since we received the attack report from our colleague. We had the email with the link, and we could follow the link to look at the payload to see if it was malicious. In such cases, to be safe, we use an isolated computer with a virtual machine designed for inspecting suspicious files and follow the link from inside that virtual machine. If the link downloads something bad, we can immediately suspend the virtual machine. The malware won’t be able do any damage.

Using this method, if we detect that a link installs malware, we would block the link so no one on the office network can download it. But what if someone had already downloaded it before we put the block in place?

This brings us to question #2—did the malicious payload detonate anywhere? To answer this question, we need to find out if anyone had downloaded the malware; i.e., who has clicked the link? If a few people have downloaded the malware, we need to immediately remove their computers from the network so the malware cannot communicate with any other systems. Malware can attack other computers on the network, and it can steal data from your computer and send it over the Internet to the attacker. These actions are called “lateral movement” and “exfiltration,” respectively.

After we remove the infected computers from the network, we inspect them to see if the malware was able to execute. Note that we cut off the attack vector to contain the damage before we actually respond to the problem. We want to stop the infection from spreading as quickly as possible. Sometimes you get lucky. Sometimes the malware only runs on Windows or only runs on a Mac so if you downloaded it onto the system it doesn’t run on, you aren’t affected.

Now let’s look at a scenario where we found out that three people clicked the link, and the malicious software was able to execute because it targeted the right kind of computer. This is where things get really dicey. We need to understand what the malware did while it was running, before the three computers were disconnected from the network. Did it exfiltrate any data over the Internet? Was it able to use lateral movement to attack another computer? Did it install a keylogger to steal passwords? The answers to these questions and to others determine how we respond and what we must do to recover from the attack.

Unfortunately, we do not have technology today that allows us to quickly determine whether anyone clicked on a link and downloaded malware. I always hope no one does this, but I would rest better at night if I could tell you with certainty that none of our systems were impacted—hence my email about wanting to track who has clicked on a link.

PagerDuty for Security

I’ll end with a challenge for you. I’ve told you how important it is for security incident response teams to respond quickly to contain the impact when a security event is detected. I’ve told you reducing risk is the reason we do our jobs and earn our paychecks. My challenge for you is to answer the following questions: How can security incident responders use PagerDuty to reduce the time it takes to respond? How much risk could they eliminate for their employers if they were able to contain the infection before it spread through lateral movement? How much money would that save?

Stay tuned for more on PagerDuty for Security! In the meantime, check out our Security resources:

• Disrupting Security: Revamping IT Security Through SecOps and Incident Management
• The Secure Developer: Keeping PagerDuty Secure
• Signal Sciences Addresses Security Anomalies Quickly, Keeping Customer Data Safe With PagerDuty