The Proliferation of Monitoring Systems

Monitoring systems have been a great boon to digital business. If you’re like most IT Operations teams, you’ve invested in several to detect slow API requests, network issues, overtaxed servers, and more. But… also like most IT Operations teams, you might still be facing challenges when it comes to incident response. In fact, according to a survey conducted by Dimensional Research and commissioned by PagerDuty, 85% of teams report missing a critical incident, and 99% say that missing alerts has the potential to hurt their business. After all, in a world where being slow is the new downtime, and downtime impacts your customer experience, missed alerts are a serious matter.

But it’s not all about monitoring. The truth is, there’s a world where monitoring stops and incident response kicks in.

Alerts Reach Epidemic Levels

With the proliferation of increasingly specialized monitoring software, there now exists an ever-increasing number of tools that are really great at detecting abnormalities. 91% of operations teams use more than one tool. These tools are generating hundreds if not thousands of events per day, and unfortunately, only 27% of teams are doing anything to aggregate and filter that data before it reaches people. The consequences? 43% of teams report receiving too many alerts. That means that team members are overburdened, and in danger of burnout and serious alert fatigue. In the worst case, your team is  so flooded they can’t tell what’s critical from what’s informational, and they miss the one notification that really matters. The result could be a proverbial fire that ignites suddenly and quickly consumes everything in its path.

The World Of Incident Response…

… is sadly manual and inefficient. Email is still used by 79% of teams for notification, even though it can often be too slow for high-stakes incident response. Additionally, email provides little to no accountability or visibility, nor does it let users automatically track incoming alerts. There’s no easy way to assign it an alert to someone else from an email, so escalations happen manually, if at all.  There also aren’t any analytics to be garnered from email alerts, so you can’t truly visualize the health of your system, nor can you get a sense of where an big issue might be lurking in the future.  We’ve blogged about this before. Too many IT teams rely on managing on-call schedules in Excel sheets, which wastes valuable minutes getting in touch. No wonder 54% of IT teams are dissatisfied with their incident response.

Don’t Stop At Detection

So what can you do about it? Monitoring systems are great for detecting problems, but that’s just the start of the process. Plan for your incident response. Avoid common pitfalls by putting best practices in place. Continually tweak your incident management process to make sure it’s working for your team. And don’t underestimate the importance of analytics: often the first step to understanding where you can improve your incident response is by using data to find pain points and strategize on next steps.

Want to learn more? Download our eBook to see how to make your monitoring systems really deliver on their promise of protecting your uptime