Blog

Give Silent Failures a Voice with Dead Man’s Snitch and PagerDuty

by Sam Lewis October 14, 2015 | 3 min read

dms-horizontal

Kiss Silent Failures Goodbye

Don’t let the hardboiled-sounding name of our latest integration scare you off, because this monitoring service is a great way to get notified when one of your mission-critical scheduled tasks suddenly sleeps with the fishes. Dead Man’s Snitch is an uptime-monitor for cron or periodic jobs like backups or batch processing, and it alerts you when your jobs don’t run so you can investigate before it becomes a problem.

Using Dead Man’s Snitch is dead easy: it only takes a few seconds to set up a job-specific alert, a.k.a. a “Snitch,” and then you have your job curl that Snitch’s unique endpoint every time it runs.

$ rake pgbackups:archive && curl https://nosnch.in/c2354d53d2

You can also pass optional messages which is good for reporting how long your job takes to run:

$ m=`time ( your_command &> /dev/null) 2>&1` && curl -d "m=$m" https://nosnch.in/c2354d53d2

If your Snitch doesn’t check in within the time period you set, Dead Man’s Snitch will send you an alert via email, push notification, or most recently, as a PagerDuty Incident so your on-call team members will know immediately if your scheduled jobs go missing.

pagerduty-dashboard

When your job starts reporting again, Dead Man’s Snitch will resolve the incident letting you know that everything is running smoothly.

“Quiet monitoring is great, but silent monitoring is terrifying. Dead Man’s Snitch gives us confidence that our monitoring systems are quiet for the right reasons.”

— Harry Marr, CTO, GoCardless.com

Integration Details

Integrating your Dead Man’s Snitch and PagerDuty accounts is easy and should take less than a minute.

  1. Log in to your Dead Man’s Snitch account and click your avatar to get to your account settings.
  2. In the “Integrations” tab, click “ADD” next to PagerDuty.

dms-pagerduty-integration-1

3. You’ll be taken to pagerduty.com where you’ll authorize Dead Man’s Snitch by entering your PagerDuty email and password.

dms-pagerduty-integration-2

4. Lastly, create a new service for Dead Man’s Snitch and you’re good to go.

dms-pagerduty-integration-3

5. Dead Man’s Snitch will now create and manage incidents when your snitches go missing.

Features

Dead Man’s Snitch offers an easy-to-use REST API and support for email check-ins, giving you multiple ways to keep an eye on your Snitches from any platform. Free iPhone app are also available and the service’s brand new Slack integration means that you can get updates on the go and right in your team’s Slack channel.

Dead Man’s Snitch is a product of Collective Idea, a software development consultancy based in Holland, Michigan.

Create your Dead Man’s Snitch account today and start kissing silent failures goodbye.

Monitoring_Ebook_728_90