Monit Integration Guide

Monit is a small Open Source program for managing and monitoring Unix systems and services. Monit is relatively easy to install and configure while still being flexible for advanced users. Monit can be integrated with PagerDuty via both the generic email and the API integrations however, this guide will cover using the API. The email integration is explained below in the FAQ. If you run into any problems with the integration, please feel free to contact us.

In PagerDuty

1. From the Configuration menu, select Services. 
2. On your Services page: If you are creating a new service for your integration, click +Add New Service. If you are adding your integration to an existing service, click the name of the service you want to add the integration to. Then click the Integrations tab and click the +New Integration button.

3. Select your app from the Integration Type menu and enter an Integration Name. If you are creating a new service for your integration, in General Settings, enter a Name for your new service. Then, in Incident Settings, specify the Escalation Policy, Notification Urgency, and Incident Behavior for your new service.
4. Click the Add Service or Add Integration button to save your new integration. You will be redirected to the Integrations page for your service.
5. Copy the Integration Key for your new integration:

In Monit

Install the PagerDuty Python library: Open a terminal on the server and run the command pip install pagerduty to automatically download and install the Python library. Install the PagerDuty-Monit scripts:

1. Open a terminal and navigate to either the system-wide monit configuration (typically '/etc/monit/') or the user's home directory that will be running Monit
2. Download the scripts with the following commands

   wget https://raw.githubusercontent.com/pinterest/pagerduty-monit/master/pagerduty-trigger
   wget https://raw.githubusercontent.com/pinterest/pagerduty-monit/master/pagerduty-resolve

3. Edit both scripts so that the PAGERDUTY_SERVICE_KEY variable is both uncommented and set to your Service API Key from earlier.
4. Make sure both scripts are executable: `chmod +x pagerduty-trigger; chmod +x pagerduty-resolve`.
5. Add `exec "/etc/monit/pagerduty-trigger nginx"` to a Monit check stanza as an action. For example:

   check process crond with pidfile /var/run/crond.pid
       if does not exist for 3 cycles
           then exec "/etc/monit/pagerduty-trigger crond"
       else if succeeded for 3 cycles
           then exec "/etc/monit/pagerduty-resolve crond"

6. Restart or reload Monit: /etc/init.d/monit restart or monit reload.

Test the integration: Depending on what you are monitoring for and how you've configured Monit to react to the event, you may be able to simply turn off the service to trigger an incident. In the above configuration example `/etc/init.d/crond stop` will trigger an incident if the service is down for 3 consecutive checks.

FAQ

Can I set up Monit to use multiple PagerDuty services?

You certainly can, you'd just need to have another copy of the 'pagerduty-trigger' and 'pagerduty-resolve' files with different names and service keys in each one. From there, you'd just modify the check stanza so it executes the correct script.

If something is resolved in Monit, will it automatically resolve in PagerDuty?

Yes! Issues resolved in Monit will update PagerDuty however, resolving things in PagerDuty does not fix them within Monit.

I'd like to use the email integration instead. How can I configure this?

The configuration settings for using a mail server with Monit can be found here. You'd need to have a mailserver such as Exim or Sendmail installed to send the email out.

I'm not receiving PagerDuty alerts from Monit. Why do I receive the error "/usr/local/bin/pagerduty: Is a directory" when attempting to manually trigger an incident?

The PagerDuty Python library was likely installed at a different location on your system, such as /usr/bin/pagerduty. You can determine the correct path by running the command which pagerduty: Simply run a find and replace in the pagerduty-trigger and pagerduty-resolve files to update /usr/local/bin/pagerduty with the location of the PagerDuty Python library on your system.