PagerDuty Hacks: Alert by Foam Dart Bombardment
by Peter Kennedy
February 10, 2016
| 3 min read
The Roman Pillar of Justice
Hackday is my favorite day of the month. Yet, it’s becoming increasingly difficult to win as PagerDuty hires more employees. It’s no longer a guaranteed winning strategy to present something technical or useful, so my approach has shifted. I’ve been building integrations for PagerDuty that are technically challenging, but also incredibly silly.
The Goal
PagerDuty integrates with loads of tools, but what functionality is it missing? Bombardment. No one, to my knowledge, has wired PagerDuty up to a turret that will fire foam darts at the person on call whenever there is a PagerDuty alert.
To add a layer of shame to it, the turret also records a video of the assault with its on-board webcam.
How It Works
To make things easy, we purchased a USB turret from Dream Cheeky. The problem was that this device had no API exposed and was only operated from an OS X application. After some research and experimentation, the device could be controlled by sending the appropriate payloads with libusb. We plugged it in and ran it on a Raspberry Pi sitting in the real-time engineering room of our San Francisco office.
We put an application in Heroku to receive PagerDuty alerts, with targets stored in a database of each engineer sitting near the turret and the relative coordinates of their desk. Webhooks from PagerDuty would mark the on-call as a target. The Raspberry Pi queries this data and fires a dart at the target on a regular interval.
The Heroku application made it easy for other integrations to be added. Slack was included so that the turret could be operated, annoyingly, by anyone at PagerDuty, even the Toronto office. The turret was intimidating, I decided to bring it to life with a little branding. The Slack integration was updated with a photo of my manager, Roman. His photo was also put on the front of the turret. The project became known as “The Roman Pillar of Justice”.
A camera was taped to the side of the turret so that videos could be recorded of each incident. These would then be uploaded to Slack as gifs.
When Justice Goes Awry
There was a brief moment while building this where Slack sent a few duplicate webhooks, and the results were awesome. Ashwin, an engineer on the real-time team, was shot by the turret multiple times for about a minute. The Slack output from this can be seen below:
Foam Dart Everyone
Want to build your own Pillar of Justice? Check out the code on Github.
