Blog

Psychological Theories Behind Agile Software Development

by John Coleman October 10, 2018 | 8 min read

Reflective Practice at PagerDuty

With an almost 20-year career in social services—including working in institutions such as The University of Chicago, the United States Peace Corps, Chicago public schools, Child Protective Services, the Catholic Church, and for the U.S. federal government in probation and parole programs—my leap into Silicon Valley was as much a culture shock as the 2.5 years I lived in the Andes Mountains of Ecuador. I’ve found that some of the most comforting experiences that helped ease my transition were in my team’s retrospective meetings and the individual support for from veteran engineers leading PagerDuty’s teams. This blog is an attempt to draw from my clinical psychological background to make sense of the psychological theories of practice driving the success of PagerDuty’s reflective practices at PagerDuty.

Agile Software Development at PagerDuty

Upon joining PagerDuty, I was immediately impressed by its implementation of Agile software development, the DevOps culture, the “you build it you own it” philosophy, and other diverse experiences necessary for companies and individual engineers to grow in today’s software market.

Since PagerDuty’s product brings tools and value to help with continuous delivery of highly available software across the digital operations market, we are interested in these types of operations for our own tech stack. To learn more about that, see this blog from our SVP of Engineering, Tim Armandpour, where he writes that the “customer is at the center of everything. We have a maniacal focus on gaining a deep understanding of the problems our customers face and creating solutions that just work. We want to continue to learn faster than ever before.” This syncs well with the the first principle of the Agile Manifesto: “Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”

Yet while we as a company hold that first principle near and dear, and crafted an extensive postmortem process designed to promote learning and reflection so we can better serve our customers, I actually was first drawn to PagerDuty’s implementation of the Agile Manifesto’s last principle: “At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.” At PagerDuty, we call these retrospective meetings.

In these meetings, I found my engineering team’s implementations to be outstandingly impressive because, even with my extensive background working at a micro level with individuals and groups, I had never before witnessed such an effective group process in a team of coworkers where the focus of the intervention is their own work quality. Our retrospective meetings embraced social and psychological theories of best practices so well that I was convinced these meetings could help engineers, particularly those who find social interactions challenging, become exemplary leaders with interpersonal soft skills.

Psychotherapeutic Theories of Practice

My team at PagerDuty is newly formed, with a relatively large number of engineers; therefore, we have evolved our reflective practices to occur in biweekly 90-minute retrospective meetings, in addition to quarterly health checks. Since most readers are likely already familiar with retrospective meetings, I decided to focus on the psychological theories underlying reflective practices at PagerDuty rather than how retrospectives are practically implemented. The main theories of practice that I will focus on are psychoanalysis, vulnerability, restorative justice, and systems theory.

Psychoanalysis

At the root of all psycho-therapeutic practice is some type of introspection that both the client and therapist together engage in understanding. Most notably, psychoanalysis, which became more well known due to contributions from Melanie Klein, Carl Jung, and Sigmund Freud, attempts to discover and interpret past, unconscious, or unknown thoughts, emotions, impulses, and memories that currently impact the client’s thoughts, feelings, behaviors, and entire life.

Similar to psychoanalysis, PagerDuty’s retrospective practices reflect on the past thoughts and behaviors of the individual and the team in order to make all group members aware of the things that perhaps only certain individuals were cognizant of. In Freud’s work, free association allows individuals to express whatever comes to mind, without censorship or even organization. Similarly, PagerDuty retrospectives allow for free and uncensored verbal and written communication by making only certain information public and keeping managers from attending meetings. Another example of psychoanalytic models is that PagerDuty has an Employee Resource Group (ERG) whose primary focus is to train retrospective facilitators and improve retrospective practices. This ERG, called RetroDuty, uses facilitator best practices to remain neutral and ask questions designed to invoke team participation, which are major components of psychoanalytic theory as well.

Vulnerability

With the recent popularity of TED Talks, many have already been introduced to Dr. Brené Brown’s research on vulnerability. Dr. Brown discusses the power of connection in the human experience across all spectrums and how people with the strongest sense of connection believe that they are worthy of healthy human connection, which increases the courage to be authentic and vulnerable through risking rejection from others.

Dr. Brown learned that vulnerability is the birthplace of joy, creativity, belonging, and love. In software development, reflective practice has the potential to cultivate a group that embraces, respects, and supports each member’s vulnerability. According to Dr. Brown’s theories, a team will become more bonded, productive, and creative when reflective practice supports members working through feelings of vulnerability, such as the uncomfortable feelings that come along with being dismissed, disrespected, not considered, or facing difficult challenges that impact technical output, job performance, and productivity. She also discusses how American society uses unhealthy amounts of certainty, blame, and perfection to numb shame and vulnerability. However, at PagerDuty, our reflective practices embrace and accept, in healthy ways, uncertainty, group responsibility, and failure.

Restorative Justice

Restorative justice models of practice focus on both the offender and the victim (and sometimes their respective communities) to come to an agreement about the response to violations that will restore balance in all parties involved. For more on restorative justice theories, you can read this comprehensive review of research on restorative justice models in schools, published by the The WestEd Justice & Prevention Research Center.

While our engineering team has not dealt with criminal activity—which is one of the main use cases for restorative justice models—we still exhibit some components of restorative justice theories when we process events that caused disturbances in team productivity or emotional well being. We do this by planning actionable items that all members support and by holding ourselves accountable to follow through on these items in our upcoming retrospective meetings. By focusing on recurring events or events with the most impactful and widespread consequences, we prioritize inclusivity of opinions of all team members, which is also imperative for social justice models.

Systems Theory

Systems theory, often considered a cornerstone of social work, suggests that a person’s psychology and behavior often affect and are affected by the individual’s family, community, work, and other societal systems. Being from Chicago, a city with a population consisting of roughly 30 percent African-American and 30 percent Hispanic people, I was taught how the white majority that I am a part of implements societal and systemic factors that impact minority lives.

To help understand systems theory, consider Dr. Deborah Prothrow-Stith’s research on violence that parallels Alexander Hamilton’s death in a duel with current black urban violence. She discusses how all the factors that reformed white men’s dispute resolutions (such as wealth, legal counsel, alternative means to power, and criminal justice interventions) were entirely unavailable to urban black youth. In fact, the net asset difference between white and black families has increased since the Civil War—despite what most perceive as advancements in equality and civil rights, darker-skinned groups have relatively less wealth compared to their white counterparts.

While I’m not commenting on diversity at PagerDuty, these observations are important because they provide systemic context to individual behavior, which is primarily attributed to the autonomy of an individual. Since reflective practices at PagerDuty provide weekly opportunities to improve team behavior, it has the potential to improve individual productivity and mental health by specifically impacting the system that individuals partake in. When individuals are often blamed for their behaviors, retrospective meetings acknowledge that teams influence individual behavior and that the entire team takes responsibility for the system and individual experiences.

Ethical Standards

James Tyack, engineering manager, emphasizes PagerDuty’s intention to prioritize People First in his blog post on 6 reasons why our engineers stand out. Similarly, the two most primary ethical principles behind social work (and enforced by governing bodies) are commitments to the client:

(1) Social workers’ primary responsibility is to promote the well-being of clients. In general, clients’ interests are primary, and (2) Social workers respect and promote the right of clients to self-determination and assist clients in their efforts to identify and clarify their goals.
– Code of Ethics, National Association of Social Workers

With these principles behind our reflective practices, PagerDuty has developed a retrospective and health check comparable to the best psychological interventions that I have seen in my career as a clinical social worker. My hope is that PagerDuty will implement reflective practices in other departments and, eventually, add supportive tools to our product to help other companies use the framework to mimic the success of our own reflective practices.