Have you ever felt you took every wrong turn possible in the process of mitigating a production incident? Did you go through a 3-hour hell during incident response and felt the incident wasn’t complex enough to justify the horrors you’ve experienced? Did it cause you to question your engineering or problem-solving skills?

Well, it’s only partially you. Our brain is wired to make decision-making simpler. In doing so, it exposes itself to biases, heuristics, and other quirks that may seem like “bad decisions” in hindsight.

In this talk, through real-life outages, we’ll project those psychological principles onto the world of production monitor, and incident management. As a responder, you’ll learn why those behavioral patterns emerge during production incidents and what can be done to limit their effect, and as a manager, you’ll learn how to enable and encourage a healthy environment to better support those patterns.

A runbook is a combination of documentation and automation enabling teams to codify processes.