Maps, Context, and Tribal Knowledge: On the Structure and Use of Post-Incident Analysis Artifacts in Software Development and Operations

Abstract: The role, functions, and mechanisms of post-incident and post-accident learning within organizations has been widely studied in various industries. With the increasing role software plays in society, large-scale software development and operations practices and organizations are having a larger impact on day-to-day life; despite this, organizational learning has not been widely examined within this context. This research involves a two-pronged methodology to investigate organizational learning within, specifically, the software development and operations industry: first, an industry survey collected broad industry data regarding post-incident analysis language, usage, and template artifacts; second, this survey served as a basis for a deep organizational case-study involving three distinct teams at a well-known Internet software development and operations company. The survey identifies some post-incident analysis behavioral and artifact-usage trends, including three specific archetypes of post-incident analysis templates: the ‘Record-Keeper,’ the ‘Facilitator,’ and the ‘Sign-Post.’ The organizational case study examines the specific mechanisms of post-incident organizational learning, including use of template artifacts to map the company’s complex, emergent socio-technical system, share intra- and inter-team context in lieu of static lists of incident remediation items, and use post-incident artifacts to curate organizational culture and transmit tribal knowledge between organizational actors. Other uses are also identified. Together, these uses provide a set of existence proofs for various methods and mechanisms for how a high-performing software development and operations organization uses, at least in part, its post-incident artifacts to aid in and sustain organizational learning.