Chapter 11: Failure-Aware Engineering

Download this chapter (PDF) ↓

Failure is central to engineering.

— Henry Petroski (Petroski 1992)

Premise. Engineering judgment develops when engineers connect decisions to their consequences and allow experience to change future decisions.

Engineering is practiced with incomplete knowledge, imperfect models, fallible people, and finite evidence. The techniques in this handbook are intended to make those imperfections apparent, so that you engineer with full awareness of the hazards. Failure is therefore a predictable engineering outcome; the question is what the engineer learns when reality violates expectations.

Throughout this book, we have used models to make engineering decisions explicit. Requirements describe outcomes that matter in the world (Requirements). Specifications describe obligations of the machine and assumptions about its environment (Specification). Architecture and design describe how those obligations will be realized (Architecture, Design). Validation connects claims about the system to evidence (Validation). Metrics connect properties we care about to observations we can make.

None of these activities makes engineering infallible. Requirements can omit something important. Specifications can misrepresent the environment. Architectural decisions can create interactions nobody anticipated. Designs and implementations can contain mistakes. Validation provides bounded evidence rather than certainty. A system can therefore fail despite substantial and responsible engineering effort.

Failure creates new evidence. The system has encountered circumstances under which some expectation did not hold. Engineers can repair the immediate problem without learning much from that evidence, or use the discrepancy to reconsider the understanding that produced the system. Failure-aware engineering concerns the latter activity. It treats failure as part of a continuing engineering process and asks what failed, why previous engineering did not reveal the problem, what the failure teaches about the system and its environment, and what should change as a result. Those changes can occur in the engineered system, in the knowledge and practices of the team, and in the judgment of the individual engineer.

Experience builds judgment

The preceding chapters have taught one important form of engineering judgment: deliberate reasoning about consequential decisions. Engineers identify the decision they face, construct appropriate models, determine which properties matter, compare alternatives, gather evidence, quantify relevant consequences where useful, and decide despite residual uncertainty.

Experienced engineers also exercise judgment through recognition. They notice that a seemingly harmless assumption deserves investigation, recognize that two different systems share a troublesome structure, or become suspicious of a result that satisfies the stated threshold but conflicts with what they have seen elsewhere. The engineer may recognize which question needs to be asked before knowing its answer.

This recognition develops from experience, but experience is not simply elapsed time. Engineers encounter decisions and observe their consequences. Reflection connects the two: what did I expect, what occurred, and what does the difference imply for a future situation? The resulting lesson becomes part of the engineer’s repertoire of situations, interpretations, and possible responses.

Donald Schön describes professional practice in terms of this relationship between action and reflection (Schön 1983). Professionals can reflect while acting, revising their understanding as a situation develops, and they can reflect afterward on what the experience revealed. In both cases, practice produces more than an immediate outcome. Properly interpreted, it changes the resources available for future judgment (Figure 7).

Figure 7. Experience becomes judgment through reflection. A decision produces consequences; reflection determines what those consequences reveal; the resulting lesson enters a repertoire the engineer can recognize a situation against later.

The relationship is reciprocal. Deliberate reasoning produces decisions whose consequences create experience. Reflection can turn that experience into patterns an engineer later recognizes. Recognition, in turn, helps the engineer identify where deliberate reasoning is needed. Mature judgment therefore depends on both the ability to reason about a known problem and the ability to recognize what kind of problem one is facing.

Learning from success

Successful engineering contributes to this repertoire. A system that repeatedly produces the intended consequences under varied conditions supplies evidence about the engineering decisions that produced it. Engineers can learn which architectures tolerate growth, which interfaces remain stable under change, which validation strategies reveal useful information, and which development practices help teams make sound decisions.

Success can nevertheless be difficult to interpret. A system may operate successfully because its architecture is robust, but it may also succeed because workloads remain forgiving, an incorrect environmental assumption happens to hold, a latent defect is never exercised, or several weaknesses never align. Observing success does not by itself distinguish among these explanations.

This is the same evidentiary problem encountered in Validation. An observation becomes evidence for a claim only through a model connecting the two. Successful operation under one set of conditions is evidence about those conditions; broader conclusions require an argument about how far the observation generalizes.

Failure often provides a sharper discrepancy. Something engineers expected to hold did not.

Failure exposes a model

Suppose a service was designed under the assumption that an upstream dependency would respond within two seconds. Months of normal operation may be consistent with that assumption without establishing it as a reliable bound. An incident in which the dependency repeatedly takes thirty seconds provides different information: the assumed bound does not describe at least some conditions the deployed system can encounter.

The immediate engineering task may be straightforward. Engineers can introduce a timeout, change retry behavior, isolate the dependency, or otherwise restore acceptable service. The incident also creates a question about the understanding under which the system was engineered. Why was the two-second assumption considered credible? Where was it represented? Which decisions depended on it? What evidence supported it? Did engineers believe the dependency guaranteed the bound, or did the assumption simply become embedded in the system without ever being made explicit?

Figure 8. An engineering model produces an expectation the world can contradict. When it does, the observation becomes evidence about the model, but only after engineers interpret what the discrepancy means.

The interpretation step matters (Figure 8). A failure does not explain itself. The first explanation engineers construct may be wrong, and the same observation can be consistent with several causal accounts. A production outage following a deployment may have been caused by the deployment, exposed by it, or merely coincident with it. Reflection therefore requires the same discipline applied to other engineering evidence: identify what was observed, distinguish observation from interpretation, consider plausible alternatives, and determine how broadly the conclusion is justified.

The distinction between repair and learning follows from this model. Repair changes the immediate system so that it again satisfies its obligations. Learning changes the understanding used in future engineering. Many incidents require both, but accomplishing the first does not guarantee the second.

Experience does not guarantee learning

Engineering organizations experience failures continually. Whether those experiences accumulate into better engineering depends on what happens afterward.

A study of software-failure learning at a national space research center found that engineers did learn from failures, but much of the learning was informal and individually driven (Anandayuvaraj et al. 2026). Reflection was not necessarily a routine organizational activity. Knowledge could remain tacit, become fragmented across documentation systems, or depend on experienced engineers remembering an earlier problem when a related situation arose. Practitioners also described related problems recurring across projects and phases. The organization could successfully repair individual problems without reliably converting each experience into durable knowledge available to future engineering decisions.

That finding illustrates a more general distinction. An event can produce substantial experience for the people directly involved while producing little persistent change in the engineering organization. An engineer may remember the incident but later leave the team. A postmortem may record what happened but be difficult for future engineers to find. A corrective change may eliminate one observed manifestation while leaving the underlying condition available elsewhere. Engineering Knowledge described this movement as the difference between what an individual knows and what the organization retains, represents, and can act upon.

The converse is also important. Engineers need not personally experience every failure from which they learn. Case studies, postmortems, design reviews, conversations with experienced colleagues, and preserved engineering knowledge allow one person’s experience to contribute to another person’s repertoire. Engineering disciplines accumulate partly by making consequential experiences available beyond the people who originally encountered them.

Failure-aware engineering therefore asks two related questions. What does this failure teach about the engineered system? How should that lesson affect engineering after the immediate participants and circumstances are gone?

Before answering them, engineers need to determine what kind of failure occurred.

What failed?

Software failure is often discussed as though it were synonymous with an implementation defect. That is the familiar case: the realized software does something its design did not intend. An incorrect bounds check permits an out-of-bounds access; a mistaken condition sends execution down the wrong path.

But software can fail even when every executed instruction behaves as its implementer intended. The engineering activities developed in this handbook provide a sequence of progressively broader questions about what an incident has revealed (Table 7).

Table 7. Locating what an incident revealed among the engineering activities.

Engineering activity	A failure may reveal that…	Example
Implementation	The realized software departs from an otherwise adequate design.	Incorrect bounds handling permits an out-of-bounds memory access.
Design	A selected mechanism has consequences inconsistent with the properties it must satisfy.	A mechanism is functionally correct but cannot satisfy required latency under realistic load.
Architecture	Responsibilities, boundaries, resources, or interactions make an important system property fragile or impossible to preserve.	A critical function shares a resource whose exhaustion by another workload prevents the critical function from operating.
Specification	The required relationship between machine and environment was represented incorrectly or incompletely.	The machine behaves as specified, but an assumed environmental condition does not hold in deployment.
Requirements	The engineering effort pursued the wrong world outcome or omitted an important obligation.	A system performs its stated functions but fails an important stakeholder need.

Implementation is the most familiar case: the realized artifact simply does something the design did not intend. Design failures are subtler. The implementation may faithfully realize a selected mechanism, but that mechanism may consume unacceptable time, memory, energy, or other resources.

Architecture widens the analysis further. Properties such as availability, isolation, and throughput can depend on interactions among components that behave correctly in isolation. A failure can therefore arise from how responsibilities, boundaries, resources, and interactions were organized rather than from incorrect behavior within any one component.

Specification and requirements expose the least familiar possibilities. A machine can faithfully implement its specification while the specification incorrectly represents the environment with which it interacts. More fundamentally, software can correctly implement both its design and specification while producing the wrong outcome because engineers committed to the wrong requirement or omitted an outcome that mattered.

These categories are analytical aids rather than mutually exclusive classifications. One incident can expose weaknesses at several levels. A requirements decision may create an unusually difficult assurance burden; an architecture may make that burden harder to satisfy; an implementation defect may provide the immediate trigger. The purpose of locating the failure is not to assign it to a single box. It is to determine which engineering decisions deserve reconsideration.

That analysis also leaves another question unanswered. Whatever weakness existed, why did the engineering process not reveal it before the failure became consequential?

Why didn’t we know?

Validation asks engineers to connect consequential claims about a system to evidence. A delivered failure provides an opportunity to examine that connection retrospectively.

Validation distinguished the claim being evaluated, the scope at which the property exists, the evidence mechanism used to observe or reason about it, the strategy used to search the relevant behavior, the strength of the resulting evidence, and the judgment made from that evidence. A failure can reveal a problem at any of these points.

Engineers may have validated the wrong claim. A team can accumulate extensive evidence that individual components satisfy their interfaces while never asking whether the composed system preserves the property that matters.

The scope can be wrong. Availability, resource isolation, end-to-end latency, and other architectural properties may exist only at system scope. Evidence about individual components cannot establish a property created by their interaction unless engineers have an argument connecting the local evidence to the system claim.

The evidence mechanism can be poorly matched to the property. Ordinary execution tests can provide strong evidence for many behavioral claims while providing little evidence about rare concurrency schedules or adversarial memory-safety conditions. The mechanism may be useful without having the detection power the consequential claim requires.

The strategy used with an appropriate mechanism can also miss the relevant condition. A testing mechanism capable of exposing a defect provides no evidence about an execution the test strategy never reaches. Similarly, operational measurements can fail to reveal a capacity boundary when observed workloads remain far below it.

Finally, engineers can interpret reasonable evidence too strongly. Evidence supports a claim under assumptions and with limitations. Those limitations are part of the engineering judgment (“Judge the strength of the evidence”).

Failure does not prove inadequate validation

A production failure does not establish that engineers should necessarily have discovered the problem before delivery. Validation produces bounded evidence rather than certainty. Some adverse outcomes remain inside residual uncertainty even after competent engineers have gathered evidence commensurate with the expected consequence.

When failure occurs, ask what the new observation adds. It may reveal a weakness in the claim, scope, mechanism, strategy, or interpretation of previous evidence. It may instead reveal an uncertainty that was already recognized and deliberately accepted.

This distinction prevents failure analysis from degenerating into hindsight. Knowing the outcome changes what appears obvious. A good engineering decision is one that was justified by the evidence reasonably available when the decision was made, not one that happens to produce a favorable outcome.

This gives engineering two complementary questions:

• Before the decision: Given what we know, what should we do?
• After the outcome: Given what happened, what should we now believe?

Failure-aware engineering concerns the second question. It reconstructs what engineers could reasonably know before the outcome occurred and asks how the new evidence should change the understanding used for future decisions.

Human error has structure

Failure analyses often identify a human action somewhere in the causal history. Describing that action as human error can be accurate while providing little guidance about what should change.

Donald Norman distinguishes between slips and mistakes (Norman 2013). A slip occurs when the intended action is appropriate but execution does not match the intention. An engineer intends to modify one configuration value and accidentally modifies another. The problem lies in carrying out the intended action.

A mistake occurs when the action is performed as intended but the intention or plan is inappropriate. An engineer deliberately modifies the intended configuration value because their understanding of the system indicates that the change is safe. If that understanding is wrong, more careful execution of the same plan does not help.

The distinction matters because the engineering responses differ. Slips can motivate interfaces that make actions easier to distinguish, automation that removes repetitive manual operations, constraints that prevent dangerous combinations, checks that expose accidental actions, or defaults that make the safe action easier.

Mistakes direct attention toward the engineer’s understanding. The relevant response may involve better models, more accessible knowledge, different review practices, additional evidence, or cues that cause engineers to recognize when a familiar decision is being made under unfamiliar conditions.

In either case, telling people to be more careful is weak engineering when the environment can economically make the error less likely or reduce its consequence. Human fallibility is one of the conditions under which engineered systems operate. Designs can account for that condition just as they account for unreliable networks, finite memory, variable workloads, and other properties of the environment.

The slip-and-mistake distinction also matters for reflection. A mistake exposes an engineering judgment that appeared reasonable to the person making it. Understanding why it appeared reasonable can reveal missing knowledge or a model that future engineers might otherwise reproduce.

Failures pass through systems of defense

An error or defect does not necessarily become a consequential failure. Engineered systems contain many mechanisms that can prevent, detect, contain, or mitigate problems: reviews, type systems, validation, access controls, architectural boundaries, redundancy, monitoring, operational procedures, rollback mechanisms, and others.

James Reason’s Swiss-cheese model provides a useful representation of how failures can pass through such defenses (Reason 1990). Individual defenses are imperfect. A weakness that passes through one layer may be stopped by another. Consequential failure can occur when weaknesses in several defenses align sufficiently for a hazard to propagate through them (Figure 9).

Figure 9. Each defense is imperfect, and each imperfection sits somewhere different. A hazard becomes a consequential failure when several of those weaknesses happen to line up.

The model discourages an analysis that stops with the person or component nearest the visible failure. Suppose an implementation defect produces an invalid state. A type or memory-safety mechanism might have prevented the state, validation might have exposed the defect, an architectural boundary might have contained its effect, monitoring might have detected the problem before substantial damage occurred, or an operational mechanism might have restored service quickly. A consequential incident can therefore reveal information about several defenses even when one defect supplied the initiating condition.

Investigations of this kind are conventionally called root cause analysis, and the term is worth keeping because engineers will encounter it. The phrase can nevertheless mislead. A consequential failure rarely has one cause; it has causes and contributing conditions whose combination produced the outcome, and a report that names a single root has usually chosen one of them.

Reason’s model should not be treated as a complete theory of causation either. Real incidents can involve feedback, adaptation, organizational incentives, changing conditions, and interactions more complicated than a sequence of defensive layers. Its value here is narrower: it directs engineering attention from the visible error toward the system through which that error became consequential.

The question is therefore not only who or what made an error. Engineers should also ask which conditions permitted the error to propagate and which defenses could economically prevent or contain a similar event.

Systemic accounts correct an important weakness in explanations that terminate at the component or person closest to the failure. Widening the causal boundary indefinitely creates a different problem. Almost any consequential failure can be connected to progressively broader technical, organizational, economic, and regulatory conditions. Each may be causally relevant without being equally useful to the decision at hand.

A practical failure analysis should therefore consider the locus of control of the people expected to act on it. Engineers can change artifacts, interfaces, tests, observability, deployment mechanisms, and parts of the immediate engineering process. Managers can additionally change staffing, ownership, incentives, schedules, and operating policies. Executives can intervene in organizational structures, investments, and relationships that extend beyond an individual engineering team.

This is not an argument for ignoring causes outside the analyst’s authority. A condition outside the analyst’s control may explain the failure, or it may require escalation to someone who can act on it. An engineer who finds that a regulatory incentive contributed to a failure cannot change the regulator, but can escalate the finding, compensate for the condition, or explain why the problem cannot be solved locally. Causal breadth and actionable intervention are separate concerns.

Engineers should match the causal breadth of an analysis to the decisions it is intended to support. A useful causal model is broad enough to explain the relationships that matter and focused enough to identify decisions that can change. Choosing that boundary is itself a modeling decision: the analyst preserves the parts of the causal structure whose distinctions matter to the decisions that follow.

How bad was the failure?

Failures differ in consequence. A misspelled label and an outage that prevents a safety-critical service from operating are both departures from intended behavior, but they should not receive the same engineering response.

Organizations commonly use severity classifications to make differences in consequence actionable. The names and boundaries vary. An illustrative four-level scheme might distinguish the classes in Table 8.

Table 8. An illustrative severity scale, of the kind often numbered from Sev 1 down to Sev 4. The labels are not universal definitions; an organization that uses such a scale must define what its levels mean for the systems and consequences it manages.

Class	Consequence model	Relationship to engineering obligations
Critical	Catastrophic, mission-threatening, safety-critical, or system-wide consequential failure	A critical obligation is violated with severe realized or credible consequence
Major	Serious loss or degradation of important capability	An important obligation is substantially violated
Moderate	Bounded degradation with limited consequence or a practical workaround	A noncritical obligation is violated
Minor	Localized inconvenience or low-consequence deviation	A low-criticality obligation is violated

The important engineering idea is what the scale represents.

Severity

Severity is a model of the consequence of violating an engineering obligation. It is not an intrinsic property of a defect.

An incorrect bounds check in a disposable local tool and the same defect in a network-facing component controlling a consequential system do not create equivalent risk. The defect is identical; the severity is not.

Context matters as well. An outage can be minor under one operating mode and critical under another because the obligations and consequences differ. A severity judgment therefore depends on the violated obligation, operating context, extent of violation, and realized or credible consequence. An organization must make these relationships operational if its severity levels are to mean approximately the same thing to engineers applying them independently.

This relationship connects failure analysis back to Specification. Specifications need not treat every obligation as equally critical. Engineers can identify obligations whose violation creates greater consequence, represent groups or classes of criticality, and distinguish priorities where doing so supports later decisions. These priorities need not form a simple total ranking. Criticality can depend on context, and two obligations can be consequential in ways that are difficult to compare directly.

Criticality also connects failure analysis to validation. Before delivery, the consequence of violating an obligation helps determine how much evidence engineers should demand before relying on it (“Consequence changes the amount of evidence we should demand”). After a failure, the violated obligation and observed consequence help determine the severity and therefore the appropriate response.

A severity classification is consequently another example of a metric as model. The label compresses information about consequence into a representation that can support escalation, resource allocation, investigation, and remediation. As with other metrics, its usefulness depends on the relationship between the measured classification and the property engineers actually care about.

What changes because we failed?

Failure analysis has little engineering value if it produces no appropriate change. The change need not always be large, and sometimes analysis will show that no additional intervention is economically justified. When a lesson is consequential, however, engineers should consider where that lesson belongs.

Three places a lesson can live

• System. Structure, safeguards, and validation can be changed so that recurrence becomes harder or its consequence smaller.
• Team. Knowledge, practices, and coordination can be changed so that engineers who did not experience the incident can still act on what it revealed.
• Engineer. An individual’s repertoire, recognition, and judgment can be changed so that a resembling situation prompts investigation.

These levels interact. A lesson learned by an engineer can be communicated to a team. A team can decide that the lesson is important enough to encode into architecture, tooling, validation, or another part of the engineering environment. A structural change can then protect future engineers who never learn the original history.

Change the system

A common response to a software failure is to add a regression test that reproduces the conditions under which the failure occurred. The test preserves useful engineering knowledge: if a later change recreates those conditions, validation can expose the recurrence before delivery. For many implementation defects, this is an inexpensive and effective response.

A regression test is memory, but extremely literal memory. It does not necessarily preserve the more general lesson exposed by the failure. A buffer overflow triggered by one input may reveal a broader weakness in how lengths are represented or checked. A capacity failure in one service may expose an architectural assumption shared by several services. A new feature can recreate the same resource dependency through a different path without reproducing the original incident closely enough to trigger its regression test.

Engineers should therefore ask what class of conditions the observed failure represents. Sometimes the appropriate response remains a regression test. In other cases the lesson belongs in an interface, architecture, specification, shared mechanism, static check, deployment control, monitoring rule, or another part of the engineering environment.

Identifying that class changes the engineering problem. Diagnosis works backward from an observed consequence to the conditions that produced it. Corrective design works forward from possible interventions to the consequences they would produce. An incident involving two workers that both believe they own the same job, for example, might motivate changes to timeout handling, lease ownership, fencing, or the idempotency of job operations. These alternatives do not prevent the same failures, and they impose different costs and constraints. Engineers must determine which class of failures warrants intervention, then compare the available mechanisms against that obligation. The causal account informs the design decision; it does not make it.

This is the same movement of knowledge discussed in Engineering Knowledge. A lesson can begin as one engineer’s experience, become an explicit representation, and eventually become part of the structure within which future engineering occurs. When a consequential and sufficiently stable lesson can economically be encoded in that structure, future engineers need not independently remember the original incident to benefit from it.

Structural change is not always possible or desirable. Some lessons depend strongly on context. Others concern tradeoffs that require future judgment rather than a fixed rule. Encoding every incident as another mechanism can also accumulate complexity whose costs exceed the risks it controls. The decision remains an engineering one: determine what the failure teaches, how broadly the lesson applies, and which representation gives the lesson appropriate durability and force.

Make the failure legible to others

Lessons that cannot or should not be fully encoded in the system may still need to become available to other engineers. A team cannot reliably learn from an incident whose interpretation remains only in the memories of the people who handled it.

A postmortem is one mechanism for making an incident available for collective reflection. Practices differ, but a useful postmortem ordinarily reconstructs the incident, its consequences, relevant conditions, response, and actions intended to prevent or contain recurrence. The blameless postmortem practice associated with site reliability engineering also discourages stopping at individual fault (Lunney et al. 2016). Engineers acted under particular information, incentives, interfaces, procedures, and system conditions; understanding those conditions can reveal interventions more useful than identifying a person who happened to perform the final action.

Failure-aware engineering adds another question to this reconstruction: under what engineering understanding did the system appear acceptable before the failure?

Suppose an outage occurs because two services unexpectedly compete for the same finite resource. A causal account might establish which workload increased, how the resource became exhausted, how requests propagated, and which component ultimately stopped responding. Those facts are necessary for understanding the incident.

A reflective account also asks why the architecture permitted the dependency. Engineers may have believed that the workloads were independent. Perhaps the shared resource was absent from the architectural representation. Perhaps it was represented but assumed to have effectively unlimited capacity. Perhaps measurements supported that assumption under earlier workloads. Perhaps everyone knew about the dependency but judged its failure sufficiently unlikely that isolation was not worth its cost.

Those possibilities imply different lessons. A useful postmortem can therefore preserve two related accounts (Table 9).

Table 9. A causal account explains what produced the failure; a reflective account explains why the available engineering knowledge permitted the system to remain credible.

Causal account	Reflective account
What happened?	What did we expect to happen?
Which conditions contributed?	What model produced that expectation?
How did the failure propagate?	Which assumptions did that model depend on?
Which defenses prevented or failed to prevent propagation?	What evidence made those assumptions credible?
What changes can reduce recurrence or consequence?	What should change in our engineering understanding?

The reflective account is not an attempt to prove that someone should have known better. Sometimes the reconstruction will show that the engineering decision was reasonable under the evidence available at the time. That conclusion still produces useful knowledge. The lesson may concern containment, margin, reversibility, monitoring, or how much consequence an acknowledged uncertainty should be allowed to create.

In other cases, the reconstruction will expose a model that should have been questioned. The important result is then not merely the historical fact that the model was wrong, but an explanation of why it appeared adequate and what future circumstances should cause engineers to reconsider a similar assumption.

This is how an incident becomes useful beyond its literal details. An incident record stating that one workload exhausted one resource can help future engineers recognize that workload and that resource. An account explaining that apparently independent critical and noncritical functions shared an unmodeled resource teaches something that can be recognized in systems with entirely different components. The later failure does not repeat the earlier incident. It resembles it in the engineering relationship that matters.

Change the engineer

Some learning necessarily remains with people. Engineering encounters situations too varied to encode every useful lesson into a rule, test, or architectural constraint.

Reflection contributes to the engineer’s repertoire by identifying which aspects of an experience deserve recognition later. The useful lesson from a failed deployment may not be the exact command that caused it. It may be that a supposedly reversible operation depended on state outside the rollback mechanism. The useful lesson from a performance incident may not be the particular workload that overloaded one component. It may be that an asynchronous boundary hid a resource dependency from the engineers reasoning about the critical path.

Such lessons influence what experienced engineers notice. A future design exposes a similar relationship, and the engineer recognizes a reason to investigate it. The new situation need not be identical to the old one. Recognition supplies a hypothesis that deliberate reasoning can then examine.

This also explains why reflection should not end with a memorable story. Stories are useful because concrete situations are easier to remember than abstract rules, but judgment requires some ability to identify which features of the story mattered. Generalize too little and the engineer recognizes only repetitions of the original incident. Generalize too far and the lesson becomes a slogan that applies indiscriminately.

Reflection therefore involves modeling. The engineer chooses which relationships in an experience explain its consequence and carries those relationships forward as candidates for future reasoning.

Turn the handbook around

Before delivery, engineering is largely prospective. Engineers ask what the system should accomplish, what behavior would satisfy those obligations, which structures and mechanisms can realize them, and what evidence justifies believing that the resulting system is acceptable. Failure permits the same reasoning to be examined retrospectively.

A reflective analysis can therefore retrace the major engineering activities developed in this handbook:

• Requirements. Reconsider whether the requirements represented the outcomes that actually mattered. Did the system fail despite satisfying its stated requirements because an important stakeholder, consequence, or obligation was omitted or misunderstood?
• Specification. Reconsider the modeled relationship between the machine and its environment. Which obligations or environmental assumptions did the failure contradict, and why had those assumptions appeared adequate?
• Architecture and design. Reconsider the responsibilities, boundaries, interactions, resources, and mechanisms chosen to realize the specification. Which of these decisions contributed to the failure, and what understanding made the chosen structure appear appropriate beforehand?
• Implementation. Determine whether the realized system departed from the intended design. If so, examine both the defect and the engineering conditions that permitted it to be introduced and remain.
• Validation. Reconstruct the argument from claim to evidence. What claim was evaluated, at what scope, using which mechanism and strategy? What did the resulting evidence actually establish, and why was its remaining uncertainty judged acceptable?
• Metrics. Reconsider the models connecting properties to observations. Did the measurements used to understand or control the system represent the properties engineers believed they represented?

This retrospective traversal does not assume that every failure can be assigned to one engineering activity (Figure 10). Its purpose is to reconstruct the engineering understanding embodied in the system and identify where the new evidence should change it. A specification assumption may have shaped an architectural decision, for example, while the resulting weakness remained undetected because validation was performed at the wrong scope.

Figure 10. Before an engineering decision, models and evidence help determine what we should do given what we know. After an outcome contradicts our expectations, the same engineering activities can be examined retrospectively to determine what the new evidence should change.

This retrospective use of the handbook is one reason a postmortem can contribute more than a chronology of an incident. The system embodies earlier engineering decisions. Reconstructing those decisions allows engineers to compare what was believed before the incident with what the incident now gives them reason to believe.

A useful reflective account therefore preserves a change in engineering understanding. It explains enough of the previous model to make the new model intelligible.

Recurrence is evidence

A failure that has occurred before deserves particular attention. Recurrence can indicate that the earlier experience changed too little, changed the wrong thing, or produced a lesson that did not reach the engineering decision in which it was later needed.

The study of failure learning at the national space research center illustrates this problem (Anandayuvaraj et al. 2026). Engineers described recurring failures even when relevant lessons existed within the organization, because those lessons did not reliably become available across projects or persist through time.

Recurrence can arise for different reasons. Engineers may repair the observed manifestation without changing the condition that produced it; a broader lesson may not be preserved or communicated; engineers may fail to recognize that it applies to a superficially different situation; or a structural intervention may prove weaker than expected. An organization may also deliberately tolerate some probability of recurrence because eliminating it would cost more than the expected consequence justifies. Recurrence therefore requires interpretation: incidents that look alike may have different causes, while incidents that look different may expose the same engineering weakness.

At scale, engineers can search incident records and postmortems for earlier failures that share the same engineering relationship rather than the same literal symptoms. Language models can help identify candidate relationships across large collections of incident reports, although engineers must still judge whether two incidents genuinely express the same underlying lesson.

A consequential recurrence nevertheless creates a useful question: what did the previous experience change, and why was that change insufficient here?

The answer can direct engineers back through the levels discussed above. If only the observed instance was addressed, the system may need a more general structural change. If the lesson remained with a few individuals, the team may need a more durable representation. If the lesson was known but not recognized as applicable, the experience may need to be generalized differently. If the recurrence was an explicitly accepted risk, engineers can reconsider whether the new evidence changes the original tradeoff.

The first failure supplies evidence about the engineered system. Recurrence can additionally supply evidence about how the engineering organization learns from failure.

Measuring learning from failure

Failure-aware engineering creates an understandable desire for measurement. If an organization invests in postmortems, corrective actions, regression tests, and other mechanisms for learning from failure, engineers should ask whether those investments are changing what happens afterward.

Organizations can observe incident frequency and severity, detection and recovery time, completion of corrective actions, recurrence within and across projects, and whether lessons become encoded into durable engineering structures. These measurements make different parts of the learning process visible, but none directly establishes that learning occurred. Mean time to recovery measures operational response; action-item completion measures whether planned work occurred; recurrence requires a model of what different incidents have in common. Table 10 summarizes these distinctions.

Table 10. Observations that make a failure-learning process visible, each paired with the inference it does not license.

Observation	What it can help reveal	What it does not establish
Failure frequency and severity	How often consequential failures are observed and how serious their consequences are	Why the rate changed, or whether engineers learned
Detection and recovery time	How effectively the system and organization expose and contain failures	Whether the underlying engineering weakness was removed
Corrective-action completion	Whether agreed changes were carried out	Whether the actions represented the right lesson
Recurrence	Whether related failure conditions continue to produce incidents	Whether two incidents share the same underlying engineering cause
Cross-project recurrence	Whether a lesson may have failed to propagate through the organization	Whether communication alone would have prevented the later failure
Structuralization of lessons	Whether lessons become tests, checks, models, architecture, tooling, or other durable engineering structure	Whether the encoded lesson is sufficiently general, or worth its cost

The final row returns to Engineering Knowledge. A lesson that moves from individual memory into a test, model, validation rule, architectural constraint, or other shared structure has become more durable. Counting such changes can describe one aspect of organizational learning; it does not establish that the right lesson was learned.

Judgment is not directly measurable

Engineering judgment itself should not be reduced to a metric. Judgment is the capacity to recognize what matters in a particular situation and reason appropriately about it. No count of incidents, postmortems, completed actions, or accumulated years of experience directly measures that capacity.

This limitation follows from the model of metrics developed earlier in the handbook. A metric connects a property we care about to observations we can make. The farther the observable quantity lies from the property of interest, the stronger the model required to interpret it. We can observe whether an engineer notices a particular hazard, asks a useful question, revises a model after contrary evidence, or makes a sound decision in a particular case. Those observations can provide evidence about judgment. They do not produce a context-independent quantity of judgment possessed by the engineer.

This matters especially when measurements become targets. Counting postmortems can encourage writing postmortems. Counting action items can encourage producing action items. Counting recurring incidents can encourage classifying incidents as unrelated. None guarantees that engineers have improved the models and recognition they bring to future decisions.

Use metrics to make the learning system observable, not to pretend that judgment itself has been quantified.

Measurement for decision-making

The retrospective traversal earlier in this chapter (“Turn the handbook around”) asked which engineering model a failure should change. Failure is the ordinary engineering loop running backward. Before a decision, engineers select observations that can test the models on which their decisions depend. When failure occurs, the observation arrives uninvited and already disagrees with an expectation. The remaining question is: Which model should change?

Metrics deserve particular attention because a metric is itself a model connecting a property we care about to something observable. Consider a service whose dashboard showed a healthy request-success rate throughout an outage because the rate was computed at a load balancer that never saw requests dropped upstream. Nothing was wrong with the measurement itself. What failed was the claim engineers had attached to it. A failure alongside untroubled indicators is therefore evidence about the indicators as well as the system.

Failures are expensive observations, and the organization has already paid for them. The engineering task is to spend that evidence on the model that deserves to change rather than on the smallest repair that makes the symptom disappear.

Becoming a reflective engineer

Failure-aware engineering is ultimately part of developing professional judgment. The engineer begins with models and evidence, acts under uncertainty, observes consequences, and revises future reasoning when those consequences reveal something important.

The models developed in this chapter support different parts of that activity. They help engineers reflect on practice, distinguish failures of execution from failures of understanding, trace how weaknesses pass through systems of defense, judge consequences, reconsider earlier evidence, and preserve lessons beyond the people who directly experienced them.

None provides a mechanical procedure for learning the correct lesson. Reflection itself is an act of engineering judgment. Engineers can infer the wrong cause, overgeneralize from one incident, preserve irrelevant details, or impose an expensive control whose benefit does not justify its cost. Lessons from failure therefore deserve the same skepticism applied to other engineering knowledge.

A useful reflective practice begins by reconstructing the expectation and the observation. What did you expect, and what occurred? Ask why the expectation appeared credible, which part of your understanding the observation actually challenges, and what alternative explanations remain. Then ask how broadly the lesson generalizes and what should change in the system, in shared engineering knowledge, or in your own future judgment.

Success, near misses, experiments, and other engineers’ experiences can contribute similarly when they provide evidence across new conditions. The purpose is not to accumulate personal failures, but to become increasingly capable of connecting engineering decisions with their consequences.

No engineer reaches a point at which this process becomes unnecessary. Systems change, environments change, organizations change, and engineers take responsibility for systems unlike those they have previously encountered. Models remain selective representations of reality, and evidence remains finite.

Engineering therefore requires both forms of judgment developed in this chapter and throughout the book. Deliberate reasoning helps engineers make consequential decisions under uncertainty. Experience and reflection expand the repertoire with which engineers recognize what deserves such reasoning in the first place. Each can improve the other over a career. A failure-aware engineer treats the discrepancy between expectation and reality as new engineering evidence, determines what that evidence supports, and allows consequential lessons to change future work.

Summary

Engineering judgment develops by connecting decisions to their consequences. Engineers use models and evidence to reason deliberately about consequential decisions under uncertainty. Experience contributes another form of judgment: recognition of situations, assumptions, and hazards that deserve attention. Reflection connects the two by asking what an outcome reveals about the engineering understanding that preceded it.

Failure provides particularly useful evidence because it contradicts an expectation. The contradiction does not explain itself. Engineers must determine what failed, why earlier engineering did not reveal the problem, and which part of their understanding the new observation should change. The answer may lie in requirements, specification, architecture, design, implementation, validation, or the models connecting measurements to properties of interest. Failure does not imply that an earlier decision was unreasonable; finite evidence always leaves residual uncertainty.

Learning requires more than repairing the immediate problem. A consequential lesson can change the system, by becoming a safeguard, constraint, test, model, or other engineering structure; the team, by becoming knowledge that remains available beyond the original participants; or the engineer, by changing the repertoire through which future situations are recognized and interpreted. Recurrence provides additional evidence about whether those changes were sufficient.

Failure-aware engineering therefore turns the handbook around. Before a decision, engineers ask: Given what we know, what should we do? After an outcome, they ask: Given what happened, what should we now believe? The goal is not to eliminate uncertainty or failure. It is to use consequential experience well: determine what the evidence supports, preserve lessons worth retaining, and allow them to improve the engineering decisions that follow.

Read Further

Schön, Donald A. The Reflective Practitioner: How Professionals Think in Action. Basic Books, 1983. How professional judgment develops through reflection on practice. Read for reflection-in-action, reflection-on-action, and the repertoire an experienced practitioner brings to a new situation.

Norman, Donald A. The Design of Everyday Things. Revised and expanded. Basic Books, 2013. The distinction between slips and mistakes, and why apparent human error often points at the conditions and artifacts through which people act.

Reason, James. Human Error. Cambridge University Press, 1990. https://doi.org/10.1017/CBO9781139062367. A systemic account of human failure, latent conditions, and defenses. Read for the reasoning behind the Swiss-cheese model rather than the diagram itself.

Petroski, Henry. To Engineer Is Human: The Role of Failure in Successful Design. Vintage Books, 1992. How engineering disciplines have treated failure as knowledge about the limits of their designs and models.

Lunney, John, Sue Lueder, and Gary O’Connor. “Postmortem Culture: Learning from Failure.” In Site Reliability Engineering: How Google Runs Production Systems, edited by Betsy Beyer, Chris Jones, Jennifer Petoff, and Niall Richard Murphy. O’Reilly Media, 2016. https://sre.google/sre-book/postmortem-culture/. A production practice for turning operational failures into shared organizational knowledge.

Anandayuvaraj, Dharun, Tanmay Singla, Zain A. H. Hammadeh, Andreas Lund, Alexandra Holloway, and James C. Davis. “Learning from Software Failures: A Case Study at a National Space Research Center.” In Proceedings of the 48th IEEE/ACM International Conference on Software Engineering (ICSE 2026). 2026. Empirical evidence on how practitioners gather, document, share, and apply lessons from failure, and what happens when that learning stays informal.