4.4 Operating MAGE

A method has to survive ordinary days, not only successful changes. Repositories fill disks, branches conflict, agents disappear, queues stall, tests flake, deployments degrade, and assumptions age. Engineers often know how to recover, but that knowledge may live only in someone's head, where an autonomous agent cannot reliably reach it. MAGE therefore brings the same Modeling and Alignment disciplines into operation. Lifecycle models represent healthy progress and important failure states. Events make consequential transitions observable. Runbooks preserve sanctioned reactions, including where judgment remains. Metrics and measurement models expose the quantities needed to judge whether the loop is healthy. Together they support an operating loop: recognize the state, surface the right reaction, take the appropriate action, and observe what happened next.

These structures play different roles. A lifecycle is a model; an event supplies evidence that a transition occurred; a runbook preserves a procedure; and a metric supplies measurement evidence. None is binding merely by existing. Constraints, validators, and gates give selected operational obligations consequence where the required state is legible and the evidence adequate. Keeping those roles separate lets the operating environment grow without collapsing representation, evidence, procedure, and enforcement into one mechanism.

4.4.1 Model the Lifecycle

Start with the healthy path of a recurring activity: how a bug moves from reported to fixed, how a feature moves from designed to shipped, or how an agent moves from dispatched to landed. Then name the failure states that matter and the sanctioned recovery from each. The result is a lifecycle model: a behavioral model specialized to operation, retaining the states and transitions needed to recognize healthy progress and recover from important breaks.

FMEA — failure mode and effects analysis — offers a useful neighboring practice: enumerate important failure modes before recovery is needed. MAGE represents the states, legal transitions, and sanctioned recoveries needed to operate the lifecycle.

Pair prohibitions with recovery paths. "Never use this destructive merge command" leaves a stuck agent with a prohibition and no exit. "When this state occurs, do not use X; use Y because it preserves the already-applied work" supplies the missing path. Where the prohibition can be made structural, encode it as a constraint. Where it cannot, preserve the guidance explicitly.

A prohibition without a recovery path can strand the actor. DocAble encountered this when agents used a destructive Git escape during stuck merges. The durable rule named the forbidden action and the sanctioned recovery that preserved already-applied work.

The fleet's own lifecycle deserves the same treatment as the product. Point the discipline at the substrate that produces the software and you get the agent-orchestration model — an agent moving through explicit states (Dispatched → Working → Landed → Tombstoned, with Abandoned and recovery paths), and the orchestrator's refill-and-bank loop as the journey that drives it. Once those states are explicit, the environment can check the transitions it can decide mechanically — refusing, for example, to tombstone work before the required commit exists. Not every lifecycle property is a state-transition check. A landed agent eventually tombstones is a liveness claim: no single bad state falsifies it, so the obligation routes to a temporal checker instead. The lifecycle supplies the vocabulary; the shape of the obligation determines the evidence.

That state machine is also why an operator never takes a report at face value. "Done" is a claim, not a fact. An agent reporting a task complete is describing its intent, not the state of your repository, and the two drift apart constantly — a sibling change broke a test, a marker rotted, the thing it "verified" quietly routed through a fallback. The report therefore should not be confused with repository state. Where completion is consequential, the environment may re-evaluate the relevant entry conditions against current state rather than relying on the report alone. In DocAble, the lifecycle names Landed as a state with an entry condition, and the orchestrator re-runs the relevant gates at HEAD before accepting that transition.

AGENT HARNESS

Inset — Subagents and Fleets

MAGE's concern with decomposition and coordination appears in today's agent harnesses when work is delegated to subagents or several agents run concurrently. A harness may give each worker a separate context, task, branch, or worktree and later collect their results. These mechanisms can increase realization capacity substantially: several bounded pieces of implementation can proceed at once instead of occupying one long reasoning episode.

Parallel execution, however, does not itself decompose the engineering problem. The work must be divided along boundaries that make concurrent realization useful. Workers need the right shared knowledge and task-specific context. Their changes need sufficiently independent scopes, and the evidence they return must be adequate to decide whether the pieces can be accepted together. Branches and worktrees can isolate file changes, but they do not make those changes semantically independent. Two agents can produce individually plausible changes that disagree about an interface, duplicate responsibility, violate the same architectural boundary in different ways, or become incompatible only when integrated.

This is why MAGE uses fleet for more than "several agents running at once." The engineering question is how much concurrent realization the environment can absorb while preserving coherence. Increasing the number of agents increases implementation capacity; models, ownership boundaries, shared state, validation, and integration structure determine whether that capacity becomes useful parallel engineering rather than faster production of coordination failures.

4.4.2 Attach Reactions to Lifecycle Events

A lifecycle model says which states matter. Operation improves when an important transition can also make the right procedure available at the right moment. Do not rely on an actor to remember to inspect the lifecycle, notice that a transition happened, retrieve the right procedure, and invoke it. Where a recurring event is mechanically observable and a standard response is useful, bind the reaction to the event.

A mechanical trigger does not require a mechanical response. The firing can be deterministic while the payload still contains judgment. An agent reaches a context threshold: a hook deterministically requests a handoff, but the agent must still summarize what matters. A deployment fails a health check: the event deterministically opens the recovery path, but an engineer may still decide whether rollback is warranted. A repeated failure is observed: the reflection procedure runs on its own, but whether that failure deserves a new mechanism stays an engineering decision.

Bind known failure events to their sanctioned reactions: the reaction fires rather than waiting for an operator to notice the failure. The environment emits structured events, and each can trigger the appropriate reaction, reducing dependence on an operator noticing and polling the signal manually. Automate the arrival of the decision procedure. Figure 4.4-1 draws the full path from a lifecycle transition to the next state.

4.4.3 Externalize Operational Judgment

A runbook preserves how an experienced engineer reasons through a recurring situation, including which steps are mechanical and which require judgment.

PRACTICE

Inset — Runbooks

A runbook is a prepared procedure for a recurring operational task or failure. Site reliability engineering uses runbooks and incident playbooks to preserve diagnostic and recovery knowledge before an outage requires it ¹1. B. Beyer et al., Site Reliability Engineering: How Google Runs Production Systems (O'Reilly Media, 2016).. An empirical study of real incidents shows how variable that recovery work is ²2. Jonathan Sillito and Esdras Kutomi, “Failures and Fixes: A Study of Software System Incident Response,” in “2020 IEEE International Conference on Software Maintenance and Evolution (Icsme),” special issue, 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME), 2020, 185–95, https://doi.org/10.1109/ICSME46990.2020.00027.. Executable runbooks can combine commands, scripts, API calls, and manual steps into a repeatable workflow ³3. PagerDuty, “Supercharging Incident Response with Runbook Automation,” PagerDuty, 2021, https://www.pagerduty.com/blog/supercharging-incident-response-runbook-automation/..

For agentic operation, one useful addition is to record who or what can decide each step. Deterministic steps can execute directly; bounded judgments can be delegated with prepared context; irreducible judgments stay with the operator.

Runbooks can drift. Their procedures therefore need the same maintenance discipline as other engineering models.

A runbook preserves the sanctioned response to a recurring operational situation, but not every step has the same decision structure. Some steps have one mechanically determined outcome; some require bounded reasoning; some require a human decision. Type each step accordingly: Execute when the outcome is mechanically determined and can be preserved as an executable operation; Delegate when the step requires bounded judgment, supplying the relevant state, choices, criteria, and required output; Escalate when the decision remains irreducibly human, preparing the evidence and surfacing the decision explicitly. This is the same split used by event reactions: the event determines when the procedure arrives; the runbook determines what happens next. Figure 4.4-2 shows the three kinds and the form each takes.

A structured operational playbook can serve as a model of engineering practice. It externalizes not only what to do, but which kind of reasoning each step requires — and it connects directly to the next chapter, where a process-skill packages this kind of recurring procedure for reuse.

Where possible, surround a judgment-laden step with deterministic work. Prepare the data mechanically before the decision; after the decision, execute and measure mechanically. Preserve the rationale for the judgment separately from the trace of what ran. This does not make the judgment deterministic; it bounds where judgment occurs while leaving the surrounding work replayable and checkable. Figure 4.4-3 draws the shape.

The split is not unique to DocAble. Systems that share no code and answer to different regulators reach for the same fence between the step a machine can decide and the step that needs a mind.

4.4.4 Measure the State That Matters

Operational measurement should expose the state the decision actually depends on. A worker count may be a poor proxy for memory pressure; measure the pressure. Process isolation may not imply resource isolation; mediate the shared resource. A marker on disk may not establish liveness; record lifecycle state explicitly. When an aggregate hides the decision you need to make, measure one level deeper — not arbitrarily deeper, but far enough to expose a useful engineering target. Keep measurement separate from enforcement: a metric supplies evidence, a validator may interpret that evidence against an obligation, and a gate may act on the resulting verdict.

ECONOMICS

Inset — When a measure becomes a target

Engineering metrics are proxies for properties we care about. Line coverage stands in for something about test adequacy; defect counts for something about quality; lead time for something about delivery performance. The proxy is useful only while it tracks the consequential property well enough for the decision being made.

Goodhart's law warns what can happen when the proxy itself becomes consequential: once a measure becomes a target, optimizing the measure can cease to improve the property it was meant to represent. A team rewarded for closing tickets may split or prematurely close them. A coverage threshold may encourage tests that execute lines without testing important behavior. An agent rewarded for reducing lint findings may suppress findings rather than remove the underlying defects. In each case, the measured number improves while its correspondence with the engineering concern weakens.

The lesson is not to avoid metrics or targets. It is to treat the metric as a model whose correspondence matters. Prefer measures closer to the consequential state; combine signals when one proxy is inadequate; and be especially skeptical when a metric starts to gate work or carry incentives. The stronger the consequence attached to a measure, the stronger the case needed that improving it still means improving what you actually care about.

Improve the signal when a proxy drives a consequential decision and no longer tracks the relevant state.

An operating loop needs evidence about its own state. A useful metric changes the next engineering decision. A whole-project cloud bill tells you what you spent; cost by service may tell you what to change. Aggregate line coverage tells you how much source ran; claim-level coverage may tell you which invariant your suite never exercised. Counting AI-assisted pull requests or generated lines tells you activity changed; pairing throughput with yield and rework ⁴4. LinearB, “The Engineering Productivity Gap: How Elite AI Teams Are Pulling Away from the Rest,” LinearB, 2026, https://linearb.io/resources/ai-engineering-productivity-gap. gives a more informative view of engineering outcomes than activity counts alone.

Ousterhout's rule is apt ⁵5. John Ousterhout, “Always Measure One Level Deeper,” Communications of the ACM 61, no. 7 (2018): 74–83.: measure one level deeper. One level, not arbitrarily deep. A metric earns its maintenance cost when the deeper cut exposes an optimization target, failure class, or decision the surface number hides. An aggregate such as "87% line coverage" reports activity at a level that may hide the engineering claim that matters — a line in a throwaway string helper counts the same as a line in the job state machine.

Some evidence is mechanically decidable and appropriate for automated admission; other evidence is intentionally judgment-laden and should point a reader toward a decision rather than masquerade as a threshold. The next two examples sit at opposite ends of that spectrum.

One hard example: coverage measured one level deeper

Requirements-based coverage joins model-to-code traceability with ordinary test coverage: it follows each modeled claim to the code that realizes it, then to the tests that exercise that code, and reports which claims have no exercising evidence. It exists because ordinary line and branch coverage measure whether code ran, not whether the properties that matter were checked — a suite can report ninety-five percent line coverage while an entire requirement stays untested. Consider a requirement that uploaded documents over a size bound are rejected without crashing: the traceability chain names the guard that enforces the bound and the test that drives an oversized file through it, so a missing test shows up as an unexercised claim rather than a slightly lower percentage. The payoff is a mechanically detectable gap the environment can name — this invariant is untested — instead of an aggregate line percentage. Because that detection is a deterministic join over explicit models, the example sits at the mechanized end of the spectrum.

One judgment-laden example: doc-derived tests

Doc-derived tests are generated from consequential claims in prose documentation and keep a trace back to the source claim. They exist because documentation makes promises the code can quietly stop keeping; turning a promise into an executable check keeps the two aligned. Suppose the documentation promises that a failed conversion preserves the user's original file. The derived test induces a conversion failure and asserts the original survives, recording the link from that sentence to the test and its result. Machinery can preserve this claim-to-test-to-result trace and re-run it on every change. What machinery cannot settle is whether an arbitrary sentence expresses a consequential, testable claim at all, and whether the generated test captures what the sentence meant — both can require expert judgment. Because the semantic adequacy of that translation stays judgment-laden, the measurement should direct a reviewer's attention rather than drive an admission threshold, which is why it sits at the judgment-laden end of the spectrum.

Let useful models drive machinery

An executable model earns more than documentation value when tools consume it directly. A deployment-topology model can tell a latency analysis which services and paths matter. A traceability graph can tell a coverage tool which code realizes a particular claim. A component graph can help a validator decide whether a finding is local to the current change or belongs to an unrelated part of the system. This is one of Modeling's practical payoffs: the representation becomes a reusable query surface. It does not itself enforce anything — the consuming validator or gate determines what follows from the model-derived result.

A model can also scope a mechanism. DocAble locates a finding and the current change in the component model and uses their relationship to determine which admission boundary owns the finding. A local finding can therefore block the current change, while unrelated debt remains the responsibility of a broader gate.

The grade is evidence read from the model at check time; the gate carries the consequence. Read the number one level below the surface, where it names a target instead of a total.

4.4.5 The Operating Rhythm

Put the pieces together and operation becomes a recurring rhythm. The lifecycle tells you what healthy progress and important failure look like. Events bind the reaction to the transition, so the right procedure arrives when it matters. The runbook gives each state a sanctioned next move and separates deterministic execution from judgment. Metrics expose enough state to tell whether the loop is improving or merely busy. When a break recurs, governance conversion asks what later work should inherit from the lesson.

The operating loop will still stall. When it does, resist the reflex to add another mechanism. First diagnose what kind of problem you have. A searcher problem calls for better search or reasoning; a model problem for better representation; an oracle problem for better evaluation; and a target problem for stopping long enough to learn what the system should actually do.

PRACTICE

Inset — When work stalls, diagnose before adding machinery

Repeated failure does not always call for another control. Before you build one, ask which part of the loop is limiting the work.

Table 4.4-1.

What may be limiting	The question	If that is the gap
Searcher	Can the available intelligence find a good candidate?	Change the model, tools, decomposition, or search strategy.
Model	Is consequential structure missing from the representation?	Make it explicit.
Oracle	Can the environment tell good candidates from bad?	Strengthen the evidence, validation, or simulation.
Target	Do we know what "good" even means?	Stop building and learn.

A stronger reasoner does not settle an unknown requirement; another validator does not supply a missing abstraction; more modeling cannot answer a product question whose answer must come from users or deployment. Diagnose what is limiting the work before adding machinery — the same distinction governance conversion already draws between failures of knowledge, obligation, evidence, evaluation, and consequence.

Run this rhythm long enough and recurring operation becomes less dependent on memory. Healthy paths become explicit; known failures summon known responses; mechanical steps execute mechanically; judgment arrives with prepared evidence; and repeated breakdowns feed back into the engineering environment. The engineer increasingly decides what the environment should know, what it should decide, and what must remain judgment. Part VI returns to that professional shift.

Brief. The written instructions handed to an agent for a task: what to do, where, and how "done" is judged; the artifact the pre-canned template produces.