4.4 Operating MAGE

A method has to survive ordinary days, not only successful changes. Repositories fill disks, branches conflict, agents disappear, queues stall, tests flake, deployments degrade, and assumptions age. Engineers already know how to recover from much of this — but often only as tacit operating knowledge. Autonomous work exposes the cost of leaving that knowledge in someone's head: the disk fills, and the instruction for what to do about it will never be in the agent's context unless you put it there.

Operating MAGE makes recurring work explicit in four connected forms. Lifecycles model healthy progress and important failure states. Events trigger intervention when relevant transitions occur. Runbooks externalize sanctioned responses and separate execution from judgment. Metrics expose the state needed to judge whether the loop is healthy. Events are the bridge between modeling the lifecycle and making the environment react.

Not all of that structure is authoritative. A lifecycle is a model. An event fires deterministically, though its payload may still carry judgment. A runbook may contain both deterministic tools and judgment guidance. A metric produces evidence. Authority appears only where constraints, validators, or gates act on those structures. Keeping those roles separate is what lets the operating environment remain understandable as it grows.

4.4.1 Model the Lifecycle

Start with the healthy path of a recurring activity: how a bug moves from reported to fixed, how a feature moves from designed to shipped, how an agent moves from dispatched to landed. Then name the failure states that matter and the sanctioned recovery from each. The result is a lifecycle model: the states and transitions an operator needs to recognize healthy progress and recover from important breaks.

FMEA — failure mode and effects analysis — offers a useful neighboring practice: enumerate important failure modes before recovery is needed. MAGE represents the states, legal transitions, and sanctioned recoveries needed to operate the lifecycle.

Pair prohibitions with recovery paths. "Never use this destructive merge command" leaves a stuck agent with a prohibition and no exit. "When this state occurs, do not use X; use Y because it preserves the already-applied work" supplies the missing path. Where the prohibition can be made structural, encode it as authority. Where it cannot, preserve the guidance explicitly.

A prohibition without a recovery path can strand the actor. DocAble encountered this when agents used a destructive Git escape during stuck merges. The durable rule named the forbidden action and the sanctioned recovery that preserved already-applied work.

The fleet's own lifecycle deserves the same treatment as the product. Point the discipline at the substrate that produces the software and you get the agent-orchestration model — the agent's life written as a typed state machine (Dispatched → Working → Landed → Tombstoned, with Abandoned and recovery), and the orchestrator's refill-and-bank loop as the journey that drives it. The model makes lifecycle properties explicit, and associated checks establish lifecycle soundness by detecting illegal transitions the environment can decide mechanically, such as tombstoning before a required commit. If Dispatched, Working, Landed, and Tombstoned are meaningful states, make their legal transitions explicit and validate the ones the environment can decide. The one liveness property — a landed agent eventually tombstones — cannot be falsified by a single bad state, so it instead routes to a temporal checker.

Learn more about this governance mechanism: agent-orchestration model.

That state machine is also why an operator never takes a report at face value. "Done" is a claim, not a fact. An agent reporting a task complete is describing its intent, not the state of your repository, and the two drift apart constantly — a sibling change broke a test, a marker rotted, the thing it "verified" quietly routed through a fallback. The report therefore should not be confused with repository state. Where completion is consequential, the environment may re-evaluate the relevant entry conditions against current state rather than relying on the report alone. In DocAble, the lifecycle names Landed as a state with an entry condition, and the orchestrator re-runs the relevant gates at HEAD before accepting that transition.

4.4.2 Attach Reactions to Lifecycle Events

A lifecycle model says which states matter. Operation improves when an important transition can also make the right procedure available at the right moment. Do not rely on an actor to remember to inspect the lifecycle, notice that a transition happened, retrieve the right procedure, and invoke it. Where a recurring event is mechanically observable and a standard response is useful, bind the reaction to the event.

A mechanical trigger does not require a mechanical response. The firing can be deterministic while the payload still contains judgment. An agent reaches a context threshold: a hook deterministically requests a handoff, but the agent must still summarize what matters. A deployment fails a health check: the event deterministically opens the recovery path, but an engineer may still decide whether rollback is warranted. A repeated failure is observed: the reflection procedure runs on its own, but whether that failure deserves a new mechanism stays an engineering decision.

Bind known failure events to their sanctioned reactions: the reaction fires rather than waiting for an operator to notice the failure. The environment emits structured events, and each can trigger the appropriate reaction, reducing dependence on an operator noticing and polling the signal manually. Automate the arrival of the decision procedure. Figure 4.4-1 draws the full path from a lifecycle transition to the next state.

4.4.3 Externalize Operational Judgment

A runbook preserves how an experienced engineer reasons through a recurring situation, including which steps are mechanical and which require judgment.

PRACTICE

Inset — Runbooks

A runbook is a prepared procedure for a recurring operational task or failure. Site reliability engineering uses runbooks and incident playbooks to preserve diagnostic and recovery knowledge before an outage requires it ¹1. B. Beyer et al., Site Reliability Engineering: How Google Runs Production Systems (O'Reilly Media, 2016).. An empirical study of real incidents shows how variable that recovery work is ²2. Jonathan Sillito and Esdras Kutomi, “Failures and Fixes: A Study of Software System Incident Response,” in “2020 IEEE International Conference on Software Maintenance and Evolution (Icsme),” special issue, 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME), 2020, 185–95, https://doi.org/10.1109/ICSME46990.2020.00027.. Executable runbooks can combine commands, scripts, API calls, and manual steps into a repeatable workflow ³3. PagerDuty, “Supercharging Incident Response with Runbook Automation,” PagerDuty, 2021, https://www.pagerduty.com/blog/supercharging-incident-response-runbook-automation/..

For agentic operation, one useful addition is to record who or what can decide each step. Deterministic steps can execute directly; bounded judgments can be delegated with prepared context; irreducible judgments stay with the operator.

Runbooks can drift. Their procedures therefore need the same maintenance discipline as other engineering models.

MAGE types each step by who or what can honestly decide it. Figure 4.4-2 shows the three kinds and the form each takes.

Runnable. One mechanically determined outcome; preserve the executable operation. Delegable judgment. Bounded reasoning supplied with state, choices, criteria, and a required output. Human judgment. A decision surfaced to the operator.

A structured operational playbook can serve as a model of engineering practice. It externalizes not only what to do, but which kind of reasoning each step requires. It connects directly to the next chapter: a mastery skill is one way to package exactly this kind of recurring judgment. Figure 4.4-3 fences the one judgment step between two deterministic ends.

Learn more about this governance mechanism: runbook.

A structured execution trace records what ran. Preserve rationale separately when judgment selected the action. The trace can become input to the next run and evidence for an independent checker.

Learn more about this governance mechanism: brief-linting.

The split is not unique to DocAble. Systems that share no code and answer to different regulators reach for the same fence between the step a machine can decide and the step that needs a mind.

4.4.4 Measure the State That Matters

Operational control fails easily when it acts on a proxy that only sometimes tracks the property that matters. A worker count is not resource pressure. Separate processes are not independent when they contend for the same machine-wide resource. A marker left on disk is not proof that an actor is alive.

The general rule: observe and govern the state that actually constrains the system, not merely the state that is convenient to count. Sometimes the answer is a better sensor. Sometimes it is an explicit registry. Sometimes it is mediation — when many actors share a resource whose capacity matters, route access through a control that knows the capacity, rather than asking each actor to infer whether proceeding is safe.

Use the simplest direct signal available.

• Govern the pressure, not the proxy. Read memory pressure rather than assuming job count predicts it.
• Mediate the shared resource. Route a contended test resource through a mediator rather than assuming process isolation provides resource isolation.
• Record the state, don't reconstruct it. Write lifecycle state to a registry rather than inferring liveness from incidental side effects.

ECONOMICS

Inset — When a measure becomes a target

Engineering metrics are proxies for properties we care about. Line coverage stands in for something about test adequacy; defect counts for something about quality; lead time for something about delivery performance. The proxy is useful only while it tracks the consequential property well enough for the decision being made.

Goodhart's law warns what can happen when the proxy itself becomes consequential: once a measure becomes a target, optimizing the measure can cease to improve the property it was meant to represent. A team rewarded for closing tickets may split or prematurely close them. A coverage threshold may encourage tests that execute lines without testing important behavior. An agent rewarded for reducing lint findings may suppress findings rather than remove the underlying defects. In each case, the measured number improves while its correspondence with the engineering concern weakens.

The lesson is not to avoid metrics or targets. It is to treat the metric as a model whose correspondence matters. Prefer measures closer to the consequential state; combine signals when one proxy is inadequate; and be especially skeptical when a metric acquires authority or incentives. The stronger the consequence attached to a measure, the stronger the case needed that improving it still means improving what you actually care about.

Improve the signal when a proxy drives a consequential decision and no longer tracks the relevant state. From here, measure one level deeper.

An operating loop needs evidence about its own state. A useful metric changes the next engineering decision. A whole-project cloud bill tells you what you spent; cost by service may tell you what to change. Aggregate line coverage tells you how much source ran; claim-level coverage may tell you which invariant your suite never exercised. Counting AI-assisted pull requests or generated lines tells you activity changed; pairing throughput with yield and rework ⁴4. LinearB, “The Engineering Productivity Gap: How Elite AI Teams Are Pulling Away from the Rest,” LinearB, 2026, https://linearb.io/resources/ai-engineering-productivity-gap. gives a more informative view of engineering outcomes than activity counts alone.

Ousterhout's rule is apt ⁵5. John Ousterhout, “Always Measure One Level Deeper,” Communications of the ACM 61, no. 7 (2018): 74–83.: measure one level deeper. One level, not arbitrarily deep. A metric earns its maintenance cost when the deeper cut exposes an optimization target, failure class, or decision the surface number hides. An aggregate such as "87% line coverage" reports activity at a level that may hide the engineering claim that matters — a line in a throwaway string helper counts the same as a line in the job state machine.

Keep measurement separate from authority. A metric is evidence. A validator may interpret it against a bound; a gate may then act on that verdict. Some evidence is mechanically decidable and appropriate for automated admission. Other evidence is intentionally judgment-laden and should point a reader toward a decision rather than masquerade as a threshold.

One hard example: coverage measured one level deeper

Requirements-based coverage is one example of measuring at the semantic unit that matters. Join model-to-code traceability with ordinary coverage and report whether each modeled claim has exercising evidence. The result names the gap — this invariant is untested — instead of hiding it inside an aggregate line percentage.

Learn more about this governance mechanism: requirements-based coverage.

One judgment-laden example: doc-derived tests

Doc-derived tests mark the other end of the spectrum. Whether every consequential prose claim has an adequate test requires judgment, so the measurement should direct review rather than drive an admission threshold.

Learn more about this governance mechanism: doc-derived test.

Let useful models drive machinery

An executable model earns more than documentation value when tools consume it directly. A deployment-topology model can tell a latency analysis which services and paths matter. A traceability graph can tell a coverage tool which code realizes a particular claim. A component graph can help a validator decide whether a finding is local to the current change or belongs to an unrelated part of the estate. This is one of Modeling's practical payoffs: the representation becomes a reusable query surface. It does not itself supply authority — the consuming validator or gate determines what consequence follows from the model-derived result.

A model can also scope a mechanism. DocAble grades findings by their distance from the current change in the component model, letting a local finding block while unrelated debt stays the responsibility of a broader gate.

Learn more about this governance mechanism: model-graded finding severity.

The grade is evidence read from the model at check time; the gate carries the authority. Read the number one level below the surface, where it names a target instead of a total.

4.4.5 The Operating Rhythm

Put the pieces together and operation becomes a recurring rhythm. The lifecycle tells you what healthy progress and important failure look like. Events bind the reaction to the transition, so the right procedure arrives when it matters. The runbook gives each state a sanctioned next move and separates deterministic execution from judgment. Metrics expose enough state to tell whether the loop is improving or merely busy. When a break recurs, governance conversion asks what later work should inherit from the lesson.

PRACTICE

Inset — When work stalls, diagnose before adding machinery

Repeated failure does not always call for another control. Before you build one, ask which part of the loop is limiting the work.

Table 4.4-1.

What may be limiting	The question	If that is the gap
Searcher	Can the available intelligence find a good candidate?	Change the model, tools, decomposition, or search strategy.
Model	Is consequential structure missing from the representation?	Make it explicit.
Oracle	Can the environment tell good candidates from bad?	Strengthen the evidence, validation, or simulation.
Target	Do we know what "good" even means?	Stop building and learn.

A stronger reasoner does not settle an unknown requirement; another validator does not supply a missing abstraction; more modeling cannot answer a product question whose answer must come from users or deployment. Diagnose what is limiting the work before adding machinery — the same distinction governance conversion already draws between failures of knowledge, obligation, evidence, evaluation, and consequence.

Run the rhythm long enough and the engineer increasingly decides what to represent, what to give authority, what evidence to require, and which judgments the environment should preserve. Part VI returns to that professional shift.

Brief. The written instructions handed to an agent for a task: what to do, where, and how "done" is judged; the artifact the pre-canned template produces.