Skip to content

§6.2 What the Theory Predicts

The theory makes four connected propositions. A useful representation can change the work required to reason about a system. It can also make selected obligations available for independent enforcement. When the resulting knowledge and mechanisms remain useful, later work inherits engineering capacity. Additional agentic capacity then acts through that inherited environment, amplifying its fit or exposing its inadequacy.

These are theoretical relationships, informed by the cases rather than established as general findings. Each needs a mechanism, conditions under which it should matter, and a way evidence could show it to be wrong. §6.4 develops a compact set of comparisons for that purpose.

6.2.1 P1: Representation Leverage

Trustworthy, task-relevant representations can reduce reconstruction and extend the work that humans and agents can complete at a given quality and resource budget. They do so by exposing the distinctions and relationships needed for the task while suppressing implementation detail that would otherwise have to be reconstructed. The benefit depends on what the representation preserves, how it is used, and what it costs to build and maintain.

DocAble encountered this problem when representing large documents for remediation. One document produced roughly 590,000 tokens of reasoning representation against a 272,000-token context window. A more costly foundation model with a larger context window was one possible response. Another was to change the representation: windowing and whole-document compression reduced the information required for each judgment. The engineering choice was not simply how much context to purchase, but how much reasoning the representation could save.

Research on external representations has long shown that how information is organized changes the work required to reason with it 12.1. Jill H. Larkin and Herbert A. Simon, “Why a Diagram Is (Sometimes) Worth Ten Thousand Words,” Cognitive Science 11, no. 1 (1987): 65–100, https://doi.org/10.1111/j.1551-6708.1987.tb00863.x.2. Jiajie Zhang and Donald A. Norman, “Representations in Distributed Cognitive Tasks,” Cognitive Science 18, no. 1 (1994): 87–122, https://doi.org/10.1207/s15516709cog1801_3. Models apply that principle to engineering questions by exposing consequential structure directly. A dependency graph, for example, can make permitted service relationships available without requiring an engineer or agent to reconstruct them from scattered implementation details.

This can shorten the reasoning path, rather than merely make each step cheaper. The environment preserves some work that would otherwise be repeated before the engineering question could even be addressed. State machines expose transitions; contracts expose obligations; assurance arguments relate claims to evidence. Such representations may bring a demanding task within the human-agent system’s effective reasoning horizon, or make an already tractable task cheaper. They cannot provide distinctions they omit, and an agent still has to use the representation appropriately.

Once a model exists, maintenance becomes a second problem. “Keep the model equal to the code” is too crude. A model of permitted dependencies should disagree with code that violates them. A requirements model may intentionally describe an intended state the current implementation has not reached. The engineering problem is to state and maintain the correspondence each model claims—for architecture, behavior, process, invariants, lineage, or other views—cheaply enough to expose meaningful divergence near the change that created it.

A synchronized model can still be useless for the question at hand. Test suites provide the familiar warning: an oracle may faithfully reproduce implementation behavior while failing to express the requirement.** Automation that optimizes against a visible oracle can satisfy that oracle while violating properties it does not check 4.4. Edward K. Smith et al., “Is the Cure Worse Than the Disease? Overfitting in Automated Program Repair,” Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE), 2015, 532–43, https://doi.org/10.1145/2786805.2786825. A recent audit of a coding benchmark likewise identified flawed tests alongside contamination 5.5. OpenAI, “Why SWE-bench Verified No Longer Measures Frontier Coding Capabilities,” OpenAI, February 2026, https://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/. Neither fidelity to an artifact nor success against a visible test establishes that the intended obligation was adequately represented. Model evaluation must therefore ask both whether the claimed correspondence is trustworthy and whether the representation preserves the distinctions needed by the decisions that use it. Stale or poorly chosen models can confidently direct reasoning toward the wrong result, adding cost rather than saving it.

The proposition does not require engineers to design every representation in advance. Chapter 4 showed how implementation signals and commodity intelligence can help recover latent models. Repeated structures may reveal a domain concept that the implementation already embodies but has never named. Recovery makes that structure explicit, leaving engineers to determine whether the observed behavior is the behavior the system ought to have.

A more ambitious possibility is representation innovation. A reasoner may propose a different decomposition, relation, or state space that makes an engineering question tractable, even when that representation was neither supplied by an engineer nor plainly latent in the implementation. The innovation is useful when it exposes consequential structure that existing representations did not make practically available. A novel diagram alone would not establish that result.

Discovery does not confer trust. Like any model in Chapter 2, a machine-proposed representation claims a relationship to the system and must earn the trust placed in that claim. A recovered model can be checked against the implementation it purports to summarize; a genuinely novel abstraction poses the harder question of what evidence establishes that it preserves the distinctions required by its intended engineering use. The stronger the reasoning, assurance, or enforcement built on the representation, the stronger that evidence must be.

P1 would be challenged if these representations add interpretation and maintenance work without lowering reconstruction cost, preserving quality, or extending the scale of work completed. The comparison must include their construction and useful life. More capable agents may reduce the need for some representations by reconstructing the relevant knowledge cheaply; they may also make other representations economical to discover and maintain. The proposition concerns the value of the representation for the work, not the quantity of models an organization accumulates.

6.2.2 P2: Representation-Enabled Enforcement

An explicit representation can make a selected obligation mechanically decidable, allowing Alignment to enforce it independently of the producing agent’s compliance. Modeling supplies a form in which the property can be evaluated; Alignment supplies the mechanism that makes the result consequential. These contributions can reinforce one another, but neither should be mistaken for the other.

Suppose an architectural rule forbids a service from depending on a database directly. Giving the agent a dependency model can help it choose a permitted design. A separate gate can compare the resulting dependency graph with the rule and reject a change that violates it. The first intervention helps the agent produce acceptable work; the second governs what is admitted when production goes wrong. The gate can still reject the forbidden dependency even if the agent never consulted the model.

The distinction also explains why equal protection can come at different cost. A producer without the model may repeatedly propose changes that the gate rejects. Supplying the model can reduce these collisions without changing the gate’s guarantee. Conversely, a clear model without enforcement still leaves fulfillment dependent on the producer and whatever subsequent review occurs. Table 6.2-1 separates these possibilities.

Table 6.2-1. Modeling and Alignment are separable levers. Modeling changes the economics and tractability of reasoning; Alignment changes whether the result can acquire consequence. The cells describe their primary contributions, not an exhaustive account.
Modeling ↓ / Alignment →No AlignmentAlignment
No ModelingRepeated reasoning from a poor surface; residual failure can escapeCovered failure cannot escape, but rejection and regeneration may be expensive
ModelingSatisfactory realization becomes cheaper or more likely, but residual failure can still escapeSatisfactory realization becomes cheaper while covered failure remains independently blocked

The strongest assurance claim is local to a covered property. If a sound admission mechanism rejects every violation of that property, admitted work cannot contain that violation, regardless of how often the producer makes the mistake. That does not establish that the property is the right one, that the representation covers every consequential concern, or that the mechanism’s implementation is sound. Those remain engineering obligations.

Most assurance is less decisive. Tests can miss failures, validators can reject useful work, and statistical checks can be uncertain. A lower escape rate may simply reflect rejecting more candidates. Comparisons therefore need independent assessment of correct delivery and false rejection as well as escaped failure, with the cost of sustaining delivery included. Independence here concerns the mechanism’s ability to act without relying on the producer to obey; it does not mean that every source of error is statistically independent.

Modeling can change which obligations admit such mechanisms. The dependency rule may be difficult to assess by repeatedly asking a reasoner to interpret source files, configuration, and conventions. Once an appropriate model exposes the relevant services and dependency edges, checking the forbidden relation becomes a repeatable predicate. The engineer still has to establish the graph’s claimed correspondence with the system; the mechanical check is only as relevant as that correspondence makes it.

The determinization frontier separates judgments that require fresh interpretation from those that can be evaluated by a repeatable procedure. Alignment can mechanize an obligation that is already decidable. Modeling can change the representation so that a previously difficult judgment becomes decidable, after which Alignment can enforce it (Figure 6.2-1). This distinction is also economic. On one side are judgments the factory must make repeatedly through probabilistic reasoning; on the other are properties it can evaluate through reusable machinery.

The Determinization Frontier Repeated judgment follows either an already-decidable route or a route through Modeling to change the representation and expose a decidable property. Both join before Alignment, where sensors provide evidence and validators evaluate it before a gate applies the result. REPEATED JUDGMENT property requires reconstruction property already decidable MODELING Change representation Decidable property (predicate over model) ALIGNMENT Encode and evaluate the predicate Sensors and Validators Gate REPEATABLE DECISION The determinization frontier: change what is decidable (Modeling), then bind the decision (Alignment).
Figure 6.2-1. The Determinization Frontier. Alignment can bind a property that is already decidable. Modeling can change the representation so that a property previously requiring semantic reconstruction becomes decidable and can then be bound by Alignment. Both routes move repeated judgment into machinery.

DocAble illustrates the distinction. Determining what an image means and writing an adequate description require semantic judgment. Once a description is attached to an object in the document model, checking its presence and placement can be mechanical. That check does not establish that the description is adequate. Modeling has made a narrower property decidable, and Alignment can enforce it without repeating the original semantic judgment.

The frontier does not imply that every judgment can become mechanical. Better representations may improve human or agent evaluation even when no definitive predicate is available. Where a property does become mechanically decidable, the question is whether the resulting evidence and enforcement are trustworthy enough for the consequence attached to them. P2 would be challenged if the proposed representation fails to make the important property checkable, or if apparent assurance gains disappear once coverage, false rejection, correct throughput, and cost are compared fairly.

6.2.3 P3: Governance Conversion and Engineering Capital

When engineering judgment becomes trustworthy, reusable structure, later work can inherit productive capacity rather than purchase the same reasoning again. Governance conversion is one route to this engineering capital. A recurring failure can become a validator; an architectural distinction can become a model; an operational lesson can become a procedure. Up-front engineering and lessons transferred from other systems can create the same kind of inheritance before a local failure occurs.

The return is broader than defect reduction. An architectural model may let an engineer assess a change that would otherwise be too expensive to understand. A typed operation can make a previously risky mutation routine. A validator can provide evidence earlier, and a recovery procedure can make a failed experiment cheap enough to attempt. In each case, earlier work has changed what later engineers and agents can accomplish, what they must reconstruct, or what risk they must bear.

This is why the metaphor is capital rather than memory. A record can preserve information without changing the productive capacity of later work. An engineering asset becomes capital when later work can use it to gain capability, reduce cost, or control risk. A useful record can certainly do that; its format does not decide the question. The test is what the next engineer or agent can inherit.

That inheritance need not depend on the continued presence of the engineer who supplied the original judgment. When an architectural distinction becomes a trustworthy model, a review judgment becomes a validator, or an operational lesson becomes a reliable procedure, some of the originating engineer's productive capacity now lives in the GEE. A later engineer or agent can benefit from the result without reconstructing the reasoning that produced it.

The transfer is necessarily partial. Durable structure carries only the judgment it successfully represents or mechanizes. When requirements change, a new failure appears, or the representation itself becomes inadequate, engineering judgment is needed again. Engineering capital therefore does not eliminate expertise; it changes how often the organization must pay again for expert judgment on the same question.

The organizational path matters. If an engineer diagnoses a recurring boundary error but can only repair the current patch, later agents may encounter the same problem. If that diagnosis changes the shared model and its validator, later work can inherit the distinction without waiting for the engineer to explain it again. The theory therefore predicts more consistent propagation where diagnosis is connected to authority over shared engineering structure. It does not require all authority to reside in one person.

The analogy to technical debt is intentional. Debt makes future change more expensive; capital makes future change more productive. Neither status is permanent. Capital depreciates as systems, obligations, and tools change. A validator can become noise, a model can drift, and a once-useful constraint can begin blocking legitimate work. Governance therefore includes maintenance, reconciliation, and retirement. Accumulation is not the objective; productive capacity is.

Estimating the return on engineering capital requires comparing later work with what the same work would have cost without the asset, under comparable capability, quality, and task conditions. The comparison must include construction and maintenance, the asset’s useful life and loss of fit, and the value of human attention. Changing foundation-model capabilities and prices affect both sides: cheaper agents may lower the cost of constructing and maintaining an asset, but also the cost of reconstructing the same knowledge on demand. Inherited capacity pays where the value of the work it enables or saves justifies those costs over its useful life. The proposition identifies what must be measured without claiming known coefficients.

This account also connects to Storey’s distinction among technical, cognitive, and intent debt in AI-assisted software 3.3. Margaret-Anne Storey, “From Technical Debt to Cognitive and Intent Debt: Rethinking Software Health in the Age of AI,” Queue 24, no. 2 (2026), https://doi.org/10.1145/3807966. A team can accumulate implementation faster than it maintains shared understanding or preserves why the system should behave as it does. Explicit models and evidence can give consequential understanding and intent a durable place to live. They do not remove the need for expertise when requirements change or the representation itself becomes inadequate.

P3 would be challenged if later work gains no useful capacity over local repair or on-demand reconstruction once the asset’s full costs are counted. A decline in recurrence alone is insufficient: the engineer may have become more experienced, or the work may have changed. Reuse by later engineers or agents, including work performed without the originator, provides a stronger test of what the GEE actually preserves. The research agenda returns to how these comparisons can be made.

6.2.4 P4: Governable Acceleration

The effect of additional agentic capacity depends on the GEE through which it acts. Where representations and assurance adequately support the work, added capacity can produce more durable progress without proportionate growth in human inspection. Where the GEE fits poorly, the same capacity can instead produce more churn, intervention, or escaped failure. Raw implementation activity may rise in either case.

Consider two agents making more changes against the same service boundary. An explicit dependency model may spare both agents repeated reconstruction; a gate may keep the covered architectural violation out of admitted work. Those mechanisms can support a higher rate of acceptable changes. If the model is stale or the gate enforces the wrong rule, more activity can instead multiply the resulting repair and review. The proposition concerns these mechanisms and their fit to the tasks, rather than a blanket advantage for a GEE described as stronger.

To test that claim, GEE quality has to be characterized independently of the outcomes it is supposed to explain. The four dimensions in §6.1 provide questions about representation usefulness, obligation coverage, coherence, and economy. They do not yet supply a validated universal score. Counting checks would be easier, but an overgrown collection of conflicting checks can make the GEE less useful. A comparison must identify which relevant properties differ, then observe what additional capacity does under those conditions.

Several outcomes are plausible. A good representation might let a weaker model perform work that previously required a stronger one. A stronger model might benefit more because it can exploit relationships that the weaker model cannot. Or capable agents might reconstruct the required knowledge so cheaply that maintaining the representation contributes little to production efficiency. Independent assurance may still matter in that last case, because being able to produce a result does not establish that it should be admitted. These possibilities are reasons to compare capability and GEE together, not to assume that more Modeling must always accompany better agents.

Acceleration can also change how quickly inadequacy becomes visible. Failures that would arrive months apart may occur close enough together for an engineer to recognize their common structure. That creates an opportunity for governance conversion, not an automatic learning mechanism. Without diagnostic capacity, the organization may simply produce patches faster. Without authority to change the shared GEE, a correct diagnosis may remain local. Useful adaptation can also be subtractive when the pressure comes from obsolete constraints or duplicated controls.

An improved GEE can in turn make larger tasks and greater concurrency economical. The organization may then attempt work that exposes different obligations and new failure classes. This is the expanding task frontier in Figure 6.1-2: engineering capital can support more ambitious work, which renews the need for diagnosis and adaptation. It does not imply unlimited improvement or that every increase in capacity is worth purchasing.

P4 would be challenged if added capacity has essentially the same effect on durable throughput, escape, and human-attention burden across otherwise comparable GEE conditions, or if the apparent advantage disappears when task difficulty and quality are matched. Faster approvals alone are not evidence of governable acceleration. The relevant result is more useful work admitted with adequate assurance, while the costs and unresolved demands remain visible.

Table 6.2-2 collects the four propositions. They overlap in an engineered system, but they ask different questions: what reasoning the representation changes, what an independent mechanism can enforce, what later work inherits, and how those conditions affect the use of additional capacity.

Table 6.2-2. Four theoretical propositions. Each identifies a mechanism, its conditions, and a comparison needed to assess it. These are claims to investigate, not measured effects.
PropositionMechanismWhat a comparison must establish
P1 Representation leverageTask-relevant structure reduces reconstructionReasoning reach or cost at matched quality, including fidelity and upkeep
P2 Representation-enabled enforcementRepresentation makes an obligation checkable; Alignment enforces itEscape, false rejection and correct delivery for adequately covered obligations
P3 Governance conversion and engineering capitalLater work inherits useful engineering judgmentReused capacity and propagation net of construction, carrying cost and lost fit
P4 Governable accelerationCapacity acts through the environment’s fit to the workCapacity-by-environment effects on durable throughput, escape and human attention

Works Cited

  1. Larkin, Jill H., and Herbert A. Simon. “Why a Diagram Is (Sometimes) Worth Ten Thousand Words.” Cognitive Science 11, no. 1 (1987): 65–100. https://doi.org/10.1111/j.1551-6708.1987.tb00863.x.
  2. Zhang, Jiajie, and Donald A. Norman. “Representations in Distributed Cognitive Tasks.” Cognitive Science 18, no. 1 (1994): 87–122. https://doi.org/10.1207/s15516709cog1801_3.
  3. Storey, Margaret-Anne. “From Technical Debt to Cognitive and Intent Debt: Rethinking Software Health in the Age of AI.” Queue 24, no. 2 (2026). https://doi.org/10.1145/3807966.
  4. Smith, Edward K., Earl T. Barr, Claire Le Goues, and Yuriy Brun. “Is the Cure Worse Than the Disease? Overfitting in Automated Program Repair.” Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE), 2015, 532–43. https://doi.org/10.1145/2786805.2786825.
  5. OpenAI. “Why SWE-bench Verified No Longer Measures Frontier Coding Capabilities.” OpenAI, February 2026. https://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/.
© James C. Davis, 2026–present