6.4 Research Agenda

The theory proposes an explanation, the predictions say what should follow if it is right, and the scope conditions bound where those predictions should hold. This chapter turns the remaining uncertainty into a research program: what to measure, which comparisons could distinguish the proposed mechanisms, which quantities remain unknown, and what evidence would strengthen—or weaken—the account.

The equations in Parts V and VI state relationships to be measured, not fixed laws. The nine hypotheses collected in the previous chapter name the quantities a study would have to measure.

The hypotheses do not require nine unrelated studies. Three designs cover much of the program.

6.4.1 Parameterizing the Model

The simple models above deliberately leave their parameters unspecified. A research program can ask what determines them.

Let $T$ denote an engineering task or task family, $M$ the reasoning model, $R$ the representation available to it, and $H$ the harness and process through which reasoning and work are organized. For a fixed engineering environment, write

for the probability that one attempt produces an acceptable candidate, and

for the expected cost of that attempt. Cost need not mean inference cost alone. Depending on the study, it can include tokens, compute, latency, tool use, realization, validation, and human intervention.

For sustained work, we reuse the dynamic formulation from Section 6.1. The environment itself becomes a state variable: writing $E_i$ for the environment inherited at step $i$,

Here the equations serve a different purpose. Section 6.1 used them to express path dependence; the research problem is to measure how much that dependence matters in practice and which features of the environment drive it.

Alignment introduces a related quantity. For an Alignment strategy $A$, let

denote the probability that an unacceptable candidate escapes the mechanisms governing the relevant obligation. In the idealized case of a sound deterministic gate over a fully covered, decidable obligation, $q=0$ for that covered failure class. Statistical validators, incomplete coverage, mechanism defects, and semantic gaps produce weaker guarantees.

What experiments would estimate

The question is not whether agents have one fixed reliability or cost. It is how these quantities vary across engineering tasks, model capability, representation, harness and process, and Alignment strategy, and how those factors interact.

The distinction matters experimentally. A stronger model may increase $p$. A better representation may increase $p$, decrease $c$, or both. A longer reasoning loop may increase the probability of eventually finding an acceptable candidate while increasing total cost. An independent gate may leave the producer's $p$ unchanged while decreasing $q$. Modeling and Alignment can therefore produce similar observed improvements in final system quality through different mechanisms.

The goal is not one universal coefficient for agentic software engineering. It is to learn when capability, representation, harness and process, iteration, and enforcement substitute for or complement one another. Experiments can ask:

• Which factors systematically move $p$, $c$, and $q$?
• How do those effects vary by task family?
• When does representation substitute for model capability, and when does it complement it?
• When does dependence through the evolving engineering environment materially change retry cost, defect risk, or durable throughput?
• How much assurance can independent mechanisms provide without increasing producer capability?

The crossed experiments below provide one way to answer these questions. Vary reasoning model and engineered environment over matched task families, then isolate representation and Alignment while holding the other fixed. Measurements of attempts, inference and tool cost, latency, intervention, admission, defect escape, and durable throughput can distinguish capability effects, representation leverage, retry economics, and independent assurance rather than collapsing them into one measure of agent performance.

These experiments measure how representation helps realization, not its full engineering value. As model capability improves, a representation may contribute less to the cost or probability of producing an acceptable change: the reasoner may reconstruct the relevant relationship cheaply and reliably without it. The same representation may still matter for retaining engineering control by giving engineers and independent mechanisms an inspectable account of consequential system knowledge. Part VII takes up that second value directly.

6.4.2 Engineering Routing

These quantities suggest a further implication for automated software engineering.

Model routing typically treats the task as given: estimate its difficulty, then select a model whose capability and cost are appropriate to it. Software engineering allows another move: change the problem presented to the model.

Suppose a migration appears to require a frontier model to inspect hundreds of service configurations and reconstruct their dependencies. A trustworthy dependency model may turn the same work into a much smaller graph problem that a cheaper reasoner can handle reliably. Changing an authentication boundary may warrant a different strategy: the representation can still simplify the reasoning, but the consequence may justify the stronger model, additional evidence, and an independent admission mechanism as well. Engineering changes both the difficulty of the task presented to the reasoner and the assurance required around its result.

A difficult code-level reasoning task may become a simpler graph, state-machine, provenance, or constraint problem when an appropriate representation exposes the consequential relation directly. Better context or tooling may change the cost of reaching the answer. Iteration may purchase additional chances. Alignment may allow a less reliable producer to operate safely because an independent mechanism prevents covered failures from acquiring consequence. The model is therefore only one part of the realization strategy.

For an engineering task $T$, an automated system could in principle select the model $M$, representation $R$, harness $H$, Alignment strategy $A$, and retry budget $k$ jointly:

subject to an engineering assurance requirement

The task-specific bound ${\epsilon }_T$ matters. Renaming a local variable, changing a user-visible layout, modifying a billing calculation, and altering an authentication boundary need not warrant the same assurance strategy. Routing should reflect the obligations and consequences of the work, not merely an estimate of how difficult its implementation appears to a language model.

Call this engineering routing. A model router asks which reasoner should receive the task. An engineering router asks how the task should be engineered for realization: what the reasoner should reason over, which tools and context it should receive, how much iteration is economical, what evidence should be produced, and which independent mechanisms should govern admission.

This reverses an important assumption in conventional routing. Task difficulty need not be treated as fixed. Engineering can change it. A task that requires an expensive frontier model over an opaque repository may require less capability when a suitable representation exposes the relevant structure. Conversely, a strong model may make additional Modeling uneconomical for a simple or infrequent task. The best choice depends on the task, available capability, environment, assurance requirement, and recurrence.

The research implication is concrete. Measuring $p$, $c$, and $q$ across task families and engineered environments could show when each strategy is preferable. The question becomes not merely how difficult is this task for a model? but how cheaply can we engineer this task to be safe to delegate? That is the stronger consequence for automated software engineering.

6.4.3 Representation Selection, Induction, and Innovation

Part II offered six useful classes of engineering model, not a proof that those classes are necessary, sufficient, or minimal. Because representation creates leverage and can expand what Alignment can govern, choosing representations is itself a research problem.

A representation is useful only relative to the questions and consumers it serves. A sufficient representation family preserves enough information to answer the declared engineering questions and govern the declared obligations to the required standard; a minimal useful family removes representations whose added reasoning, assurance, or independence benefit does not justify their carrying cost. Mathematical minimality is not the target, since a strictly minimal set may be cognitively awful. We can test redundancy in principle: two representations that encode overlapping information can both be valuable when one serves a different question, reasoner, or independent assurance path, while a representation that changes none of the relevant answers, governable obligations, independent evidence, or reconstruction cost may be carrying cost without leverage.

A concrete example makes the idea clearer. Suppose engineers repeatedly reconstruct permitted service dependencies from handlers, configuration, and code. Adding an architectural graph costs something to create, reconcile, and maintain—but if later engineers and agents can answer the dependency question directly, and a validator can use the same representation to reject forbidden edges, the reconstruction and residual-assurance cost falls. A second representation that answers no additional question and adds no independent evidence adds cost without leverage.

Part IV already treated representation induction as a practical engineering move. An existing implementation can contain stable structure that no explicit model names. Static analysis, clustering, repeated failures, architectural context, and GenAI-assisted interpretation can help recover that latent structure and turn it into an explicit representation. This is not representation innovation in the stronger sense: the induced model makes explicit consequential structure that the existing system already contains.

A stronger research question is whether commodity intelligence can innovate representations. Given an engineering system and a set of recurring questions, failures, and decisions, can a reasoner propose a representation that makes important relationships tractable even when that representation was neither supplied in advance nor plainly latent in the implementation? Such a system might propose a new decomposition, intermediate state, relation, abstraction boundary, or combination of existing model forms because that representation better supports the engineering questions being asked. Together, induction and innovation form the ordered claim of H4: recover what is latent, then propose representations the existing set does not supply.

The problem can also be framed as learning. Given a distribution of engineering questions, failures, and decisions, one could ask which representation preserves the distinctions needed to answer them while suppressing irrelevant state—the question representation learning asks in machine learning. MAGE brings a strong engineering prior: software and systems engineering have spent decades developing abstractions for architecture, state, ownership, interfaces, requirements, quantities, provenance, and assurance, and the experience in this book suggests many are also unusually usable by foundation models. A complementary direction is to train models specifically for engineering abstractions ¹1. Safayat Bin Hakim et al., “Neuro-Symbolic Agentic AI: Architectures, Integration Patterns, Applications, Open Challenges and Future Research Directions,” Computer Science Review 60 (2026): 100902, https://doi.org/10.1016/j.cosrev.2026.100902. ²2. Miltiadis Allamanis et al., “Learning to Represent Programs with Graphs,” in “International Conference on Learning Representations,” special issue, International Conference on Learning Representations, 2018. ³3. Linhao Luo et al., “G-Reasoner: Foundation Models for Unified Reasoning over Graph-Structured Knowledge,” in “International Conference on Learning Representations,” special issue, International Conference on Learning Representations, 2026., which might let smaller models perform reasoning that currently requires expensive general-purpose intelligence. Where only machine reasoners and trustworthy mechanisms use a representation, it might trade human readability for compactness or stronger mechanical analysis, provided the results can be shown to humans where human judgment resumes. The open question is not whether learned representations should replace engineering models, but which representations best divide the reasoning among humans, agents, and mechanisms for the question at hand.

An experiment can distinguish induction from innovation. Compare expert-authored, machine-induced, and machine-innovated representations against both the raw implementation and one another. For induction, measure whether the proposed model faithfully recovers consequential structure already present in the system. For innovation, use held-out engineering questions and failures to test whether the proposed representation exposes useful structure that the existing representations did not. In both cases, measure reasoning success, reconstruction cost, correspondence failures, carrying cost, human comprehensibility, and benefit to later humans and agents.

6.4.4 Experimental Designs

Separating Modeling from Alignment

A factorial design can cleanly separate Modeling from Alignment. Hold the task, reasoning engine, and admission mechanism fixed while varying whether the producer receives the structured representation used by the environment. If Alignment provides the claimed independence, covered defect escape should remain unchanged; if Modeling provides the claimed leverage, attempts, inference cost, latency, or intervention required to reach admission should fall when the representation is available.

Multi-project longitudinal design

This design should vary or observe engineered-environment quality while tracking durable throughput, defect escape, and human-attention burden over weeks or months. The strongest version is a crossed experiment: several reasoning engines from weaker to frontier, several otherwise comparable environments from thinly to richly engineered, and the same task families across every cell. The results would distinguish three possibilities. If richer environments primarily substitute for raw model capability, weaker models should close part of the gap in stronger environments. If the two are complementary, the strongest models should gain most from strong environments. If environment quality contributes little, the cells should move mainly with model capability. This design directly tests H1, H8, and H9.

The crossed experiment should also measure where human work occurs and how much productive capacity resides outside the agent. Hold the task family and reasoning engine fixed, vary environment quality, and record how much human attention remains across coding, debugging, test construction, review, environment setup, integration, recovery, and other activity classes, alongside task scale, reconstruction cost, intervention burden, defect escape, and durable throughput. A richer environment that reduces only code-production time would support the narrower productivity account; one that moves several classes of realization work out of repeated human execution while maintaining quality would support MAGE's broader account. Kumar et al.’s sixteen activity classes provide an existing measurement vocabulary, avoiding the need for another MAGE-specific taxonomy ⁴4. Sukrit Kumar et al., “Time Warp: The Gap between Developers' Ideal Vs Actual Workweeks in an AI-Driven Era,” in “Proceedings of the 2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-Seip),” special issue, Proceedings of the 2025 IEEE/ACM 47th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), 2025, 12–22, https://doi.org/10.1109/ICSE-SEIP66354.2025.00007..

Subsystem-level design

Compare engineering surfaces that differ in representation and governance while holding as much surrounding repository context constant as possible. Track model correspondence, reconstruction effort, intervention, churn, and defect escape. Stepped introduction of a model or mechanism is particularly useful because the same subsystem supplies its own pre-intervention baseline. This tests whether representation leverage, determinization, and mechanized assurance appear where the theory predicts, rather than merely accompanying well-run projects.

Event-centered longitudinal design

Follow one failure class through its whole life: first occurrence, repeat exposure, structural recognition, local repair, any durable conversion, propagation to later work, later recurrence, and eventual maintenance or retirement of the resulting asset. This is the strongest way to separate "the engineer got better" from "the environment learned." Applied across organizations, it also tests whether authority pathways determine whether a local diagnosis becomes shared engineering capital.

What the designs still need

All three designs require credible measures of durable throughput, human-attention burden, environment quality, and engineering-capital return. Until those are available, support ratio, mechanism count, and repository size should remain descriptive rather than proxies for maturity or value.^** Candidate measures such as development completeness, redundant-encoding density, and agent-legibility remain exploratory and are not part of MAGE's canonical vocabulary.

Commodity intelligence may make some of these experiments unusually tractable: the reasoning engine, repository state, task family, and engineered environment can be held approximately fixed while one representation or mechanism varies.

The theory may also have a broader domain than software engineering. Its main mechanisms are not inherently code-specific: purposeful representations can reduce reconstruction, explicit obligations can support independent evidence and enforcement, and recurring judgment can become durable structure. Other engineering disciplines already rely heavily on models, analyses, tolerances, and assurance; some forms of professional knowledge work likewise use structured representations, rules, evidence, and approval. This book does not test that generalization. Part VII gives several concrete examples. Testing MAGE beyond software engineering remains part of the research agenda.