§1.4 Agentic Machinery and Its Properties
In developing DocAble, I deliberately inspected almost none of the code the agents produced. This was partly a practical necessity, given the volume of implementation, and partly an engineering experiment: could I retain informed control over a system without personally examining its implementation? The answer depended on understanding what the agents could do, where their limitations lay, and how their work could be directed and checked.
Engineering methods must fit the machinery they govern. Let us therefore examine where these capabilities came from, the properties that matter to engineering, and the harnesses through which agents act.
1.4.1 From Curiosity to Engineering Capability
As a professor of computer engineering, part of my job is to evaluate emerging technologies and decide what they mean for engineering practice and education. This includes deciding whether new techniques belong in undergraduate or graduate courses. I had followed the development of large language models (LLMs) through 2024 and 2025, but my assessment was that they were useful engineering aids rather than a substantial means of implementing software.
That assessment changed in late 2025. One team in my software engineering course completed its project—approximately 30,000 lines of code—over a weekend and earned the highest score on the course autograder. When I asked how they had accomplished this, the students explained that they had used Claude Code.
The result established a consequential capability: students could now use an agent to produce a substantial working implementation in a fraction of the time I would previously have expected. An autograder measures the properties its tests exercise; the score alone could not establish that the software was well engineered, maintainable, or suitable for production. Whatever one believed about the internal reasoning processes of LLMs, their observable capabilities had become difficult to dismiss.
My interest was piqued. If students could produce that much working software over a weekend, what would happen when an experienced engineer used the same machinery to develop and maintain a production system? And what engineering methods would make that possible without sacrificing control?
1.4.2 Where Commodity Intelligence Comes From
The coding agents available in 2026 are built primarily around large language models, which serve as their foundation models. These acquire general capabilities through training on large collections of text, software, and other data. For coding foundation models, publicly available source code, including open-source repositories, has been an important source of training material 1.1. Mark Chen et al., Evaluating Large Language Models Trained on Code, arXiv preprint arXiv:2107.03374, 2021, https://arxiv.org/abs/2107.03374. Training mixtures can also include licensed material, and their full composition is not always disclosed 2.2. OpenAI, GPT-4 Technical Report, arXiv preprint arXiv:2303.08774, 2023, https://cdn.openai.com/papers/gpt-4.pdf. Training typically begins with a prediction objective: given a sequence of tokens, predict the next one. Tokens are units of representation, often words, parts of words, or punctuation. Training adjusts numerical parameters, or weights, to improve these predictions.** Engineering predates machine learning in its use of models as purposeful representations of systems. Machine learning later adopted the same word for a learned approximation of a function—also reasonably a model, but a different kind. This book gives the engineering usage precedence: unqualified model means a purposeful representation of an engineered system or some property of it. I therefore use foundation model for the trained machine-learning system and agent for the acting system built around one through a harness, tools, and working state. Where the distinction matters, I will not ask the reader to infer which “model” I mean.
Additional training can improve how a foundation model follows instructions and responds to human requests 3.3. Long Ouyang et al., “Training Language Models to Follow Instructions with Human Feedback,” Advances in Neural Information Processing Systems 35 (2022): 27730–44, https://proceedings.neurips.cc/paper_files/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract-Conference.html. At inference time, the trained foundation model processes the context supplied to it and generates a response. An agentic harness turns some of those responses into actions, such as searching a repository, modifying files, running tests, or invoking development tools. Tool results can then be returned as context for further reasoning 4.4. Shunyu Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models,” International Conference on Learning Representations, 2023.
The origins of these capabilities matter when agents work on a particular system. A foundation model trained on public software may know programming languages, libraries, common architectures, and established engineering practices. Its training ordinarily will not have included the current state of the production system an engineer asks it to modify. That system may be proprietary, may have changed since training, or may contain local design decisions that general programming knowledge cannot supply.
For large, evolving systems, a seemingly local change may depend on an architectural boundary established years earlier, a configuration convention shared across services, or an operational constraint that appears nowhere near the code being edited. The agent must recover the relevant knowledge from the existing codebase, documentation, configuration, and other available evidence. General programming knowledge must be connected to the particular system and the obligations its changes must preserve.
Engineers can influence a language-based agent in two broad ways. They can change the foundation model through training or fine-tuning that modifies its weights. They can also change what the agent receives and the environment in which it operates: instructions, system knowledge, tools, permissions, and feedback. Training a frontier foundation model requires resources far beyond those available to an ordinary software engineering project. Fine-tuning is more accessible, especially with techniques that update only a small fraction of its parameters 5.5. Edward J. Hu et al., “LoRA: Low-Rank Adaptation of Large Language Models,” International Conference on Learning Representations, 2022, https://openreview.net/forum?id=nZeVKeeFYf9. It still requires suitable training data, evaluation, and maintenance, and may not be supported for the particular foundation model an engineer wishes to use. By contrast, engineers can often configure instructions, supply relevant project knowledge, expose tools, and establish checks without modifying the underlying weights.
This distinction establishes the principal locus of control in MAGE. The engineer acquires general-purpose intelligence as a service or software component, then engineers the environment in which that intelligence performs consequential work. Figure 1.4-1 places these two sources of engineering capability and control in one system: the foundation model supplies general reasoning capabilities, while the harness and governed engineering environment connect those capabilities to the particular system being developed.
1.4.3 Do LLMs Reason?
Predictive training can produce useful problem-solving capabilities and representations of the processes underlying a sequence. The scientific question is what a model learns in order to predict tokens. A game’s moves depend on its board state, and a program specifies changes to computational state. Experiments can investigate whether learning to predict such sequences also produces representations of those relationships.
Early debate helped sharpen the question. Bender and colleagues’ On the Dangers of Stochastic Parrots challenged the identification of linguistic fluency with understanding 6.6. Emily M. Bender et al., “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021, 610–23, https://doi.org/10.1145/3442188.3445922. Subsequent behavioral studies documented increasingly broad capabilities. In Sparks of Artificial General Intelligence, Bubeck and colleagues examined an early GPT-4 through mathematical problem solving, programming, and other tasks. Their demonstrations included constructing programs from instructions and combining knowledge from different domains 7.7. Sébastien Bubeck et al., Sparks of Artificial General Intelligence: Early Experiments with GPT-4, arXiv preprint arXiv:2303.12712, 2023, https://arxiv.org/abs/2303.12712. These experiments established observable problem-solving behavior; other researchers investigated the representations supporting prediction.
Li and colleagues trained a transformer to predict legal Othello moves from game transcripts. Training supplied sequences of moves, without explicitly supplying board positions or rules. Classifiers called probes could recover the board state from the network’s activations. The researchers then altered those activations to represent a counterfactual board and observed corresponding changes in move predictions 8.8. Kenneth Li et al., “Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task,” International Conference on Learning Representations, 2023, https://openreview.net/forum?id=DeG07_TcZvT. Nanda and colleagues identified a simpler, linear representation relative to the current player and controlled predictions through vector changes to the activations 9.9. Neel Nanda et al., “Emergent Linear Representations in World Models of Self-Supervised Sequence Models,” Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP (Singapore), 2023, 16–30, https://doi.org/10.18653/v1/2023.blackboxnlp-1.2. The experiments connected a learned representation of the game’s state to the computation producing its next move.
Jin and Rinard studied a particularly relevant case for software engineers: programs in a small language for navigating a two-dimensional grid. Training examples contained input-output specifications followed by programs. Intermediate execution states were absent from the training targets, yet probes could recover information about them from the language model’s internal states. Those states describe the program’s behavior between its visible inputs and outputs. The researchers tested the specificity of this representation by changing the interpretation of program operations while preserving syntax. Better decoding under the original semantics than under these alternatives provided evidence that semantic structure had been learned by the language model, rather than supplied entirely by the probe 10.10. Charles Jin and Martin Rinard, “Emergent Representations of Program Semantics in Language Models Trained on Programs,” Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 235 (2024): 22160–84, https://proceedings.mlr.press/v235/jin24e.html.
These studies examine different systems and kinds of evidence. GPT-4 demonstrations establish breadth of behavior; the controlled game and programming experiments identify representations and test their role or specificity. Together they demonstrate useful capabilities and learned structure. Establishing the dependability of agents working on arbitrary production software remains an engineering task.
For this book, reasoning is a functional capability: using available information to draw inferences, evaluate alternatives, and work toward a solution. LLM-based systems can supply enough of that capability to undertake substantial engineering work. By late 2025, my students had provided a compelling practical demonstration of its usefulness. The opportunity was to put that capability to work under informed engineering control.
1.4.4 Four Properties of Commodity Intelligence
Four properties of contemporary foundation models matter to the engineering argument.
Broad semantic reasoning. A foundation model can interpret instructions, synthesize information, draw inferences, and search for solutions across open-ended domains. Give it an abstraction—a schema, state table, architectural model, or policy—and it can work with that abstraction rather than merely execute a fixed script. This makes it possible to delegate implementation decisions that engineers have not specified in advance.
A plausible decision may nevertheless be inappropriate for the system at hand. An agent may select a familiar architecture or design pattern that works well in other systems but violates a local constraint, introduces unnecessary complexity, or conflicts with an established design decision. Its general knowledge must therefore be supplemented with the knowledge and obligations of the particular system.
Probabilistic output. A foundation model can resolve consequential engineering questions incorrectly. Repeated invocations may produce different implementations or judgments, and its confidence in its own result is not independent evidence that the result is correct. An agent may also fail consistently when its attempts share an incorrect assumption. Asking another foundation model to review the work can improve reliability, but the errors of the producer and evaluator may be correlated.
The engineering response is to establish obligations and evidence outside the producing judgment. Types and permissions can constrain selected actions or states; validators and tests can supply evidence about results; gates can use that evidence to refuse unacceptable work. Each mechanism has its own assumptions and coverage limits, so engineers must still decide which properties it adequately establishes.
Bounded working state. The information available to a foundation model during an invocation is finite. An agentic harness can extend its effective working state through retrieval, external memory, compaction, and repeated tool use, but information needed for a particular decision must be available when that decision is made.
Large software systems exceed what can be presented in a single reasoning episode. Agents must therefore select and reconstruct relevant system knowledge, just as human engineers use abstractions and external representations to manage complexity. Explicit models can reduce this reconstruction burden by preserving consequential relationships in a form suited to the questions being asked.
Inexpensive repeated reasoning. Human engineers are fallible reasoners too. They misunderstand requirements, overlook interactions, make incorrect assumptions, and introduce defects. Software engineering has long responded with specifications, types, tests, reviews, and other means of checking work independently of its author. Artificial reasoners introduce a new source of implementation capacity, not a new need for engineering assurance.
Their advantage is the speed and low cost at which they can explore alternatives. An agent can propose an implementation, run checks, revise its work, and try again, often many times in the time a human engineer would need for one attempt. This is particularly valuable when the engineer can state what constitutes an acceptable result. The agent can explore how to satisfy the requirement while the engineered environment supplies criteria for evaluating its attempts.
These capabilities make natural language a productive interface for delegating implementation. In the more radical practice known as vibe coding, developers accept and revise generated programs without closely examining their implementation. The interface is convenient, but consequential decisions can remain implicit: a request may admit several plausible architectures or error-handling policies, and later requests may displace earlier requirements.
The FlowCheck study illustrates the latter difficulty. Three coding systems were asked to build a speaker scheduler, initially with a rule that each speaker could occupy only one time slot. A later request allowed several speakers to share a slot. All three systems lost the earlier one-slot-per-speaker rule while making this change 11.11. Reya Vir et al., “FlowCheck: Helping End-Users Specify and Verify Intent in Vibe-Coded Web Apps,” Proceedings of the 2nd ACM SIGPLAN International Workshop on Language Models and Programming Languages (Oakland, CA, USA), 2026, https://doi.org/10.1145/3843750.3843844. The new behavior was compatible with preserving that rule; the earlier obligation needed to survive the later instruction. Making consequential decisions explicit outside the conversation gives engineers and agents a durable basis for implementation and evaluation.
1.4.5 Engineering the Environment
An agentic harness is the runtime that turns the foundation model’s reasoning into action. It manages the context supplied to the foundation model, invokes tools in response to its outputs, and returns their results for further reasoning. It may also provide memory, task management, parallel execution, and hooks at which additional code can run. These facilities support three complementary capabilities: action through tools, context management to assemble and maintain working state, and interposition at points where the environment can observe, constrain, redirect, or deny an action.
The engineered environment is the project’s deliberate arrangement of representations, tools, processes, permissions, and checks around that machinery. It includes the existing codebase and the knowledge embodied in it, together with the requirements, architecture, documentation, models, tests, and operational infrastructure through which the engineer understands and governs the system. The distinction is one of roles: a hook facility belongs to the harness; installing a project check through it is part of engineering the environment. For example, the hook may run a check that refuses edits to generated files. Build checks and review procedures also belong to the environment even when they operate independently of the harness.
Recent work on harness engineering similarly treats autonomous capability as a property of the foundation-model–harness–environment system 121314.12. Hailin Zhong and Shengxin Zhu, AI Harness Engineering: A Runtime Substrate for Foundation-Model Software Agents, arXiv preprint arXiv:2605.13357, 2026, https://arxiv.org/abs/2605.13357.13. Justin Young, “Effective Harnesses for Long-Running Agents,” Anthropic, November 26, 2025, https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents.14. Garry Tan, “Thin Harness, Fat Skills,” April 9, 2026, https://github.com/garrytan/gbrain/blob/master/docs/ethos/THIN_HARNESS_FAT_SKILLS.md. MAGE concentrates on the environment for two reasons. First, it is where engineers can exercise substantial control without developing a foundation model themselves. Second, it is where engineering knowledge and obligations can persist independently of any particular reasoning episode. Maintaining an architectural model that identifies components and their obligations, choosing which tools an agent may invoke, and deciding which checks must pass are all ways to exercise that control.
The environment can influence the agent’s decisions by supplying relevant knowledge and instructions. It can also constrain the agent’s actions and evaluate its results. These functions are distinct. Supplying an architectural model may help an agent preserve a dependency boundary; a validator can detect a forbidden dependency, and a build gate can refuse the change even when the agent fails to follow the model. This is the division between Modeling and Alignment developed in the following chapters. Modeling makes consequential knowledge and intent explicit, reducing the amount of system understanding that must be reconstructed during each task. Alignment makes selected obligations enforceable through constraints that restrict actions or states, sensors that produce evidence, and gates that evaluate evidence and control whether work may proceed.
A language model may acquire internal representations of a system through training or inference. MAGE instead asks engineers to maintain selected representations explicitly, outside the reasoner, where they can be inspected, corrected, shared, and used to enforce obligations. This does not mean that external models are inherently more accurate. It is a matter of engineering authority: an engineer can revise an architectural model and check its consistency with the system. Although engineers generally cannot inspect and correct the corresponding representations inside a frontier LLM, they do not need to.
The engineer’s responsibility has always been to determine how a system ought to behave and design how it ought to be constructed. MAGE provides a way to exercise that responsibility while benefiting from the generative capabilities of AI.
1.4.6 The Reasoning Horizon
Scale has always created problems. An engineer may be able to hold a small program in mind, but a large system soon exceeds that capacity. Human working memory is limited: we can actively hold and work with only a small amount of information at once, even when we have much more knowledge in long-term memory 15.15. Nelson Cowan, “The Magical Number 4 in Short-Term Memory: A Reconsideration of Mental Storage Capacity,” Behavioral and Brain Sciences 24, no. 1 (2001): 87–114, https://doi.org/10.1017/S0140525X01003922. A change that looks simple in one file may require us to recall an interface elsewhere, a prior design decision, and a constraint on acceptable behavior. We use notes, diagrams, and other external representations so that we do not have to keep all of that information in our heads 16.16. Donald A. Norman, The Design of Everyday Things, Revised and expanded (Basic Books, 2013).
An agent encounters a comparable limit through its foundation model’s context window: the bounded input available in a single invocation. The harness may store a plan, earlier decisions, or tool results outside that window and retrieve them later. That lets work continue across many invocations, but information needed for a particular judgment must be supplied when that judgment is made. A long-horizon task requires a sequence of related decisions, with earlier results or commitments carried into later steps. Merely retaining a record does not ensure that the agent will find and use it.
Memory management can nevertheless expand what the whole system can work over. Research on agent memory and context management has demonstrated this through external storage, summarization, and selective retrieval 1717. Charles Packer et al., MemGPT: Towards LLMs as Operating Systems, arXiv preprint, 2023, https://doi.org/10.48550/arXiv.2310.08560. 18.18. Kuang-Huei Lee et al., “A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts,” Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 235 (2024): 26396–415, https://proceedings.mlr.press/v235/lee24c.html. The model’s context window stays bounded, but the surrounding machinery makes more information usable across the task.
Having information available does not guarantee that it can be used effectively. Studies of long-context language models find that usable context can fall short of nominal capacity and that information within the context is not uniformly accessible 1919. Cheng-Ping Hsieh et al., RULER: What’s the Real Context Size of Your Long-Context Language Models?, arXiv preprint arXiv:2404.06654, 2024, https://arxiv.org/abs/2404.06654. 20.20. Nelson F. Liu et al., “Lost in the Middle: How Language Models Use Long Contexts,” Transactions of the Association for Computational Linguistics (TACL) 12 (2024), https://arxiv.org/abs/2307.03172. Repository-scale studies likewise report substantial navigation and reconstruction work as relevant state spreads across a codebase 21.21. Sourcegraph, “Why Coding Agents Fail in Large Codebases,” Sourcegraph, 2026, https://sourcegraph.com/blog/why-coding-agents-fail-large-codebases.
In this book, we use the term reasoning horizon to describe the practical limit of the work a reasoner and its supporting machinery can carry through reliably, given the available representations and resources. For an agent, this depends on the foundation model together with the harness, external memory, and tools.** We borrow the term from Ramezanali and colleagues, who use it in seqBench to describe the length of a sequence a model can reliably complete 26.26. Mohammad Ramezanali et al., “seqBench: A Tunable Benchmark to Quantify Sequential Reasoning Limits of LLMs,” Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, 33771–92, https://doi.org/10.18653/v1/2025.emnlp-main.1714. Here we extend it to the practical limits of the complete reasoning system, including its harness, external memory, and tools.
AGENT HARNESSInset — Context Windows and Compaction
Intuitively, a harness can extend an agent’s reasoning horizon by preserving useful information and making it available when needed. Context windows and compaction make that idea concrete for agents built around large language models (LLMs). A context window limits how much material a model can process in one invocation. This capacity is measured in tokens. For text, tokens may be words, parts of words, or punctuation; multimodal models also encode images, audio, and other inputs into representations that occupy space in the context. Instructions, earlier exchanges, selected file contents, and tool results occupy space in the window, with room also needed for the response. A long-running task can accumulate far more history than one window will hold.
The harness can make room by compacting that history. Compaction replaces part of the accumulated context with a shorter account, often produced by asking an LLM to summarize it. Later invocations receive that account alongside selected original material and new input. For example, pages of investigation and test output might become a handoff recording the current plan, decisions already made, checks completed, and work still open.
Compaction can lose distinctions that later matter. If a summary preserves “the tests passed” but drops which tests ran or which version they exercised, the next invocation may treat that result as stronger evidence than it is. Records kept outside the window can supply the missing detail, provided the harness or agent knows to retrieve them. The engineering problem is choosing what must survive compaction and what can safely be recovered later 22.22. Prithvi Rajasekaran et al., “Effective Context Engineering for AI Agents,” Anthropic, September 29, 2025, https://www.anthropic.com/research/effective-context-engineering-for-ai-agents.
The generation of commodity intelligence that motivated this book is built around language models with bounded context windows. We deliberately abstract from that machinery: the reasoning horizon concerns what a reasoner and its supporting machinery can carry through reliably, given the available representations and resources. The concept is thus broader than a context-window limit, and is intended to remain useful as new kinds of intelligence bring different capabilities and constraints.
Research on external representations has long shown that how information is organized affects the work needed to reason with it 2324.23. Jill H. Larkin and Herbert A. Simon, “Why a Diagram Is (Sometimes) Worth Ten Thousand Words,” Cognitive Science 11, no. 1 (1987): 65–100, https://doi.org/10.1111/j.1551-6708.1987.tb00863.x.24. Jiajie Zhang and Donald A. Norman, “Representations in Distributed Cognitive Tasks,” Cognitive Science 18, no. 1 (1994): 87–122, https://doi.org/10.1207/s15516709cog1801_3. A useful engineering model preserves relationships that would otherwise have to be reconstructed from implementation. Figure 1.4-2 illustrates how this can reduce repeated reconstruction effort. Better representations may therefore make engineering work cheaper without necessarily extending the range of tasks the system can handle.
The reasoning horizon can expand in two ways: better machinery can preserve more relevant state, while better representations can reduce how much state the task requires. The Representation Problem asks: how should a large, evolving system be represented so that the properties relevant to an engineering question can be reasoned about without continually reconstructing them from implementation? Chapter 2 develops the corresponding part of the MAGE methodology: Modeling.
1.4.7 Probabilistic Work Needs External Enforcement
The second problem concerns assurance. Software engineering has always separated authorship from assurance. We compile code to check that it compiles and run tests to evaluate whether it behaves as required. Consequential systems add review, static analysis, formal methods, operational evidence, or independent approval. The producer's confidence is insufficient evidence of correctness.
Cheap, fast, repeatable agentic production makes that separation more consequential. An agent can propose a change, inspect it, explain it, and often repair it, but another probabilistic judgment from the same productive substrate does not automatically create engineering assurance. If an obligation matters, the environment must establish what evidence counts and what follows when the obligation is not met.
The U.S. defense community's AI4SDLC guidance identifies the same scaling problem: as agents produce more work asynchronously, the people reviewing it can become the bottleneck 25.25. Code.mil, AI and Agentic Workflow Design and Governance, AI for the SDLC, 2026, https://code.mil/AI4SDLC/plays/ai_sdlc_workflows-play/. Requiring human approval is therefore only part of the answer; reviewers also need representations and evidence that let them make an informed judgment about what the machinery has produced.
Commodity intelligence is probabilistic. This book focuses on the implications for engineering. A foundation model’s answer does not ordinarily come with a guarantee that its engineering judgments are correct. There are several sources of uncertainty: generation can vary between runs, a request may admit several interpretations, and the model may take different reasoning steps or choose different tools. A task can depend on many such judgments before it produces a result. The reliability of this work also depends on how the agent’s harness and engineered environment organize it. They can arrange retries, decompose the task, combine multiple answers, retrieve additional context, or ask another model to evaluate the first. These techniques can improve reliability, but none by itself turns an inferred judgment into an engineering guarantee.
A simple model shows how an agent’s reliability on individual features affects its chance of implementing them all correctly. Suppose an agent implements features in a product, numbered 1 through . Let denote the probability that the agent implements feature correctly. Under the simplifying assumption that these implementation outcomes are independent, the probability that the agent implements all the features correctly is
Remember, when you multiply fractions between zero and one, the product is smaller than any of the factors. The effect is small when the agent is highly reliable and has only a few features to implement. If an agent has a high probability of correctly implementing each feature, say 99%, then its probability of getting three features right is . As we ask the agent to implement more features, its chance of getting them all right falls: for ten features, and for a hundred. If the agent is less reliable, say 95% on each feature, its chance of correctly implementing all ten falls to . An agent that usually gets individual features right can therefore have a much lower chance of getting every feature in a product right.†† Human code review and cleanup, or a final pass with a more capable but more expensive model, may supply what the initial agent leaves undone. The relevant cost is that of the whole process, including finding and fixing mistakes; the result must still meet its requirements.
The decisions that depend on agentic inference form the system’s probabilistic surface. Other parts of the system may be deterministic, such as a tool that applies a fixed transformation or a test suite that evaluates explicit conditions. Better models and iterative loops can make the agent’s judgments more reliable. As we will discuss throughout the book, we can also build mechanisms that enforce particular requirements even when the agent makes a mistake.
On the other hand, even if agentic implementation were perfectly reliable, that would not tell us what to build. Engineers must still choose and justify the requirements, and determine what validation evidence would establish that the resulting system meets the needs it is meant to serve.
Autonomous work can be constrained and checked at the interfaces through which it acts. This is the Enforcement Problem: how should engineering intent be enforced over autonomous work, so that satisfying important obligations does not depend on the agent remembering, agreeing, or self-certifying each time? Chapter 3 develops the corresponding part of the MAGE methodology: Alignment.
Works Cited
- Chen, Mark, Jerry Tworek, Heewoo Jun, et al. Evaluating Large Language Models Trained on Code. arXiv preprint arXiv:2107.03374, 2021. https://arxiv.org/abs/2107.03374.
- OpenAI. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774, 2023. https://cdn.openai.com/papers/gpt-4.pdf.
- Ouyang, Long, Jeffrey Wu, Xu Jiang, et al. “Training Language Models to Follow Instructions with Human Feedback.” Advances in Neural Information Processing Systems 35 (2022): 27730–44. https://proceedings.neurips.cc/paper_files/paper/2022/hash/b1efde53be364a73914f58805a001731-Abstract-Conference.html.
- Yao, Shunyu, Jeffrey Zhao, Dian Yu, et al. “ReAct: Synergizing Reasoning and Acting in Language Models.” International Conference on Learning Representations, 2023.
- Hu, Edward J., Yelong Shen, Phillip Wallis, et al. “LoRA: Low-Rank Adaptation of Large Language Models.” International Conference on Learning Representations, 2022. https://openreview.net/forum?id=nZeVKeeFYf9.
- Bender, Emily M., Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 2021, 610–23. https://doi.org/10.1145/3442188.3445922.
- Bubeck, Sébastien, Varun Chandrasekaran, Ronen Eldan, et al. Sparks of Artificial General Intelligence: Early Experiments with GPT-4. arXiv preprint arXiv:2303.12712, 2023. https://arxiv.org/abs/2303.12712.
- Li, Kenneth, Aspen K. Hopkins, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. “Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task.” International Conference on Learning Representations, 2023. https://openreview.net/forum?id=DeG07_TcZvT.
- Nanda, Neel, Andrew Lee, and Martin Wattenberg. “Emergent Linear Representations in World Models of Self-Supervised Sequence Models.” Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP (Singapore), 2023, 16–30. https://doi.org/10.18653/v1/2023.blackboxnlp-1.2.
- Jin, Charles, and Martin Rinard. “Emergent Representations of Program Semantics in Language Models Trained on Programs.” Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 235 (2024): 22160–84. https://proceedings.mlr.press/v235/jin24e.html.
- Vir, Reya, Lydia Chilton, Zhuo Zhang, and Eugene Wu. “FlowCheck: Helping End-Users Specify and Verify Intent in Vibe-Coded Web Apps.” Proceedings of the 2nd ACM SIGPLAN International Workshop on Language Models and Programming Languages (Oakland, CA, USA), 2026. https://doi.org/10.1145/3843750.3843844.
- Zhong, Hailin, and Shengxin Zhu. AI Harness Engineering: A Runtime Substrate for Foundation-Model Software Agents. arXiv preprint arXiv:2605.13357, 2026. https://arxiv.org/abs/2605.13357.
- Young, Justin. “Effective Harnesses for Long-Running Agents.” Anthropic, November 26, 2025. https://www.anthropic.com/engineering/effective-harnesses-for-long-running-agents.
- Tan, Garry. “Thin Harness, Fat Skills.” April 9, 2026. https://github.com/garrytan/gbrain/blob/master/docs/ethos/THIN_HARNESS_FAT_SKILLS.md.
- Cowan, Nelson. “The Magical Number 4 in Short-Term Memory: A Reconsideration of Mental Storage Capacity.” Behavioral and Brain Sciences 24, no. 1 (2001): 87–114. https://doi.org/10.1017/S0140525X01003922.
- Norman, Donald A. The Design of Everyday Things. Revised and expanded. Basic Books, 2013.
- Packer, Charles, Sarah Wooders, Kevin Lin, et al. MemGPT: Towards LLMs as Operating Systems. arXiv preprint, 2023. https://doi.org/10.48550/arXiv.2310.08560.
- Lee, Kuang-Huei, Xinyun Chen, Hiroki Furuta, John Canny, and Ian Fischer. “A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts.” Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 235 (2024): 26396–415. https://proceedings.mlr.press/v235/lee24c.html.
- Hsieh, Cheng-Ping, Simeng Sun, Samuel Kriman, et al. RULER: What’s the Real Context Size of Your Long-Context Language Models? arXiv preprint arXiv:2404.06654, 2024. https://arxiv.org/abs/2404.06654.
- Liu, Nelson F., Kevin Lin, John Hewitt, et al. “Lost in the Middle: How Language Models Use Long Contexts.” Transactions of the Association for Computational Linguistics (TACL) 12 (2024). https://arxiv.org/abs/2307.03172.
- Sourcegraph. “Why Coding Agents Fail in Large Codebases.” Sourcegraph, 2026. https://sourcegraph.com/blog/why-coding-agents-fail-large-codebases.
- Rajasekaran, Prithvi, Ethan Dixon, Carly Ryan, and Jeremy Hadfield. “Effective Context Engineering for AI Agents.” Anthropic, September 29, 2025. https://www.anthropic.com/research/effective-context-engineering-for-ai-agents.
- Larkin, Jill H., and Herbert A. Simon. “Why a Diagram Is (Sometimes) Worth Ten Thousand Words.” Cognitive Science 11, no. 1 (1987): 65–100. https://doi.org/10.1111/j.1551-6708.1987.tb00863.x.
- Zhang, Jiajie, and Donald A. Norman. “Representations in Distributed Cognitive Tasks.” Cognitive Science 18, no. 1 (1994): 87–122. https://doi.org/10.1207/s15516709cog1801_3.
- Code.mil. AI and Agentic Workflow Design and Governance. AI for the SDLC, 2026. https://code.mil/AI4SDLC/plays/ai_sdlc_workflows-play/.
- Ramezanali, Mohammad, Mo Vazifeh, and Paolo Santi. “seqBench: A Tunable Benchmark to Quantify Sequential Reasoning Limits of LLMs.” Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, 33771–92. https://doi.org/10.18653/v1/2025.emnlp-main.1714.