Skip to content

§4.2 One Problem, Many Models

A large PowerPoint presentation exposed a memory problem in DocAble. The system handled ordinary documents, but this input exceeded the worker's memory budget. I needed to understand why memory consumption grew with document size before deciding how to change the implementation.

Several familiar techniques were available: reduce the working set, separate structure from media, partition execution, or change where data resides. Each could reduce memory consumption, but each would also affect other properties of the system. The engineering question was which relationship needed to change.

The initial problem was memory exhaustion. Later changes raised questions about latency, throughput, and execution cost. I did not begin with a plan to construct a family of models. Each model emerged because an observed failure or a proposed improvement exposed a relationship worth understanding.

4.2.1 One Document, One Worker

DocAble originally treated an entire document as one unit of execution. A long-running Kubernetes worker opened the document, held the state needed for remediation in memory, modified it, and produced the result. Figure 4.2-1 traces the architectural progression from whole-document execution through page chunking to skeletonized residency.

Evolution of DocAble’s document working setThree equal panels: whole-document execution, page-chunked execution, and skeletonized execution. Chunking partitions execution; skeletonization separates resident structure from externally stored media. Media access and the GenAI connection are distinct. (a) Whole-document execution (b) Page-chunked execution (c) Skeletonized execution Document Worker Structure Media (all resident) Output Document Worker Pages 1–10 Structure Media Worker Pages 11–20 Structure Media Worker Pages 21–30 Structure Media Merge Output Document Worker (skeletonized) Structure Media references GenAI service External media • One worker retains the complete document. • Structure and all media are resident. • Memory scales with the document payload. • The document is partitioned into page chunks. • Each worker retains the chunk’s structure and media. • A smaller working set per worker reduces memory. • The worker retains structure and media references. • Media is not resident, but streamed on demand. • Media is processed by a GenAI service.
Figure 4.2-1. Three stages of DocAble's document processing architecture. (a) Initially, one worker retained the complete document, including its media. (b) Page chunking reduced the working set per worker, but each chunk still retained its media. (c) Skeletonization separated document structure from media, retaining only structure and references in worker memory while streaming media on demand. Chunking changed the execution unit; skeletonization changed the residency requirement.

The document therefore served as both the user's artifact and the worker's in-memory unit of computation. Media needed by GenAI operations was read from that resident representation and sent to the foundation model as needed.

I had specified how DocAble should process documents, but I had forgotten to tell Claude what memory consumption was acceptable. The resulting architecture worked on ordinary inputs. Larger documents exposed the omission. Teaching presentations carried more slides, images, and other media, and handling the whole document in one process began to put enough pressure on worker memory that the execution unit needed to shrink.

4.2.2 Pages as the Unit of Scale

DocAble's response was ChunkWorker, which divided a document into page ranges (Figure 4.2-1, panel b).

The new representation treated contiguous pages as independent work partitions. For a presentation, a chunk contained a range of slides together with the document state needed to remediate them; the system later recombined the resulting patches into the complete artifact.

Each worker now handled only part of the document, and multiple chunks could run concurrently. More importantly, chunking reduced the memory required by an individual worker. Memory affects both infrastructure cost and cold-start behavior, so a smaller worker was useful even for documents that did not strictly require chunking to fit.

Chunking worked across a much broader workload: ordinary teaching presentations and hundreds of documents from multiple universities. Page ranges gave each ChunkWorker a practical bound on its working state, and experience provided no reason to replace that design merely because another architecture was possible.

This architecture accommodated the inputs encountered so far. The page boundary gradually became part of how DocAble was understood: a document was divided into chunks because chunks were how large documents were processed. A roughly 250 MB media-heavy presentation eventually exceeded that operating regime.

Smaller chunks could reduce memory consumption further, but they would also reduce the context available for semantic judgments and increase coordination costs. Chunking therefore presented a tradeoff between memory use and the information available to each operation.

The 250 MB presentation made me question the tradeoff itself. Most of the file consisted of media, but many remediation operations needed only the document's structure. Why was the worker retaining all that media?

That question led to a different representation rather than another reduction in chunk size.

4.2.3 A New Class of Input

A 250 MB PowerPoint file does not imply that remediation needs 250 MB of document content resident in memory.

Modern Office documents use Office Open XML (OOXML). A .pptx, .docx, or .xlsx file is a package containing many parts. Much of its semantic structure is XML: text, slides, relationships, identifiers, and references among objects. Large binary assets such as images and other media live alongside those XML parts inside the package.

The structural portion is comparatively small. It is largely text, and even an unusually large Office document may require only low megabytes to represent its semantic skeleton. The media payload can be hundreds of megabytes. Most remediation operations need only the structural portion; they may reach into the media, but not every media byte need be resident at once.

Until this point, DocAble had largely accepted the storage representation supplied by the document format as its working representation. For Office documents, that meant opening the OOXML package through the document library and letting its structure and media participate in the in-memory object graph; other formats followed analogous format-native representations. That was reasonable while the input sizes fit the worker. Handling a roughly 250 MB media-heavy file without allocating correspondingly large workers required reconsidering the arrangement itself.

The existing representation coupled the two. Figure 4.2-1, panel c, shows how skeletonization separated them.

The architecture therefore carried an implicit scaling relation: total media size drove resident worker state. Finer chunking could reduce that state, at the cost of more coordination and less semantic context, but could not eliminate the relationship. The new file made that limitation consequential.

A residency model made the alternative visible (Figure 4.2-1, panel c): the structural pipeline could operate on a lightweight representation while media lived in object storage, and operations that needed media could reach it by reference.

4.2.4 Remove Media from the Working Set

The memory problem came from coupling document structure to media residency: total media size increased the worker's persistent working set even though most remediation operations needed only the document's structural skeleton. Skeletonization removes that coupling. For OOXML documents, the pipeline streams through the package, moves media into external storage, and leaves stable references in a much smaller structural skeleton.

Remediation then operates on the skeleton (Figure 4.2-1, panel c). If a GenAI operation needs an image, the worker follows the reference and streams that media through the network to the GenAI service without first materializing the document's complete media payload in worker memory. Recombination follows the same discipline in reverse. The worker no longer needs memory proportional to total media size.

The corresponding memory invariant states the property directly:

peak worker RSS <= C

where, within the declared operating envelope,

C =
    rolling stream buffer
  + structural skeleton
  + handles
  + remediation working set

and C is independent of aggregate media payload size within that envelope.

Chunking had shown that the observed documents worked. Skeletonization supports a structural claim over a declared class of inputs: peak worker memory is bounded independently of total media size, provided the modeled path never retains the complete payload.

That claim depends on bounded streaming buffers, a structural skeleton whose size remains inside the declared envelope, and bounded active state at each media consumer. The write path must stream too; buffering the complete payload before writing would merely move the memory spike. Pathological structure, oversized individual objects, excessive concurrency, or an unmodeled resource can still exhaust memory.

The implementation enforces the assumptions behind the bound as invariants; measurement checks that the realized system behaves as the model predicts. The PDF path already shows approximately flat peak-RSS scaling across a count-scaled media sweep when each object is flushed; the corresponding OOXML media-size sweep remains a designed validation step.

While this architecture was being developed, agents sometimes proposed removing operations or state that looked expensive but were essential to correct remediation. The fixtures available at the time would not necessarily have exposed every resulting semantic failure. The engineering model supplied a distinction the surface objective did not: which relationships the system depends on, and which resource costs follow from the current realization. Agents could use that structure to correct their account of the system before changing it.

The model also let me judge which relationships an optimization had to preserve while the realization changed faster than I could reconstruct its behavior from code and tests.

Changing the architecture to satisfy the residency model is active alignment. Once correspondence is established, the model and invariant checks provide passive alignment: later changes must preserve the memory property or justify revising the model and its checks. The memory obligation became part of the alignment kernel for the execution changes that followed.

4.2.5 Why One Level Deeper Matters

An end-to-end measurement says whether the outcome moved. It does not say why it moved. Peak worker memory is the outcome; media residency is the mechanism that produces it. A model of the mechanism answers the further question: which relationship generated the improvement, and whether that explanation is compatible with the work the system must still perform.

Measurement tests the residency model's explanation. A peak-memory figure reports what happened on the inputs in one sweep; the model predicts which changes to those inputs should change the result. Comparing the two can expose a missing relationship, as a later sweep did in this case. The engineer needs both: measurements to test the account, and an account of memory growth to assess the next unfamiliar document.

Ousterhout puts the same instruction compactly: measure one level deeper 1.1. John Ousterhout, “Always Measure One Level Deeper,” Communications of the ACM 61, no. 7 (2018): 74–83. One level, not arbitrarily deep. The deeper cut earns its cost when it exposes an optimization target, a failure class, or a decision that the surface number hides. Media residency did all three, and the surface number — peak resident memory — would have named none of them.

4.2.6 Memory Did Not Stop Mattering

Bringing memory under governance did not make memory disappear. It changed the conditions under which later performance work could proceed. The next questions increasingly concerned latency, throughput, and compute cost, but proposed answers remained subject to the memory relationships already made explicit. A faster design that exceeded the deployment's memory budget was not an improvement.

Memory came first because the two failures differ in kind: out-of-memory failure prevented processing altogether, whereas inefficient execution increased latency and cost while still producing a result. The later service-sizing model recorded that priority in a one-sided safety margin: exhausting memory kills the process, while over-provisioning costs money. Another domain might prioritize safety, latency, or numerical accuracy; the ordering must follow the consequences.

That coupling became explicit in the engineering machinery. Concurrency models delegated memory feasibility to the memory model of record rather than doing memory arithmetic of their own. Fan-out width was represented as a peak-memory multiplier, so a throughput author had to size the width against a memory budget. Some throughput changes carried fail-loud guards that refused to run a fan-out whose estimated memory exceeded its budget. A batched-rendering design was chosen over a concurrent one partly because the concurrent alternative measured past the service's memory ceiling. And in one latency campaign a memory sweep blocked the production deploy altogether. The earlier resource model had become part of the environment within which the later one operated.

4.2.7 Why Are We Still Chunking?

Page chunking had originally served two purposes: it partitioned remediation work and reduced the memory required by each worker. Skeletonization largely removed the second reason. I could now reconsider the execution boundary without immediately exceeding the worker's memory budget.

The question was no longer how many pages a worker could hold. It was which operations could execute independently, and which had to wait for information produced by other operations.

A page tells us where content occurs, not which computations depend on which others. DocAble already represented remediation as a dependency graph. Consider alt-text generation as a concrete example. Each page can first be analyzed independently. Those analyses feed an operation that builds shared document context. Once that context exists, the GenAI queries that generate alt text for individual images become ready to execute independently. The actions and their information dependencies form a remediation graph; the currently executable actions form its frontier.

The same graph admits different execution policies. Figure 4.2-2 shows the distinction.

Two execution policies over one remediation graph of actions One graph of actions — analyze page 1, 2, and 3; build document context; and GenAI query for image 1, 2, and 3 — with information dependencies among them: each page analysis feeds the document context, which feeds each GenAI query. (a) Page-oriented execution groups the actions into three page columns by where their artifacts occur, so a page owns both an upstream analysis and a downstream query, and the shared document-context action spans all pages. (b) Frontier execution groups the same actions by when they are ready: the three page analyses run together, their results build the document context, and that context makes the three GenAI queries ready at once. Same work and dependencies; different boundary. (a) Page-oriented execution (b) Frontier execution PAGE 1 analyze page 1 GenAI query image 1 PAGE 2 analyze page 2 GenAI query image 2 PAGE 3 analyze page 3 GenAI query image 3 build document context group work by where it occurs READY FRONTIER 1 analyze page 1 analyze page 2 analyze page 3 READY FRONTIER 2 build document context READY FRONTIER 3 GenAI query image 1 GenAI query image 2 GenAI query image 3 page-analysis results document context alt-text results group work by when it is ready SAME WORK · SAME DEPENDENCIES · DIFFERENT EXECUTION BOUNDARY
Figure 4.2-2. Two execution policies over the same remediation graph. The graph contains actions—page analysis, construction of document context, and GenAI queries—and information dependencies among them. (a) A page partition groups actions by the page their artifacts occur on, so it can span several dependency levels. (b) A frontier groups actions by readiness: page analyses execute first, their results enable construction of document context, and that context enables independent GenAI queries for alt text. The work and its dependencies are unchanged; only the execution boundary changes — so the choice between (a) and (b) is a scheduling decision, not a remediation redesign.

The graph explains the ordering. Independent work can run together; a consumer waits only for information it actually needs from upstream. Pages do not create those dependencies. Information does. Expressed as a directed acyclic graph, the work currently ready to execute forms a frontier: the nodes with no unmet dependencies. A worker executes that frontier, materializes its outputs, and thereby exposes the next frontier.** This is ordinary topological traversal of a directed acyclic graph, familiar from compilers, build systems, and dependency schedulers.

This reframes ChunkWorker. Page-oriented and frontier-oriented execution are not different remediation graphs. They are different ways of partitioning the same graph. Page execution groups work by where it occurs; frontier execution groups work by when it is ready.

Skeletonization removed the memory pressure that had made page ranges the natural execution boundary, so page identity no longer needs to determine execution. It can survive as metadata on graph nodes—useful for locality, debugging, rate limiting, or failure isolation—without remaining the fundamental unit of executable remediation.

For the present problem, the architecture therefore needs only a clean seam between the remediation graph and its executor: the graph supplies ready work and its dependencies; the executor consumes them. The executor can schedule by frontier now and exploit page locality later if measurements justify it. There is no need to build a strategy hierarchy merely because two policies are conceivable. The graph/executor seam preserves the degree of freedom without spending it. In the realized architecture, that execution-driving representation remains distinct from richer analytical projections built over the same computation identities. The distinction matters later: a model may govern realization while another view of the same structure serves analysis.

4.2.8 Analyze Before You Build

Representing remediation as a dependency graph does more than provide an execution plan. The graph can be analyzed before an executor—or a proposed architectural change—is implemented. Figure 4.2-3 illustrates how critical-path analysis can identify a bottleneck and suggest two kinds of redesign: decomposing expensive work or changing the dependencies among operations.

Analyzing dependency graphs to redesign execution A weighted graph branches after A. The path through B and D takes 12 seconds, while the path through C and E takes 6 seconds. Three concurrent 3-second parts of B shorten the critical path to 7 seconds before overhead. In DocAble, two image-description operations become independent when each receives its image and shared rendered page context. (a) Identify the critical path Diagnosis A (1 s) B (8 s) C (2 s) D (2 s) E (2 s) F (1 s) 12 s 6 s Critical path: A → B → D → F (12 s) Two modeling responses Given this analysis, the engineer can explore two redesigns. • Decompose the dominant operation (b) Reduce its cost while keeping the same dependencies. • Redesign the dependencies (c) Remove an unnecessary serial constraint. (b) Decompose an operation Same dependencies A (1 s) B₁ (3 s) B₂ (3 s) B₃ (3 s) D (2 s) F (1 s) New critical path A → Bᵢ → D → F (7 s) Illustrative, ignoring overhead (c) Remove an unnecessary dependency Change the information structure Image 1 Gen alt 1 Image 2 Gen alt 2 ⇒ Shared rendered context Gen alt 1 Image 1 Gen alt 2 Image 2 Before: serial dependency After: independent operations consume shared context Legend Dominant or modified operation Other operation Dependency (must complete before)
Figure 4.2-3. Analyzing dependency graphs to redesign execution. (a) Critical-path analysis identifies the longest weighted dependency chain, which sets a latency floor of 12 seconds in this illustrative graph. Two possible responses are shown. (b) Decomposing the dominant operation into independent suboperations reduces the modeled critical path to 7 seconds, excluding coordination overhead. (c) Changing the information dependencies can remove a serial constraint altogether. In DocAble’s image-description pipeline, replacing propagation of previously generated descriptions with shared rendered context allowed the image-description operations to proceed independently.

Panel (a) shows a weighted dependency graph. Its critical path, A→B→D→F, takes 12 seconds, compared with 6 seconds for the alternative path. Adding workers cannot shorten the critical path without changing the work or its dependencies. One possibility is to decompose the dominant operation. In panel (b), splitting B into three independent operations reduces the modeled critical path from 12 to 7 seconds.

Another possibility is to reconsider the dependencies themselves. In DocAble’s contextual image-description pipeline, each generated description was passed to the next image-description operation. The resulting chain had depth proportional to the number of images. For N images, with each GenAI call taking roughly t, the serial latency floor approached N⋅t. Adding workers could not remove that dependency.

The model exposed a distinction between the existing implementation and the information the operations actually needed. Each image required adequate surrounding context, but not necessarily the preceding image’s generated description. Rendering the surrounding pages once supplied a shared visual representation that each image-description operation could consume independently. Panel (c) shows how changing the information dependencies removed the serial chain.

The second resource model

The critical-path model explained which dependencies could determine latency. A second model captured a different source of cost: repeated whole-document work. Let R be the number of remediation rounds, F the cost of producing the whole document once, and G the remaining GenAI work. The model represented execution without page chunks as

T=(R+1)F+G

For example, suppose a document requires four remediation rounds, whole-document production takes 10 seconds, and the remaining GenAI work takes 20 seconds. The model predicts T = (4+1)(10) + 20 = 70 seconds. Fifty of those seconds come from producing the document five times: once for each remediation round and once more to apply the accumulated changes. Cutting the GenAI work in half would save 10 seconds; eliminating two production rounds would save 20. The model therefore identifies repeated whole-document production as a quantity worth attacking.

The model was also falsifiable. Its calibration procedure inverted observed execution times to recover its components, and it rejected any fit whose inferred costs came out negative. Measurement could therefore disagree with the engineering account rather than merely supplying another number to optimize against.

Predicting latency

Critical-path analysis also supports quantitative latency predictions when operation costs are available. For the representative workload used during this analysis, the relevant GenAI chain had three dependent levels whose per-call times add to an irreducible floor (Figure 4.2-4).

The GenAI dependency chain in a schematic remediation graph Section summary feeds document summary and then alt text, alongside illustrative independent work. The three call durations sum to about 7.7 seconds. GenAI critical path (dependent chain) Section summary 1.777 s (estimated) Document summary 1.777 s (estimated) Generate alt text 4.1049 s (measured) Extract document Other independent work (illustrative) Repair structure (validators, rules, …) Repair metadata (types, properties, …) Apply changes GenAI latency floor: 1.777 + 1.777 + 4.1049 ≈ 7.7 s GenAI call Must precede
Figure 4.2-4. The GenAI critical path in a schematic fragment of DocAble’s remediation graph. Independent remediation computations can execute concurrently, but section summarization, document summarization, and alt-text generation form a dependent chain. For the modeled workload, their estimated and measured durations contribute approximately 7.7 seconds of serial latency, which additional workers alone cannot eliminate.

For the representative 37-image workload, local concurrency of six required seven waves, yielding a predicted latency of about 33 seconds. Modeled cloud concurrency of 64 could execute that frontier in one wave; including estimated cold-start and round-trip costs gave about 11.5 seconds, a roughly 2.9× improvement. The prediction depended on those overhead estimates and required staging measurements before cutover. The local cap also reflected memory: concurrent ChunkWorkers each held format-native document state. Skeletonization reduced that per-worker residency, making wider execution feasible without multiplying the same large media working set.

Predicting the cost of a design change

A proposed intermediate representation can be inserted into the dependency graph before it is implemented. Recomputing the critical path, execution rounds, peak width, and overhead shows where it could add cost. An IR off the critical path may add no serial latency; one that deepens the chain adds its call time to the floor and may require another distributed round. These are predictions to test, but they help decide which candidate is worth building.

Use prediction error as evidence

Implementation eventually supplies measurements to compare with the model’s predictions. A residual — the gap between prediction and observation — turns into an engineering question. Figure 4.2-5 connects the general engineering loop in panel a to an illustrative prediction error in panel b and the refinement of the model in panel c.

Modeling through prediction and measurement Three panels: an engineering loop, an illustrative latency residual, and revision until the residual is acceptable. (a) The engineering loop (b) One prediction error (c) Revise and repeat Problem Model Analyze Prediction / design Implementation Measurement Explain residual refine model Model predicts 11.5 s Observed 17.8 s Residual: +6.3 s Wrong parameter Wrong relation Missing mechanism Update the model Reanalyze (e.g., new prediction) Implement change Measure again Residual acceptable? No Keep the change Yes
Figure 4.2-5. Modeling through prediction and measurement. (a) An engineering problem motivates a model whose analysis informs implementation. Measurements test its predictions and may require the model to be revised. (b) A discrepancy between predicted and observed latency directs investigation toward an incorrect parameter, an incorrect relationship, or a missing mechanism. The numerical values are illustrative. (c) The revised model is tested again until its residual is acceptable for the question being investigated.

The queue overhead may have been underestimated. Two apparently independent nodes may share a serialized resource, or media loading may introduce a cost the latency model omitted. A close prediction supplies evidence that the model captures dominant mechanisms; a miss directs investigation toward a wrong parameter, a missing relation, or an implementation problem.

4.2.9 What Carries Forward

DocAble did not turn observed wall-clock time into the durable control. Wall time varies with hosts, services, cold starts, and other environmental effects, so a benchmark score from one machine makes a poor obligation. The performance ratchet holds structural quantities instead — the quantities the model says contribute to execution cost: production rounds, GenAI calls, capture invocations, and render invocations. Each is counted deterministically on a fixed fixture. A change may lower these quantities freely; raising one requires a named justification recorded beside the baseline. The environment therefore preserves part of the explanation for performance rather than a number that only reproduces on the machine that produced it.

A fifth quantity, the number of GenAI hops, was observed without blocking while its behavior was still being characterized. The distinction is worth keeping: a counter that gates and a counter that only reports are different mechanisms, and a new one usually begins as the second.

Models get falsified; the obligation survives

A ratchet does not make the current model true forever. Several memory hypotheses in this episode were later falsified. One held that a PDF library's whole-file object model kept roughly 2.4 GiB resident; measurement found about 1 MiB live, because the file was memory-mapped and the image bytes never entered the managed heap. Another projected that moving rendering off the worker would recover 90 to 120 MiB; the landed measurement recovered about 6, because rendering was already running off the worker. A later adversarial sweep found a scaling dimension the bound had not covered at all: peak memory rose with decoded image area rather than with stored media size. The response was to extend the model and add an admission gate that refuses inputs outside the declared envelope. A capacity model elsewhere in the same environment was eventually caught about twofold too low against dozens of measured sweeps, and its parameter was recalibrated rather than the measurement discounted.

The measurements carried their own provenance. A figure taken on a development host was marked as such and did not become the value of record until the deployment environment reproduced it. A process-level measurement may omit memory included in the container's total. Recording the measurement's origin makes that difference visible and helps engineers decide which parameters need to be measured again.

Nor was the memory work finished before the execution-time work began. A memory front was still open while the time models were being built. What became durable was not a particular bound but the requirement to reason explicitly about the resource. Later work could challenge the model, recalibrate it, or replace it with a better one. It could not silently return memory behavior to an unexamined implementation detail. The ratchet preserves the engineering question and the obligation to answer it, even when the answer changes.

Engineering technique — Ratcheteering

Governance conversion is an individual engineering move: it turns a recurring failure, judgment, or newly discovered relationship into durable structure. Ratcheteering is the continuing practice of accumulating and correcting that structure across engineering episodes.

The ratchet preserves the obligation, not necessarily the current answer. A later measurement may refute a model, change a parameter, expose another scaling dimension, or require a different control. The engineering environment should permit those corrections. What later work cannot do silently is return the consequential property to an unexamined implementation detail.

Each episode may add, revise, or retire the structure that later engineering work inherits. The engineer retains models, procedures, evidence, and controls selectively, when their engineering value justifies the maintenance they require.

4.2.10 The Resulting Model Family

The large-file requirement produced a connected family of models, each answering a different engineering question. Table 4.2-1 collects them.

Table 4.2-1. Models for large-file remediation.
ModelQuestion
Media residencyWhat must reside in worker memory? How should memory scale with media size?
Batch executionHow do batch size and concurrency affect latency, cost, and truncation risk?
Critical pathWhat sets the latency floor? Which nodes are optimization targets?
DistributionWhere should ready work execute? What coordination costs does that introduce?
Remediation graphWhat work exists, and what depends on what?
Fact / IR catalogueWhich derived facts feed later computations? How deep is the information-dependency chain?
Image fidelityWhat representation of media is required, and what does rendering it cost?

These models take different forms because they answer different questions. Some are typed graphs, some quantitative functions, some declarations of entities and invariants. They stay separate and compose through shared identities and explicit relationships. The remediation graph and fact catalogue supply the static dependency structure, and the batch model supplies per-call cost; the critical-path model combines the two into latency. The distribution model supplies the execution topology the coordination taxes are charged against. The media-residency model supplies the independent memory bound that makes the execution design affordable.

Some answers follow from structure; others depend on calibrated measurements. The critical path identifies ordering that more workers cannot remove, while a latency prediction depends on operation times and coordination costs. Keeping those sources of knowledge distinct makes disagreement informative: a measurement can require a new parameter, a revised relation, or another model.

Modeling Inside the Implementation Loop

This sequence took about three days inside the existing governed engineering environment. It included skeletonization and streaming, typed-patch work, reconsidering ChunkWorker, remediation and fact graphs, quantitative analysis, the chunkless execution design, and corrections when assumptions proved wrong. The timescale matters because modeling could participate in the implementation loop rather than follow it as a separate documentation phase.

The seven models arrived one at a time. Larger inputs exposed a memory problem; changing how media was stored made the execution boundary worth reconsidering; the dependency graph exposed latency limits and possible redesigns. Each model answered a question raised by the work. Together, they let me predict the consequences of proposed changes and test those predictions against measurements. When the measurements disagreed, I revised the models.

Works Cited

  1. Ousterhout, John. “Always Measure One Level Deeper.” Communications of the ACM 61, no. 7 (2018): 74–83.
© James C. Davis, 2026–present