The Built System

The timeline told what happened. This chapter tells what got built. Before any of the governance in the rest of the book makes sense, you need the shape of the thing it governs. DocAble is a real system, live in beta, that takes a document a university actually distributes and hands back a version a screen reader can read. It runs on frontier models for the parts that need to understand a slide, and on ordinary deterministic code for everything else. The interesting engineering lives in how those two halves are joined.

What the system does, end to end

A user uploads a file. The system figures out what kind of document it is, breaks it into work, remediates each piece, checks the result against a real accessibility standard, stamps what it changed, and returns the corrected file with evidence that the job was done. Six moving parts carry that load.

• A front door. A website and an API take the upload, authenticate the user, meter the usage against a quota, and put the job on a queue. A professor drops in a slide deck; the system takes it from there.
• A dispatcher. The document is split into chunks — slides, pages, sheets — and each chunk becomes a unit of work handed to a worker. Splitting is what lets a hundred-slide deck finish in about a minute instead of an hour.
• A remediation core. A command-line engine, written in a typed language, does the actual repair. It walks the document through a typed model of its format, and for each thing that needs fixing it either applies a deterministic rule or asks a model. This is where alt text gets written, reading order gets set, titles get added, contrast gets fixed.
• A check engine. After every pass, a standards-grounded checker maps each remaining finding to a specific clause of the accessibility standard. The system does not trust an outside tool to tell it whether the job is done; it owns the definition of done.
• A validator. A content-fidelity gate asserts that the meaning of the input survived into the output. Passing a checker is not the same as keeping the author's words. The validator catches the case where the file got more accessible and less true.
• A provenance layer. Every artifact the system inserts is stamped, so a finished document's history can be reconstructed and any single change explained or reversed. Trust that can be audited is the only kind a university will accept.

Two design commitments run underneath all six. Every format is touched through one typed model, never the raw library, so a fix applied once holds everywhere. And the system fails loud: when the model is unreachable, the pipeline stops rather than quietly shipping a degraded file. A silent wrong answer is the worst outcome for a system whose whole promise is that the meaning survived.

The document is the hard part

None of this would be worth building if a document were just text in a box. It is not. A slide is a visual and semantic artifact — figures, tables, equations, an order the eye follows, emphasis carried by layout and color. To make it accessible, the system has to recover enough of that meaning to present it through another channel. For years that was out of reach. The turn came with vision-language models that can look at a rendered slide and say what it means: read the equation, describe the figure, name what the slide is trying to teach.

That capability is real, and it is not a system. You can paste a small deck into a chat model and get useful output. You cannot get a corrected file back, a guarantee it satisfies the standard, or any evidence the meaning survived the trip. The model understands the slide and understands nothing about the file format, the standard, the university, or the cost of being wrong. Turning that flash of capability into something a university can trust is the engineering. The next section is the pattern that does the turning.

Designing with a probabilistic component: the LLM as a function call

Here is the central problem of building on a model. A model is a probabilistic component. Ask it the same question twice and you may get two answers. Ask it a question slightly outside what it saw in training and it will answer anyway, confidently, wrong. A deterministic function you can reason about; a probabilistic one you cannot. And yet the model is the only thing that can look at a figure and describe it. You have to build a reliable system out of an unreliable part.

The pattern that makes this work is to treat the model as a typed function call. The idea is not ours. A recent verification system, facing the same tension in a different domain, names it directly: deterministic algorithms control the workflow and define the objective of each step, and when a step needs semantic understanding, the system delegates a bounded task to a tool-equipped model — then a deterministic module validates the returned artifact before it is allowed in. The workflow is deterministic. The model is a subroutine inside it. The subroutine is never trusted on its word.

Read the pattern in three moves.

• The caller is deterministic, and it owns the workflow. Ordinary code decides what happens and in what order. It decides when a step needs the model at all — most steps do not. When a step is a matter of judgment rather than a matter of rule, and only then, the caller reaches for the model.
• The call has a typed contract on both sides. The task handed to the model is packed into a typed input — this figure, this context, this question — and the answer is required to come back in a typed output shape the caller can act on. A free-form paragraph is not a return value. A structured, checkable artifact is.
• A deterministic validator sits at the trust boundary. The model's answer is a candidate, not a result. Before it is incorporated, deterministic code checks it: does it satisfy the standard, does it preserve the input's meaning, does it fit the contract. On a pass, the caller incorporates it and moves on. On a fail, the caller retries or falls back. The trust boundary is the validator, not the model.

This is why DocAble can make a defensible claim on top of an infinite midwit. The model supplies the one thing deterministic code cannot: it looks at the figure and understands something. Everything around that understanding — when to ask, what to ask, whether to believe the answer — stays in code you can reason about and test. The probabilistic part is sealed inside a bounded call with a check on the way out. Guidance aims; machinery holds.

This discipline was earned the hard way: put a model somewhere without it and it will find the exception you did not guard. Agents are the fastest road to a working demo and the fastest road to a subtle, confident, wrong result. The function-call shape is what keeps the first from becoming the second.

Three views of the same system

A model shows a view. The sections below draw three architectural views of DocAble from its real system models. Each answers a different question. The first asks how the services are shaped, and why that shape let the whole system change its deployment target without a rewrite. The second asks how the front end is decomposed, and how the editor changes a document through a small edit language. The third connects that edit language to the automated pipeline, and shows the join as a concrete instance of the function-call pattern above. Read together, they are the standing structure the rest of the book governs.

View 1 — a reactive service seam, and how it made serverless natural

The back end is a set of small services wired to react to events. A user's upload lands at a web front door, which splits the document into chunks and enqueues each chunk as a unit of work. A queue hands each chunk to a stateless worker. The worker remediates its chunk and publishes a completion. A fan-in step collects the completions and assembles the finished file. Every hop is an event handed forward, not a component polling for work. Two commitments hold the seam together: services stay reactive and stateless, and the split of duties is declared — Redis carries communication, the database carries truth.

That shape is what made a hard migration easy. For most of the system's life the workers ran on a Kubernetes cluster, and the dispatch worked by poll: workers reached into size-stratified Redis queues and claimed the next chunk, while a custom autoscaler watched queue depth and added or removed worker pods. The cluster billed around the clock, which for a bursty, low-traffic product was the wrong cost shape. The fix was to move to a serverless push plane: a managed task queue hands each chunk to a worker that scales from zero, a managed topic fans in the completions, and the platform's native autoscaler replaces the custom one.

In the industry a re-platforming like that is a quarters-long migration with a rewritten coordinator and a nervous cutover. Here it was close to a change of deployment target. The reason is the seam. A reactive, input-triggered handler is the serverless execution model (an event in, work out), so the workers did not need new logic to be pushed to instead of polling. A stateless worker can be spun up on demand and thrown away, because it carries nothing between invocations. And the split that kept coordination in Redis and truth in the database meant the durable state did not live in any worker that serverless would recycle. The poll-to-push inversion then deleted complexity rather than adding it: the custom autoscaler, the one-cluster-per-prefix rule, and the idle-scaling controller all fell away, replaced by the platform's native scale-to-zero.

None of the three hardening pushes that produced this shape — the reactive conversion, the move to statelessness, the modeling of the cross-service invariants — was aimed at serverless. Each was motivated on its own terms. Their convergence is the lesson: a system built reactive, stateless, and modeled is, almost by accident, a system that is safe to run serverless. The reactive seam was not a feature of the migration. It was its precondition.

View 2 — the front end as model-view-controller over a shared edit language

The front end is a small single-page application plus a set of supporting surfaces — an account view, a job history, an operator dashboard, and the editor. The piece worth drawing is the editor, because it shows the cleanest decomposition. The editor is a model-view-controller loop over one document.

• The view is what the user sees: two panes. The left renders the page with overlay boxes marking each structural element; the right is a list of cards, one per element, with editable fields for alt text, role, decorative-or-not, and reading order.
• The controller takes a user gesture — edit this alt text, change this role, drag these children into a new order — and does one disciplined thing. It does not reach into the document and mutate it. It translates the gesture into a single typed edit operation.
• The model is the document's intermediate representation. Its only input is an edit operation. It applies the operation, returns the updated state, and the view re-renders from that state.

The operation the controller emits is drawn from a small, closed vocabulary: set the alt text, set the role, mark an element decorative, reorder the reading order, set the document title, set the language, override the displayed text. That vocabulary is a little language for changing a document — an edit language. The point of routing every gesture through it is the same point that runs under the whole system: a document has exactly one way to be changed. The editor never edits the document. It speaks the edit language, and the model is the only thing that touches the document.

The read-only diff views for slides and word processor files share this shape, differing only in the adapter that fetches their structure. One surface, one edit vocabulary, many formats. The uniformity is the payoff: a fix or a constraint applied to the edit language holds for every format and every producer that speaks it. This is the Modeling Thesis in miniature — a typed model binding what a change means to how the document is touched, so intent and implementation cannot drift apart.

View 3 — the edit language as the target of automated remediation

Here the two halves of the system join, and the join is the function-call pattern from earlier in the chapter, seen once more.

The edit language the editor speaks is not only the editor's. It is the same target the automated pipeline produces. When the pipeline decides a figure needs alt text, the result it emits is an edit operation — the very set-alt-text the editor would emit if a human did it by hand. The human path and the automated path are two producers of one language, and they flow into one document model that applies each operation and stamps it.

The mapping from a remediation task to an edit operation is a function: task in, operation out. And that function is a concrete instance of the LLM as a function call. Inside it, a bounded task — this figure, this context, this question — is packed into a typed input and handed to the model as a call; the model returns a candidate; and a deterministic validator checks the candidate before it is allowed to become an edit. The edit language is the typed output contract. An honest caveat: this task-to-edit function is mostly hardcoded today and will be fleshed out. What matters now is the shape and the trajectory, not a finished implementation — the concept is that automated remediation and hand editing are the same operation reached two ways.

Why route through the language at all

The frontier models can already skip all of this. Opus and the project's own agent can take a slide deck and remediate it directly — at least for slides — with no pipeline, no edit language, no validator. So why build the machinery? Because direct remediation buys neither of the two things a university needs. You cannot audit what a direct edit changed: the model hands back a file, not a list of typed changes you can inspect, explain, and reverse. And you cannot trust it: there is no deterministic check that the standard was met and the meaning survived. Routing every change through one edit language and one validated boundary buys both. Traceability comes free, because every edit is a typed, validated, stamped result with a history you can reconstruct. Confidence comes from the validator that sits at the trust boundary and refuses a candidate that fails.

There is a third dividend, and it connects this view to a habit the book returns to later. Scoping each model call to one bounded task also scopes the reasoning. A model asked to describe this figure under this context does less work, and costs fewer tokens, than a model asked to remediate a whole document in one shot. That token-scoping is the same move as building the remediation as a series of small, bounded transformations rather than one giant leap. It is that idea seen from the cost angle instead of the pipeline angle: a chain of scoped passes is cheaper to run and cheaper to reason about, because each link asks the model for exactly one small thing. (See: transformation — everything a model does well is a sized transformation, and sizing the leap is the skill.)

Keeping the models honest — the traceability substrate

The three views above were not drawn by hand. They were rendered from DocAble's real system models — typed records the fleet reasons through to operate a codebase no single agent can hold in its window. A model is only as good as its agreement with the code. Say the model still describes the Kubernetes poll plane after the serverless push plane replaced it, and every agent that trusts the model plans against a system that no longer exists. The map has to equal the territory, or the map lies.

The tempting way to keep them equal is a checklist: when you retire a subsystem, remember to update the model too. DocAble tried that and it failed the same way it always fails. A cutover retired the poll plane, the retirement doc dutifully reconciled the architecture docs and the agent boot context, and it forgot the models. The forgotten edge stayed green because nothing re-checked it. This is the pattern under every silent drift: a hand-maintained list rots the moment the code moves without it, and no one notices until the stale fact bites.

That failure is not one anecdote. The right sensor was not guessed at a whiteboard — the models were left deliberately ungoverned for weeks and the fleet allowed to drift them, so the fix could be read off real drift instead of imagined. An audit of the closed work, every job green at its own done-check, turned up two dozen silent divergences at a signal-to-noise near one: a checker tightened past the producer it now rejected, a typed function shipped and wired to nothing, a reconciliation lint gone red the instant its work closed. Sort those cases by what caught them and one line separates the clean from the drifted. The clean ones had a sensor that re-read the code at check time. The drifted ones had a second copy of the fact, maintained by hand, that quietly fell behind.

So DocAble does not keep a checklist. It keeps a traceability graph, and every edge in it is derived — re-checked against the code at scan time rather than trusted from the last time someone looked. The nodes are the things the system already has: a model element, the lint that enforces it, the code entry point that realizes it, the test that verifies it, a registry row, a doc surface. The edges are the joins between them — this invariant governs that code root, that test verifies this invariant. Each edge carries the mechanism that re-proves it: run the resolver against the target and see whether the symbol still exists.

Two decisions make the graph resist the rot a checklist suffers.

• Anchor to symbols, never to line numbers. An edge points at a function or class definition, resolved statically — not at a file-and-line. Line numbers churn under every edit above them; a symbol survives every edit that does not delete it. When five journey anchors drifted stale under a refactor, the cause was line-anchoring. A symbol anchor moves with the code it names.
• A missing symbol is a finding, not a crash. When an edge points at a symbol that no longer resolves — deleted, renamed, moved out — the edge goes red, and that red is exactly the drift becoming visible. The sharper case: when the model wants to point at a piece of logic for which no clean symbol exists, buried in a god-function, the absence is a signal that the code is missing an abstraction the model needs. The anchor doubles as a probe for where the code is under-factored.

The sensor that walks the graph runs at a slower cadence than the per-commit gates, and on purpose. Resolving a symbol against the code — a jedi or Roslyn round-trip per anchor — is expensive, far more than the cheap membership lookups a commit hook can afford. So the derived edge check fires at review and done-checking time, not on every commit: the cadence follows the cost. A second, cheaper guard sits on the anchors themselves, watching for the case a symbol walk cannot catch — a pointer left naming a dead implementation while a replacement was built beside it. That pointer-drift guard was earned by the most expensive drift of the build, a retired deploy driver whose stale pointers fired a prod-blocking deploy.

This is the same commitment as the one typed model under every format, one level up. There, a fix applied through the model holds for every format because there is exactly one seam. Here, a model stays true to the code because the join between them is re-derived, not remembered. The principle is worth stating flat: derived sensors defend; snapshotted ones drift. A sensor that reads the source of truth at check time cannot fall behind it. A hand-maintained copy of the same fact always can.

Field note — the drift you invite on purpose

Keeping models honest sounds like a rule you enforce from day one. DocAble did the opposite, and on purpose. The models were left deliberately ungoverned for weeks — built, watched, but not enforced — so the fix could be read off real drift instead of guessed at a whiteboard. Then came the audit.

The drift showed up exactly as the theory predicted. A checker was tightened past the very producer it now rejected. A typed function shipped and got wired to nothing. A model still named a service seam the code had moved off of. A test kept pinning a value the source had since changed. Two dozen of these turned up across the closed, green work — signal-to-noise near one, not a single false alarm.

The reason to invite the drift was to learn its shape. A sensor built from two dozen real divergences answers a question the whiteboard cannot: would this actually have caught the drift we saw? That is why the substrate above earns its keep — every edge in it defends against a divergence someone already watched happen.

Same song, second verse

Notice the shape of the argument, because the book is about to make it again at a larger scale. A model can do the one thing deterministic code cannot — look at a figure and understand it — and it is unreliable. You do not make it reliable by trusting it more. You make a reliable system by wrapping it: bound what you ask, type what comes back, validate before you believe. Inside DocAble, the wrapped component is a vision-language model writing alt text.

The rest of this book zooms out one level. The coding agents that built DocAble are the same kind of component: powerful, probabilistic, capable of a confident wrong turn. The methods in the chapters ahead are the same pattern applied to them — bound the task, type the interface, validate before you trust. You have now seen the pattern once, in the small, where it is easy to hold in your head. Watch for it again in the large. Same song, second verse.