Provenance Models: What Did the System Do, and What Did We Observe?

James C. Davis

2.7 Provenance Models: What Did the System Do, and What Did We Observe?

The preceding model families describe what a system contains, what it may do, who may control something, what choices are permitted, or how much of some quantity is acceptable. Provenance looks backward at a realized execution: what did the system actually do, and what evidence did it retain about what happened?

Software engineers already make running systems observable. A log records selected events. A trace connects events across an execution. Metrics summarize quantities over many events. Audit records preserve actions that may later require explanation. Together, such mechanisms can expose an enormous amount of runtime history.

But observable history is not automatically useful engineering knowledge. A production system may emit millions of events while still making a consequential question difficult to answer: what changed this artifact, which operation changed it, and what evidence supports that account? A provenance model selects and structures the history needed to answer such questions.

2.7.1 The Canonical Move: From Observability to Provenance

The canonical move is to turn selected observations into structured history. An execution exposes far more than an engineering question usually needs: function calls, retries, intermediate values, network requests, model invocations, allocations, and other events. A sufficiently detailed trace may capture much of that execution. A provenance model deliberately keeps less.

The model identifies the consequential entities and operations, gives them stable identities, and records the relationships needed to reconstruct what happened. An artifact may identify the source from which it was derived, the operation that produced it, the component responsible for that operation, and the evidence retained with the change. The exact representation can vary. What makes it provenance is the meaning of the recorded history, not a particular graph notation or data format (Figure 2.7-1).

Figure 2.7-1. From observability to provenance. Logs, traces, and other runtime observations can expose far more of an execution than an engineering question needs. A provenance model retains the consequential entities, operations, relationships, and evidence needed to reconstruct selected history.

That reduction makes questions about realized history tractable. An engineer can trace an artifact backward to its source, follow an operation forward to the artifacts it affected, determine which operation produced a change, or reconstruct the evidence available during an audit. A question that would otherwise require searching and interpreting logs becomes a query over selected, structured history.

2.7.2 DocAble: Attribution

For DocAble, the consequential history is the history of changes to the user's document. A general execution log could record every function call, retry, model request, intermediate value, and service interaction involved in remediation. A distributed trace could connect much of that work across components. Those mechanisms make execution observable, but they do not by themselves answer the question DocAble needs to preserve: what changed in this document, what operation changed it, and what evidence supports that change?

DocAble therefore records consequential mutations as structured operations. Each operation identifies its target, the mutation made, the remediation pass responsible, and the evidence retained about the change. The resulting record is narrower than a general execution trace and more directly useful for explanation, debugging, and audit.

MODEL CARD
Attribution model · Provenance

Engineering question — What happened to this artifact, which operation produced the change, and what evidence records it?
Model — a run made of operations, each naming its target, its mutation, the pass that made it, and the evidence it left.
Property — consequential operations have structured attribution linking the action, target, mutation, responsible pass, and retained evidence.
Quality attribute — auditability, debuggability, and consistency between what changed and what is reported.

Figure 2.7-2 draws a reduced provenance model: a run contains operations, and each operation names a target, a mutation, the pass responsible, and the evidence it left. This is not a trace of every function call. That would reproduce execution rather than model it. The model keeps only the events needed for explanation and audit.

Figure 2.7-2. Attribution provenance model. A run contains consequential operations; each operation records its target, mutation, responsible pass, and retained evidence. Audit and explanation can be derived from this shared history.

Human-facing explanations can be derived from the same structured history rather than maintained separately. Record the consequential action once, then derive the audit view, debugging view, or changelog from it. That reduces the chance that what the system did and what its documentation says will drift apart.

Recording execution is not the same as modeling its history. An execution contains far more activity than an engineer ordinarily wants to preserve as provenance. Too coarse a record collapses distinct consequential changes into an opaque "document modified" event. Too fine a record becomes another execution trace that later reasoning must reconstruct into meaning. Choosing the grain is therefore part of the modeling decision.

For DocAble, the useful grain is the consequential mutation. The provenance record answers the questions for which it exists: what changed, what operation changed it, what evidence was retained, and what later explanation can be derived from that history.

2.7.3 When Provenance Becomes Executable

DocAble's edit record supports one additional capability: the recorded history can be replayed. During a PDF remediation session, document mutations are recorded as typed edit values in a per-session log — roughly thirty variants covering the changes the editor can make. A replay engine can apply the same sequence deterministically. The same representation can therefore explain what happened and reproduce the resulting mutations.

Replay is not what makes the record a provenance model. It is an additional capability enabled by this particular representation. The record remains a purposeful reduction of execution: it omits the reasoning, analyses, control decisions, retries, and other activity that produced the edits. It records the consequential mutations that resulted.

That separates three engineering questions that are easy to conflate. The computation graph asks what computations may compose. The provenance record says which consequential mutations did occur in this run. Replay asks whether that realized history can be reproduced. No one representation subsumes the others. Figure 2.7-3 sets the static structure and the realized history side by side — the reader's first sight of two heterogeneous models describing the same operation from different sides, which is the work the next chapter takes up.

Figure 2.7-3. Computation structure versus realized history. The computation graph models declared computations and their typed composition; the per-session edit record logs the consequential mutations one run actually produced. They remain separate reductions today. A shared computation identity could join runtime edits to the static model when an engineering question requires that relation, without collapsing the two representations into one.

One limit closes the chapter. A provenance model does not guarantee that every consequential operation actually leaves the required record. DocAble addresses that separately, by requiring typed mutation verbs to emit attribution and checking that obligation structurally. That check is a hard presence guarantee over a soft content claim: it can establish that every mutator leaves an attribution record, but not that the semantic account inside that record is correct — that remains the author's. Part II models the evidence an operation should leave; Part III asks what mechanism enforces that obligation.

The next and final chapter of this Part steps back from any single model to ask how the six fit together as a system.