Appendix C - 7. ContentValidator (input ⊆ output fidelity)

The Structure of ContentValidator (input ⊆ output fidelity) — its shape at a glance:

The gate extracts content from input and output and checks the subset relation. A subset passes and the job proceeds; a violation fails the job. A staging per-pass variant localizes the drop to a specific pass.

flowchart LR
  In[/Input content/] --> G{Input ⊆ output?}
  Out[/Output content/] --> G
  G -->|yes| Pass([job proceeds])
  G -->|no| Fail([job fails])
  G -.->|per-pass, staging| Which([which pass dropped it])

Accessible description: the fidelity gate extracts content from the input and the output and checks whether the input's content is a subset of the output's. A subset lets the job proceed; a violation fails the job. A staging-only per-pass variant reports which pass dropped the content.

Projected from the catalogue entry validation-and-conformance / ContentValidator (input ⊆ output fidelity).

On this page: Intent · Motivation · Applicability · Structure · Sample Code · Consequences · Example use within DocAble · Related Patterns

Intent

Intent — A deterministic gate asserting that every piece of the user's original content survives remediation (input content ⊆ output content), run in production, with a per-pass variant in staging that pinpoints which pass dropped content (our instance: the ContentValidator fidelity gate).

Motivation

Remediation mutates the document across many passes and four formats. A bug in any pass could silently drop or alter user content: a deleted paragraph, a mangled table, a lost caption the author never sees go. For a fidelity-critical tool that is the worst possible outcome: the output looks fine and quietly isn't what the author wrote. The failure recurs on every remediation pass.

Applicability

A content extraction comparable across input and output — the subset check is only as good as what "content" is defined to be.
A subset predicate that tolerates legitimate reformatting/reordering without false positives.
A run point post-remediation and pre-delivery, plus a per-pass hook for localization.

Structure

The Structure diagram appears at the top of this page.

Sample Code

The gate is a subset check over normalized content. Extract content from input and output, normalize away legitimate reformatting (whitespace, order), and assert nothing in the input is missing from the output. Everything rests on the extractor and the normalization — too loose and it cries wolf, too tight and it misses a real drop.

def normalize(items) -> set[str]:
    # collapse legitimate reformatting so reordering/whitespace don't trip the gate
    return {" ".join(s.split()).casefold() for s in items if s.strip()}

def fidelity_gate(input_content, output_content) -> list[str]:
    """Deterministic post-condition: every piece of input content must survive to
    the output. A missing item is a silent drop — the worst failure for a
    fidelity-critical tool, and invisible to a spot-check."""
    src, dst = normalize(input_content), normalize(output_content)
    return [f"content lost in remediation: {item!r}" for item in sorted(src - dst)]

# in production: any finding fails the job
def run_job(extract, produce_output, source):
    findings = fidelity_gate(extract(source), extract(produce_output(source)))
    if findings:
        raise FidelityError(findings)      # fail the job — don't ship lossy output

Consequences

Everything rests on the extractor. A lossy or over-eager content extractor produces false positives (blocks good output) or false negatives (misses a real drop).
Subset semantics are subtle. Reordering, whitespace, and reformatting must be normalized or the gate cries wolf.
It runs on every job — a real but accepted cost for a fidelity guarantee.

Example use within DocAble

ContentValidator — the production input-⊆-output fidelity gate.
The staging per-pass variant (a dedicated fidelity marker on stdout localizes the offending pass).
The accessibility-remediation policy it enforces (never drop content the user wrote).

Layer — with standards-rule-engine: both are product gates over the artifact, fidelity (nothing lost) and conformance (standards met).
Counterpart — the per-pass staging variant localizes what the prod gate only detects.
See also (sibling) — coherence-lints: the other deterministic checks in this family.