Appendix C - 13. Test-onion tiers (Smoke / Lite / targeted / full)

The Structure of Test-onion tiers (Smoke / Lite / targeted / full) — its shape at a glance:

Each decision runs the cheapest tier that covers it: a targeted filter during editing, Lite and Smoke as fast gates, the full suite at deploy. Touching a high-blast-radius surface escalates to the full tier.

flowchart TD
  Edit[Edit loop] --> Targeted["Targeted filter (<5s)"]
  PreCommit[Commit] --> Lite["Lite (seconds)"]
  Deploy[Deploy] --> Smoke["Smoke pre-gate"]
  Smoke --> Full["Full suite (~minutes)"]
  Hot{High-blast-radius surface?} -->|yes| Full

Accessible description: the edit loop runs a sub-five-second targeted filter, a commit runs the Lite tier, and deploy runs a Smoke pre-gate then the full suite. A decision node routes any change touching a high-blast-radius surface straight to the full tier.

Projected from the catalogue entry regression-tests / Test-onion tiers (Smoke / Lite / targeted / full).

On this page: Intent · Motivation · Applicability · Structure · Sample Code · Consequences · Example use within DocAble · Related Patterns

Intent

Intent — A tiered test suite that stratifies ~4000+ tests by cost and coverage (Smoke, Lite, targeted, full) so cheap tiers gate fast iterations and the full tier gates deploy, pinning behaviour at the right price per decision.

Motivation

Four thousand-plus tests are too slow to run on every change, but skipping them lets regressions land. Left unstratified you get one of two failures: unusably slow iteration (run everything, always) or missed regressions (run nothing, hope). It recurs on every change and every deploy; the tension is constant.

Applicability

• A trait/filter system on the tests to slice them into tiers.
• Tiers defined by cost + coverage, and a discipline for which tier gates which decision.
• The 1-second rule + escalation rules, or Lite bloats and iteration-tier under-tests.

Structure

The Structure diagram appears at the top of this page.

Sample Code

Tiers are a selection over tagged tests, matched to the decision at hand. A trait on each test names its tier; the runner picks the cheapest tier that covers the decision. The escalation rule overrides the default: if the change touches a named high-blast-radius surface, force the full tier regardless.

TIERS = {"smoke": 0, "lite": 1, "targeted": 2, "full": 3}   # cheap → expensive

def select_tier(decision: str, touched: set[str], hot_surfaces: set[str]) -> str:
    """Pick the cheapest tier that gates this decision — unless the change touches a
    high-blast-radius surface, which forces the full tier no matter the decision."""
    if touched & hot_surfaces:
        return "full"                                       # escalation overrides
    return {"edit": "targeted", "commit": "lite", "deploy": "full"}[decision]

def run(tests: list[dict], tier: str) -> list[dict]:
    ceiling = TIERS[tier]
    # a test runs if its own tier is at or below the selected ceiling
    return [t for t in tests if TIERS[t["tier"]] <= ceiling]

Consequences

• Tier assignment is judgment. A mis-tiered slow test in Lite erodes the whole tier's speed promise.
• Escalation depends on discipline. The rules only help if agents actually run the full tier when they touch a named surface.
• The full tier is still ~4min — the deliberate price of the deploy gate.

Example use within DocAble

• The Suite=Smoke / Suite=Lite / targeted / Format= full tiers.
• The 1-second rule for new pre-deploy tests; the Tier-3 escalation surfaces.

Related Patterns

• See also (sibling) — property-tests, fuzz-campaigns, ddt-pin-trailers: the other behaviour-pinning bodies in this family.
• Layer — the full tier gates staged-deploy-gates (agent side): the deploy stair runs this suite.