Coverage Closure

Functional Coverage · Module 19 · Page 19.7

The Engineering Problem

Every prior chapter in this module built or read coverage; this one decides with it: when is verification actually done? That decision — coverage closure — is where functional coverage meets the project, and it is fraught, because the obvious answer ("when coverage hits 100%") is wrong in both directions. 100% is not sufficient: a model at 100% with weak checking has verified nothing (Module 19.1's quadrants), and 100% of a badly-written model (coarse bins, missing coverpoints) is false confidence (19.3–19.4). 100% is not necessary: some bins are unreachable (so 100% is impossible — 19.6), and some gaps are legitimately waived. Worse, the pressure to "close coverage" tempts teams to game the number — deleting hard-to-hit coverpoints or coarsening bins until the model reads 100% — producing a number that ships the very bugs the model was meant to catch. The problem this chapter solves is closure as a defensible decision: when verification is done — relative to a plan, combining all the metrics, with every gap accounted for — signed off and auditable, not a number crossing a line.

Coverage closure is the defensible decision that verification is done — made relative to a coverage plan and sign-off criteria, combining functional coverage with checking, code coverage, and a clean regression, with every gap accounted for (closed, ignored, or waived with documented justification). It is a judgment, reviewed and signed off, not a raw percentage. The plan (the features/scenarios to exercise, from the spec — 19.1) and the criteria (which covergroups must reach what, which gaps are waivable) define the target before you start. 100% is neither necessary (unreachable bins make it impossible; justified waivers make it unneeded) nor sufficient (100% of a weak model, or without checking, verifies nothing). Closure is multi-dimensional — functional coverage and checking and code coverage and a passing regression must all hold (the conjunction) — and every unhit bin must have a disposition: closed (exercised and checked), ignored (unreachable), or waived (deferred/low-risk, with a documented, reviewed reason). And the cardinal discipline: you close the gaps or waive them honestly — you never weaken the model to hit the number. This chapter is coverage closure: the plan and criteria, why 100% is the wrong target, the multi-metric conjunction, waivers, and defensible sign-off.

What is coverage closure — how do a coverage plan and sign-off criteria define when verification is done, why is 100% neither necessary nor sufficient, how does closure combine functional coverage with checking and code coverage and a clean regression, and why is weakening the model to hit the number false closure?

Motivation — why "done" must be defined, not declared at 100%

A verification effort without a defined, defensible notion of done either never ends or ends arbitrarily. The reasons closure is a discipline, not a number:

• "Done" needs a definition agreed up front. Without a coverage plan and sign-off criteria, "closure" is a gut call — someone decides it's "enough." The plan defines what "done" means before the work, making the decision objective and defensible.
• 100% is the wrong target — both ways. It's not sufficient (100% of a weak model, or without checking, verifies nothing) and not necessary (unreachable bins make it impossible; justified waivers make it unneeded). Aiming at 100% misframes the goal.
• Closure is multi-dimensional. Functional coverage alone isn't done — checking must be in place (else coverage measures exercise without verification), code coverage closed (no unexecuted RTL), regression clean, assertions covered, bug-rate trending down. Closure is the conjunction of all the evidence.
• Every gap must be accounted for. A defensible closure has no unexplained gap: each unhit bin is closed, ignored (unreachable), or waived (with a documented, reviewed reason). Unaccounted gaps are exactly where escaped bugs hide.
• The number must not be gamed. Under schedule pressure, weakening the model (deleting coverpoints, coarsening bins) to reach 100% is a temptation that produces a meaningless number and ships the corner bugs. Closure must measure the real plan, honestly.

The motivation, in one line: a verification effort needs a defined, defensible "done" — not a gut call and not "100%" — so closure is a discipline: a decision relative to an agreed plan and criteria, combining functional coverage + checking + code coverage + regression, with every gap accounted for (closed / ignored / waived-with-reason) and the model kept honest (never gamed) — signed off and auditable, the defensible answer to "are we done?".

Mental Model

Hold coverage closure as balancing the books and signing the audit — every gap is a line item that must reconcile, and the sign-off certifies the books balance against the agreed standard:

Coverage closure is closing the books and signing the audit. Every gap in the coverage is a line item, and closure means every line item reconciles to a documented disposition: closed (exercised and checked), ignored (unreachable, so it never belonged on the books), or waived (deferred or low-risk, with a written, reviewed justification). The sign-off is the audit certificate — a reviewed statement that the books balance against the agreed standard: the plan's items are all dispositioned, checking is in place, code coverage is met, and the regression is clean. It is not "the total hit 100%"; it's "every line is explained, and the standard is met." Picture an accountant closing the books at period-end. Closure is not "the balance is a round number" — it's "every line item reconciles." For verification, each gap (each unhit bin) is a line item that must be explained. A closed item is exercised and checked — the scenario happened and was verified. An ignored item is unreachable — it never belonged on the books (a reserved value, an impossible combination), so it's struck off with a note (19.4/19.6). A waived item is deferred or low-risk — a deliberate, documented, reviewed decision to exclude it from the closure requirement with a stated reason ("this feature is out of scope for this tapeout," "this combination is low-risk and tracked for the next revision"). What closure forbids is an unexplained line — a gap that's neither closed, ignored, nor waived, just quietly below the bar. The sign-off is the audit certificate: a reviewed statement that the books balance against the agreed standard — every plan item dispositioned, checking in place, code coverage met, regression clean — signed by someone who vouches for it. Two frauds the audit catches: cooking the books (waiving real gaps without justification to hit the date) and changing the standard to match the books (weakening the plan/model — deleting coverpoints, coarsening bins — so the number reads 100% without doing the verification). A clean audit resists both: the standard (plan) is fixed up front, every gap is honestly dispositioned, and the certificate is defensible under review.

So coverage closure is balancing the books and signing the audit: every gap is a line item that must reconcile to a disposition (closed / ignored / waived-with-reason), the standard is the agreed plan and criteria (fixed up front), and the sign-off is the reviewed certificate that the books balance — checking in place, code coverage met, regression clean. 100% is not the goal (the books can balance below it via ignores/waivers, and a round number doesn't mean they balance); honest reconciliation against a fixed standard is. Close the books honestly — disposition every gap, keep the standard fixed, and sign only what you can defend.

Visual Explanation — closure is a conjunction of dimensions

The defining picture is the conjunction: closure is not functional coverage alone — it requires functional coverage AND checking AND code coverage AND a clean regression, all satisfied.

The figure shows closure as a conjunction. Functional coverage closed (the brand-colored top) — the plan's scenarios exercised, every meaningful bin closed, ignored, or waived (Modules 19.1–19.6). AND checking in place — the scoreboard verified what was exercised (coverage without checking measures exercise, not correctness — 19.1). AND code coverage closed — no unexecuted RTL (the automatic safety net catching dead code and unentered branches). AND a clean regression — all tests passing, assertions covered, bug-rate trending to zero (the project-level evidence). The crucial reading is the AND: closure is the conjunction of all these dimensions, not any single one. Functional coverage at 100% with weak checking is not closure (the scenarios were exercised but not verified — 19.1's covered-but-not-checked quadrant). 100% functional coverage with failing tests is not closure. 100% functional with dead code (unclosed code coverage) is not closure (something is unexercised the functional model didn't even measure). Each dimension catches what the others miss: functional coverage measures intent (did we exercise the features?), code coverage measures execution (did we run the RTL?), checking measures correctness (was it right?), and regression measures stability (does it all still pass?). The brand-colored functional-coverage and checking layers are the intent dimensions (this module and Module 18); the success-colored code-coverage and regression layers are the execution and stability dimensions. The sign-off certifies every dimension is met — not that one number is high. The diagram is the closure conjunction: functional AND checking AND code AND regression — all satisfied, together, is done; any one alone is not. Closure is the AND of all the evidence, not a single high number.

RTL / Simulation Perspective — the sign-off criteria and gap dispositions

In practice, closure is written down — a sign-off checklist of criteria and a disposition for every gap. The example sketches the closure record.

=== COVERAGE CLOSURE SIGN-OFF: block "dma_engine", tapeout rev B ===
 
PLAN & CRITERIA (agreed up front):
  - functional coverage: all covergroups 100% of REACHABLE bins (ignores excluded)
  - checking: scoreboard active on every transaction; 0 unexplained mismatches
  - code coverage: >= 95% line/branch; every gap reviewed
  - regression: 5000 seeds, 100% passing; bug-rate zero for 2 weeks
 
GAP DISPOSITIONS (every unhit bin accounted for — closed / ignored / waived):
  cp_resp.error_resp ........ CLOSED   (directed test err_inject exercises it; scoreboard checks it)
  cp_mode.bins[5:7] ......... IGNORED  (reserved modes; DUT cannot produce — ignore_bins, unreachable)
  x_op_size.rmw_max ......... WAIVED   (RMW-at-max deferred to rev C; low-risk, tracked in JIRA-1234)
  cp_len.boundary_max ....... CLOSED   (directed test hits len==MAX; off-by-one checked)
 
MULTI-DIMENSIONAL STATUS:
  [x] functional coverage: 100% of reachable bins   [x] checking: active, 0 unexplained
  [x] code coverage: 96% (gaps reviewed, dead code removed)   [x] regression: 5000/5000, bug-rate 0
 
  → SIGN-OFF: verification lead + design lead review and sign. Defensible, auditable "done".
 
✗ FALSE CLOSURE (forbidden): coverage stuck at 92% on hard corners →
   "fix" by DELETING cp_len.boundary_max and COARSENING cp_size to one bin → model reads 100%
   → the corner bugs the model was built to catch now SHIP. Gaming the metric is not closure.

The record shows closure as a written, defensible decision. Plan & criteria (agreed up front): functional coverage (all covergroups 100% of reachable bins), checking (scoreboard active, 0 unexplained mismatches), code coverage (≥95%, gaps reviewed), regression (5000 seeds, 100% passing, bug-rate zero). Gap dispositions — every unhit bin accounted for: cp_resp.error_resp CLOSED (a directed test exercises it, the scoreboard checks it); cp_mode.bins[5:7] IGNORED (reserved modes, unreachable — 19.4); x_op_size.rmw_max WAIVED (deferred to rev C, low-risk, tracked in a ticket); cp_len.boundary_max CLOSED (directed test, off-by-one checked). Multi-dimensional status — all four dimensions checked (functional, checking, code, regression), then SIGN-OFF by the verification and design leads — defensible, auditable. The false closure (the forbidden anti-pattern): coverage stuck at 92% on hard corners, "fixed" by deleting cp_len.boundary_max and coarsening cp_size to one bin so the model reads 100% — now the corner bugs the model was built to catch ship. The shape to carry: closure is a record, not a number — criteria fixed up front, every gap dispositioned (closed/ignored/waived-with-reason), all dimensions satisfied, reviewed and signed. The dispositions are the core: no gap is unexplained — each is closed (exercised+checked), ignored (unreachable), or waived (deferred, with a reason and a tracking ticket). And the false-closure line is the cardinal warning: gaming the model to hit the number is not closure — it produces the number while destroying the measure. Write the criteria, disposition every gap, satisfy every dimension, and sign — don't game the number.

Verification Perspective — why 100% is neither necessary nor sufficient

The central closure insight is that 100% is the wrong target — in both directions. Seeing both failures together reframes what closure is.

The figure shows why 100% is the wrong target. Not sufficient: 100% functional coverage with weak checking verifies nothing (scenarios exercised but unchecked — 19.1), and 100% of a weak model (coarse bins, missing coverpoints) is false confidence (19.3–19.4) — a bad plan at 100% is meaningless. Not necessary: unreachable bins make 100% impossible (they must be ignored — 19.6), and low-risk or out-of-scope gaps are legitimately waived — so defensible closure is often below 100%. The verification insight is that both failures flow from the same error: treating 100% as the goal instead of defensible coverage of the agreed plan. The not-sufficient side shows that the number can be high while the verification is hollow — because the number says nothing about checking (correctness) or the model's quality (whether the bins mean anything). The not-necessary side shows that the number can be below 100% while the verification is complete — because some bins can't be hit (unreachable) and some shouldn't block the date (justified waivers). So 100% is neither a guarantee of done nor a requirement for done — it's orthogonal to the real question: is the agreed plan defensibly covered, with all dimensions met? The brand-colored not-sufficient / not-necessary branches both point away from the warning-colored "aiming at 100%": the target isn't a percentage, it's a defensible disposition of every gap against a fixed plan. This reframes closure entirely: don't ask "did we hit 100%?" — ask "is every meaningful scenario exercised and checked, every unreachable bin ignored, every remaining gap waived with justification, and every other dimension met?" The diagram is the reframe: 100% is neither necessary nor sufficient, so closure is defensible coverage of the agreed plan, not the number. Stop aiming at 100%; aim at a defensible disposition of every gap against a fixed plan.

Runtime / Execution Flow — reconciling every gap to a disposition

The operational form of closure is a reconciliation: walk every unhit bin and assign it a disposition, then check all dimensions and sign. The flow shows it.

The flow shows closure as a reconciliation. Walk (step 1): from the analysis (19.6), enumerate every gap — no gap may be left unexamined. Disposition (step 2): assign each one — closed (exercise and check it, via a directed test), ignored (unreachable — reserved or impossible), or waived (low-risk/out-of-scope, documented and reviewed, with a tracking ticket). Verify dimensions (step 3): functional coverage, checking, code coverage, and a clean regression — the conjunction, not just functional. Sign (step 4): verification and design leads review the dispositions and dimensions and sign — a defensible, auditable done. The runtime insight is that closure is exhaustive and explicit: every gap gets a disposition — there is no "unexplained" category. This is what makes closure defensible: a reviewer (or an auditor after a silicon bug) can trace every gap to a documented decision — "this was closed by test X," "this was ignored as unreachable," "this was waived as low-risk, tracked in ticket Y." The discipline is in step 2: a waiver is not a silent skip — it's a documented, reviewed decision with a reason and a ticket, so waiving is honest (the gap is acknowledged and tracked, not hidden). And step 3 enforces the conjunction — functional coverage closed is necessary but not sufficient; checking, code coverage, and regression must also hold. The sign-off (step 4) is the certificate: reviewed by more than one person, vouching that the books balance. The flow is the closure reconciliation: walk every gap → disposition each → verify all dimensions → review and sign — an auditable trail from every gap to a decision, and from all dimensions to a signed "done." The defining property is that nothing is unaccounted for: every gap is explained, every dimension is met, and the decision is signed. Disposition every gap explicitly, verify every dimension, and sign — defensible closure leaves nothing unexplained.

Waveform Perspective — the closure trajectory of a project

Across a project, closure is a trajectory: coverage rises toward the target, the bug-rate falls toward zero, and closure is declared when the criteria are met — not at an arbitrary 100%. The waveform shows the project-level shape.

The waveform shows the project-level closure trajectory. Functional coverage (cov_at_target) rises over the weeks toward the plan's target and reaches it. The discovered bug-rate (bugs_found) starts high (testing actively finds defects), then falls toward zero as the design stabilizes. The checking and regression criteria (criteria_met) come true as the scoreboard runs clean and the regression passes. Closure (signoff) is asserted only when all criteria hold together. The crucial reading is the conjunction at sign-off: closure is not declared when coverage alone hits a number — it's declared when coverage is at target AND bug-rate is at zero AND checking/regression are clean, all together (at week 9 here), which is below an arbitrary 100% and gated on the conjunction. The bug-rate trajectory is a key closure signal beyond coverage: a falling bug-rate that has reached zero (and stayed there) is evidence that the testing has stopped finding bugs — a complementary indicator to coverage (which measures what was exercised). A rising coverage with a still-high bug-rate is not closure (bugs still being found); a plateaued coverage with a zero bug-rate for a sustained period and the criteria met is. The picture to carry is that closure is a multi-signal decision over time: coverage to target, bug-rate to zero, checking and regression clean — converging to a defensible sign-off, not a single number crossing a line. Reading the trajectory this way — are all the criteria met together, sustained? — is reading closure as a project state. The sign-off asserting only when all criteria hold is the signature of defensible closure: gated on the conjunction, declared when the agreed bar is met across every dimension, not when a percentage happens to read 100%. Closure is the conjunction of criteria met and sustained, not a number on one axis.

DebugLab — the gamed 100% that shipped the corner bug

✗ GAME the metric — weaken the model until it reads 100%:
  // coverage stuck at 92%; hard corners (boundary_max, err_during_burst) won't close under schedule
  // "fix":  delete cp_len.boundary_max;  delete x_err_burst;  coarsen cp_size to a single bin
  // model now reads 100% → "closure achieved" → SIGN OFF
  // but the corner SCENARIOS are still UNEXERCISED and UNCHECKED → the boundary-burst bug SHIPS
  // 100% of a DEGRADED model measures nothing — the gaps didn't close, the MEASURE was destroyed
 
✓ KEEP the model honest — close the gaps or waive them with justification:
  // option A (close): add directed tests for boundary_max and err_during_burst → exercise + check them
  // option B (waive): if genuinely low-risk/out-of-scope, WAIVE with a documented, reviewed reason + ticket
  //   "x_err_burst deferred to rev C; risk assessed low; tracked in JIRA-5678; design lead approved"
  // either way the gap is HONESTLY dispositioned and the MODEL still measures the real plan
  // closure = defensible coverage of the AGREED plan, never a number reached by degrading the measure

Common Mistakes

• Treating 100% as the goal. It's neither necessary (unreachable/waived gaps) nor sufficient (weak checking or a weak model); aim at defensible coverage of the agreed plan.
• Closing on functional coverage alone. Closure is the conjunction — functional AND checking AND code coverage AND a clean regression; one dimension alone isn't done.
• Gaming the model to hit the number. Deleting coverpoints or coarsening bins to reach 100% destroys the measure and ships the corner bugs; close or waive honestly.
• Waiving without documentation. A silent skip is sweeping a gap under the rug; a waiver is a written, risk-assessed, reviewed, ticketed decision.
• Leaving gaps unexplained. Every unhit bin must have a disposition — closed, ignored, or waived; an unaccounted gap is where escaped bugs hide.
• No agreed plan or criteria. Without a coverage plan and sign-off criteria fixed up front, closure is an arbitrary gut call, not a defensible decision.

Senior Design Review Notes

Interview Insights

Exercises

1. Write the criteria. Draft sign-off criteria for a block, naming the functional, checking, code-coverage, and regression thresholds.
2. Disposition the gaps. For four unhit bins — an error path, a reserved value, a deferred feature, and a boundary case — assign each a disposition and justify it.
3. Argue the number. Explain why a defensible closure at 96% can be stronger than a gamed 100%, citing both why 100% isn't necessary and isn't sufficient.
4. Catch the gaming. Describe what you'd inspect to determine whether a jump to 100% near sign-off was real or a weakened model.

Summary

• Coverage closure is the defensible decision that verification is done — relative to a coverage plan and sign-off criteria, combining functional coverage with checking, code coverage, and a clean regression, with every gap accounted for (closed / ignored / waived-with-reason) — a signed-off, auditable judgment, not a raw percentage.
• 100% is neither necessary (unreachable bins make it impossible; justified waivers make it unneeded) nor sufficient (100% of a weak model, or without checking, verifies nothing — a bad plan at 100% is meaningless), so closure is defensible coverage of the agreed plan, not the number.
• Closure is a conjunction: functional coverage AND checking AND code coverage AND a clean regression must all hold; functional coverage alone is necessary but not sufficient.
• Every gap gets a disposition — closed (exercised + checked), ignored (unreachable), or waived (low-risk/out-of-scope, documented, risk-assessed, reviewed, ticketed) — and nothing is left unexplained.
• The durable rule of thumb: closure is defensible coverage of the agreed plan, signed off across all dimensions — fix the coverage model up front and never weaken it to hit the number, disposition every gap (close it with stimulus, ignore it as unreachable, or waive it with documented review), and require functional, checking, code, and regression all met; 100% is neither necessary nor sufficient, and a number reached by gaming the model is false closure that ships the very corner bugs the model was built to catch.

Next — Stimulus Strategy: the next module turns to where coverage gets closed — constrained-random verification. Coverage measures what was exercised; constrained-random is how you exercise it at scale. The module opens with stimulus strategy: directed versus constrained-random versus coverage-driven, when each fits, and how the stimulus plan and the coverage plan together drive a design from first test to closure.