Verification Challenges

Verification Foundations · Module 1 · Page 1.4

The Engineering Problem

We have established that verification exists, that it must be independent, and that it dominates the budget. None of that explains why it is genuinely hard — and "expensive" is not the same as "hard." You could throw money at an easy-but-large problem and finish. Verification resists money because it runs into four obstacles that no amount of brute force removes:

1. Controllability — you can only push the design from the outside (its inputs). You cannot teleport it into an arbitrary internal state, so reaching the state where a bug lives can require a precise, rare sequence you must construct.
2. Observability — you can only see what leaks out at the boundary (its outputs). A bug can corrupt internal state and stay invisible for thousands of cycles — or be masked entirely — so even when you reach it, you may not detect it.
3. Reproducibility — random and concurrent failures may not repeat on a naive re-run, so you can hit a bug and then be unable to get back to it to diagnose it.
4. Concurrency — outcomes depend on the ordering of parallel processes and clock domains, so the same stimulus can pass or fail depending on interleavings the simulator chooses.

The engineering problem of this lesson: a chip is a vast, dark, non-deterministic system you can only poke from the edges and only watch at the edges. How do you find the bug in there?

Motivation — why brute force and intuition both fail

The instinct is "drive lots of inputs and watch the outputs." That instinct collides with all four obstacles at once:

• Random input alone rarely constructs the rare, ordered sequence that reaches a deep state (controllability). The interesting bug needs the FIFO full and an error injected and a back-pressure stall, in that order — random poking lands there by luck, if ever.
• Watching outputs alone misses bugs that corrupt internal state silently or get masked before they reach a pin (observability). The boundary is a keyhole into a mansion.
• A bug that appears once in a thousand random runs and then hides on re-run cannot be debugged by staring harder (reproducibility).
• A test that passes is not proof when the result depended on which of two processes the scheduler happened to run first (concurrency).

So verification cannot be "more inputs, watch outputs." It must be an engineered attack on each obstacle — which is exactly what the discipline is.

Mental Model

Hold this picture:

Verification is searching a vast, dark maze you can only enter from one door and only watch through one window. The maze (the state space) is astronomically large. The door is the input boundary — you can only nudge the design inward from there (controllability). The window is the output boundary — you can only see what reaches it (observability). And the maze rearranges itself between runs (non-determinism from concurrency), so the path that found a monster yesterday may not find it today (reproducibility).

The discipline is the set of tools that make the maze smaller (coverage tells you which corridors you've walked), the door more capable (constrained-random + targeted stimulus reaches deeper rooms), the walls transparent (assertions and white-box monitors light up the interior), and the maze hold still (seed capture so a path replays exactly).

Visual Explanation — the two gaps: controllability and observability

Every bug hunt is bounded by two gaps between you and the bug. You act only at the input boundary; you see only at the output boundary; the bug lives in the dark interior between them.

Naming the two gaps is the whole foundation of test design. "How do I make the bug happen?" is a controllability question (answered by stimulus). "How do I make the bug visible?" is an observability question (answered by checking). Confuse them and you build a testbench that reaches the bug but never sees it, or sees clearly but can never get there.

RTL / Simulation Perspective — a bug that is hard to reach and hard to see

Consider a small write-buffer with a parity-protected entry. The bug: when an entry is overwritten while a read of it is in flight, a stale parity bit is latched — corrupting the entry silently.

// The corruption requires THREE things in order (controllability is hard):
//   1. entry valid,  2. a read launched against it,  3. an overwrite same cycle.
// And the corrupted parity is only checked on a LATER read (observability is hard).
always_ff @(posedge clk) begin
  if (wr_en) begin
    data[waddr]   <= wdata;
    parity[waddr] <= ^wdata;          // BUG: not held stable across an in-flight read
  end
  if (rd_en) rdata_q <= data[raddr];  // read is pipelined — parity checked a cycle later
end

To even reach the bug you must construct the exact overlap of read-in-flight and overwrite (controllability). Having reached it, the corruption sits in parity[] and produces no wrong output until some future read of that entry checks parity (observability). A directed test that writes, then reads, then writes — politely, never overlapping — never reaches it; a test that watches only rdata never sees it until cycles later, somewhere else.

Verification Perspective — how the discipline attacks each obstacle

The discipline is not one trick; it is a matched counter for each hard problem. (You will build each of these later in this track — here, just see the map.)

The pattern is consistent: stimulus attacks controllability, checking attacks observability, seed discipline attacks reproducibility, defined-region sampling attacks concurrency, and coverage attacks the unobservable question of completeness. Every component you will later build in UVM is one of these counters made reusable.

Runtime / Execution Flow — concurrency makes "it passed" conditional

Concurrency is the obstacle that turns a pass into a maybe. When two processes touch shared state, the result depends on which the scheduler runs first — and that can be seed- or tool-dependent. The same stimulus has two legal interleavings with two different outcomes.

This is why a concurrency bug can pass 999 random runs and fail the 1,000th, and why "I ran it and it passed" is not evidence in a concurrent system. The discipline's answer is twofold: sample shared state in a defined scheduling region (so the race resolves deterministically and observably), and make checks order-independent, so a legal interleaving cannot masquerade as a failure — or, worse, a bug cannot masquerade as a pass.

Waveform Perspective — observability: a bug invisible at the boundary

Watch the write-buffer corruption. The internal state goes bad early, but the output stays clean — the bug is unobservable from the boundary — until a later read finally exposes it.

The lesson is the observability half of the discipline: if you only check out, you are blind from cycle 3 to cycle 8, and in a real design that blindness can span thousands of cycles or hide the bug forever (if that entry is never read). The counter is white-box observability — assertions and monitors that watch internal contracts directly, so the violation is reported where it happens, not where it eventually leaks.

DebugLab — the bug that passed 999 seeds and failed one

driver:  bus <= new_value;        // updates at the active edge
monitor: sampled = bus;           // reads at the SAME edge → caught old OR new, seed-dependent

Common Mistakes

• Checking only the outputs. This is the observability trap: internal corruption that is masked or delayed at the boundary goes undetected. Core contracts need white-box assertions/monitors, not pin-watching.
• Assuming random reaches everything. The controllability trap: deep, ordered states need constructed (constrained / targeted) stimulus, not undirected poking — random alone almost never lands the precise sequence.
• Trusting a single passing run in a concurrent design. A pass under one interleaving is one sample of a non-deterministic outcome, not proof. Concurrency makes "it passed" conditional.
• Not capturing the seed. A bug you cannot reproduce is a bug you cannot fix. Treat seed + tool version as part of every failure report.

Senior Design Review Notes

Interview Insights

Exercises

1. Classify the failure. For each, say whether it is primarily a controllability or an observability problem, and the one technique you'd reach for: (a) a deep error-recovery state is never entered by your tests; (b) a counter overflows internally but the wrong value is overwritten before any output reads it; (c) an illegal FSM transition occurs but no test checks the FSM's internal encoding.
2. Construct the sequence. The write-buffer bug needs read-in-flight + overwrite, same cycle, same entry. Describe the stimulus constraints (not full code) that would construct this overlap deliberately rather than waiting for random luck — and the coverage point that proves you hit it.
3. Light up the interior. For that same bug, write (in words or SVA) the one assertion on the internal write path that would report the corruption at the cycle it occurs, instead of cycles later at a downstream read.
4. Pin the race. You have a 1/2000 non-reproducing scoreboard mismatch. List, in order, the first three things you do — and explain how each one attacks reproducibility, observability, or concurrency specifically.

Summary

• Verification is hard, not merely expensive, because a chip is a vast, dark, non-deterministic system you can only drive from its inputs and only watch at its outputs.
• The two foundational challenges are controllability ("can I drive the design into the buggy state?") and observability ("if the bug happens, can I detect it?"); reproducibility and concurrency make both worse by adding non-determinism.
• Each obstacle has a matched counter: stimulus for controllability, checking (assertions / scoreboards) for observability, seed capture for reproducibility, defined-region sampling + order-independent checks for concurrency, and coverage for completeness.
• The durable rules of thumb: check the interior, not just the boundary (observability), and a rare, non-reproducible failure is a race until proven otherwise (capture the seed first).
• UVM is how the industry packages these counters into reusable components — the obstacles are inherent to hardware; the methodology just makes the cures reusable. You now know what makes the problem hard.

Next — Verification Methodologies: with the obstacles named, we survey the approaches engineers built to overcome them — directed, constrained-random, coverage-driven, and assertion-based verification — and how they combine into the modern flow that UVM industrialises.