Directed Testing

Verification Foundations · Module 1 · Page 1.6

The Engineering Problem

The previous lesson called directed testing the scalpel — precise, narrow, the tool you reach for when you know exactly what you want to check. Now we go deep on it, because it is both the oldest verification methodology and the one every engineer reaches for first — and understanding precisely why it works and exactly where it breaks is the foundation for everything that follows.

Directed testing means you hand-write a test for a specific scenario you already have in mind. You construct the situation deliberately, apply a known stimulus, and check a known response. Its great virtue is total control: the test does exactly what you wrote, every time, deterministically.

That same virtue is its ceiling. Your effort grows with the number of scenarios you can think of and type; the design's corner space grows combinatorially. So the engineering problem of this lesson is two-sided:

When is hand-construction exactly the right tool — and where, precisely, is the wall it cannot climb?

Motivation — why directed testing exists and never goes away

Even in a modern constrained-random shop, directed tests never disappear. Four properties keep them essential:

• Total control. You decide every input, every cycle. When a scenario is precise — "release reset, write 0xA5, read it back" — directed testing constructs it directly instead of hoping random stimulus stumbles into it.
• Determinism. A directed test does the same thing on every run. That makes it the ideal regression net: reproduce a bug once as a directed test, and it guards that corner forever.
• Readability. A directed test is its scenario, written out in order. A new engineer reads it top to bottom and knows exactly what is being checked — invaluable for spec-mandated sequences and for documentation-by-test.
• Spec mandates and errata. Some scenarios must be checked exactly: a protocol's required power-on handshake, a published erratum, a safety requirement. For these, "construct it precisely" beats "generate it randomly" every time.

The motivation, then, is not that directed testing is the whole answer — the previous lesson showed it is blind to the unimagined — but that for the slice of work that is known and specific, nothing is more direct, more readable, or more reliable.

Mental Model

Hold this picture:

A directed test is a script you wrote for a play whose plot you already know. It has four beats, always in the same order: Arrange (get the DUT into a known starting state), Act (apply the one exact stimulus this test exists to check), Expect (compute the spec-defined correct response), and Check (compare actual against expected, and fail loudly if they differ). Every good directed test is these four beats and nothing else — self-contained, deterministic, and readable as the scenario it represents.

The discipline is keeping the four beats clean: a known precondition (not "whatever the last test left behind"), a single deliberate stimulus (not a tangle), a spec-derived expected value (not one you eyeballed from the waveform), and a check that actually executes and actually fails on mismatch.

Visual Explanation — the anatomy of a directed test

Every directed test, in any language or framework, is the same four-beat data flow: drive the DUT to a known state, apply the stimulus, and compare its output against an independently-computed golden response.

The non-negotiable detail is the Expected node sitting beside the DUT, fed from the spec — not from the design. The most common way a directed test lies is checking the DUT against a value derived from the DUT's own behaviour: such a test passes whether the design is right or wrong. The golden value has to come from an independent source — the specification, a reference model, or a hand-calculation traceable to the spec.

RTL / Simulation Perspective — a directed test, beat by beat

Take a synchronous FIFO (depth 4) and verify one precise scenario: write a single known value and read it straight back. Watch the four beats map directly onto the code.

// DUT: synchronous FIFO, depth 4. Scenario: single-element write then read.
initial begin
  // ── ARRANGE ── drive the DUT into a known precondition (empty FIFO)
  rst_n = 0; wr_en = 0; rd_en = 0; din = '0;
  @(posedge clk); @(posedge clk);
  rst_n = 1;                       // reset released → FIFO empty, fully known state
  @(posedge clk);
 
  // ── ACT ── apply the one deliberate stimulus this test exists to check
  din = 8'hA5; wr_en = 1;          // write a single known value
  @(posedge clk);
  wr_en = 0;
  rd_en = 1;                       // read it straight back out
  @(posedge clk);
 
  // ── EXPECT + CHECK ── compare against the spec-defined golden response
  rd_en = 0;
  if (dout !== 8'hA5)              // golden 0xA5 comes from the scenario, not the DUT
    $fatal(1, "single-element R/W failed: got %h, expected A5", dout);
  $display("PASS: single-element write-then-read");
  $finish;
end

Read it as the four beats: Arrange holds reset to guarantee an empty FIFO; Act writes 0xA5 and then reads; Expect is the literal 8'hA5 (derived from the scenario, independent of the DUT); Check is the !== comparison with a loud $fatal on mismatch. Notice how concrete and readable it is — and notice that it verifies exactly one path. The full FIFO is correct only if you also check empty-read, full-write, wrap-around, simultaneous read+write, and reset-while-busy. Each is another hand-written test. That multiplication is the wall.

Verification Perspective — the scaling wall

Directed testing's cost is linear in the scenarios you write; the design's behaviour space is the product of its independent features. Those two curves diverge fast.

This is the structural limit, and it is arithmetic, not a matter of effort or skill. Directed tests add coverage one scenario at a time; real designs multiply their corner space with every independent feature, mode, and timing relationship. Long before you exhaust the corners, you exhaust the engineer-hours — and the corners you never reached are exactly where the escaping bugs live. (This is the precise problem the next lesson, random testing, exists to attack.)

Runtime / Execution Flow — directed tests as the regression net

The place where directed testing is uniquely irreplaceable is regression. Every bug you find — in review, in simulation, or in silicon — should become a directed test that reproduces it, so the corner can never silently come back.

This is the role directed testing keeps forever, even in a fully constrained-random environment: the deterministic, readable record of every corner that must never break again. Random stimulus is excellent at finding the bug the first time; a directed regression test is what guarantees it stays found. The two methodologies are complementary in exactly this way — discovery versus permanence.

Waveform Perspective — the deterministic, hand-crafted stimulus

A directed test's signature is on the waveform: the stimulus is deliberate and repeatable, with every transition placed by the engineer. Here is the FIFO test above, cycle by cycle.

Compare this to the random waveform from the previous lesson, where overlaps emerged unpredictably. Here every edge is intentional and identical run to run. That determinism is directed testing's superpower for regression — and, simultaneously, its limitation: it only ever exercises the one path you drew. Breadth has to come from somewhere else.

DebugLab — the directed test that passed for the wrong reason

intended precondition:  FIFO full (4/4)  →  5th write must be DROPPED
actual precondition:    FIFO 3/4 (retimed full flag)  →  5th write ACCEPTED normally
the check (dout == expected):  PASS  ← but it never exercised the overflow path
overflow corner actually tested:  NONE for ~12 months

Common Mistakes

• Checking the DUT against itself. Deriving the expected value from the design's own output (or a model that shares the design's bug) makes the test pass regardless of correctness. The golden value must come from the spec or an independent reference.
• Assuming the precondition instead of asserting it. If the Arrange beat silently stops creating the intended state (after a refactor, a retime, a port rename), the test keeps passing while testing nothing. Assert the precondition you depend on.
• Relying on leftover state between tests. A directed test that depends on what the previous test left behind is non-deterministic and order-sensitive — the opposite of directed testing's whole value. Each test must establish its own known precondition.
• Treating directed testing as the primary strategy on a large block. Hand-writing one test per corner cannot keep up with a combinatorial space. Directed testing is for the specific, known, high-value scenarios and for regression — breadth must come from constrained-random.
• A check that never executes. A $finish before the comparison, a branch that skips it, or a stimulus that ends early can leave the check unreached — and an unreached check always "passes." Make the check unconditional and loud.

Senior Design Review Notes

Interview Insights

Exercises

1. Label the beats. Take the FIFO code in this lesson and annotate exactly which lines are Arrange, Act, Expect, and Check. Then identify the single line you would add to make the test assert its precondition, and explain what failure that line would have caught.
2. Find the self-referential check. Sketch a directed test for an ALU's ADD that would pass on a broken adder, and then rewrite the Expect beat so it cannot. State in one sentence the rule your rewrite enforces.
3. Estimate the wall. A block has 4 independent features, each with 4 corners, plus 3 reset/error modes that each interact with all of them. Estimate the number of crosses a thorough directed suite would need, and argue whether hand-writing them is realistic — and what you would do instead.
4. Design a regression test. A silicon bug corrupts stored data when a FIFO write and read land on the same cycle while full. Write the four beats (in prose) of a deterministic directed test that reproduces it, including how you would prove the test actually reaches the full-and-simultaneous corner.

Summary

• A directed test is a hand-written, four-beat scenario — Arrange a known precondition, Act with one deliberate stimulus, Expect a spec-derived golden, Check loudly — that trades breadth for total control, determinism, and readability.
• It is the right tool for the specific, known, high-value slice: spec-mandated sequences, reproduced bugs, named corners, and especially regression, where a deterministic test locks a found bug closed forever.
• It does not scale as a primary strategy: directed effort grows by addition while the corner space grows by multiplication, so you exhaust engineer-hours long before the corners — and remain blind to the scenarios you never wrote.
• A directed test can silently lie — a self-referential golden, a rotted precondition, or an unreached check all stay green on a bug. Make each beat prove itself: derive the golden independently, assert the precondition, back it with a coverage bin, and check the effect you actually claim to verify.
• The durable rule of thumb: construct what is known and specific with directed tests; explore what is broad and unknown with something else — and never let a directed test pass without proving it reached its own corner.

Next — Random Testing: directed testing's wall is the unimagined corner. The next lesson takes up the methodology built to climb it — generating legal-but-unpredictable stimulus that reaches the states no one would ever hand-write.