Latency

Chapter 14.1 covered throughput (the rate); this chapter covers latency — the time for one transfer to complete, from issue to data. The decomposition:

latency = address phase (1 cycle) + data phase (1 cycle + W wait states) + path

So a zero-wait transfer with no extra hops takes 2 cycles (address + data); each wait state adds 1; and crossing extra hops adds their cycles — an interconnect/matrix may add routing/pipeline-register cycles (chapter 13.6), and an AHB-to-APB bridge adds its setup/access cycles. Crucially, latency ≠ throughput: latency is the time per transfer (how long one access takes), while throughput is the rate of many transfers (chapter 14.1). They're independent — pipelining and interconnect register stages can raise throughput (more transfers in flight) while increasing the latency of each one. Latency matters for single dependent accesses (a CPU load whose result is needed now), while throughput matters for streaming bulk data (a DMA copy). This chapter decomposes latency, distinguishes it from throughput, and shows when each matters.

1. What Is It?

Latency is the time from when a transfer is issued to when its data is available — the duration of a single transfer. Its components:

• Address phase — 1 cycle (the master drives the address/control).
• Data phase — 1 cycle + W wait states (the slave provides/accepts data; W = wait states it inserts).
• Path additions — interconnect/matrix routing or pipeline-register cycles (chapter 13.6); AHB-to-APB bridge setup/access cycles.
• Base case: zero-wait, no extra hops = 2 cycles (address + data).

So latency is the end-to-end time for one transfer — the sum of the phases (address + data, the AHB pipeline's two cycles) plus the delays (wait states from the slave, routing/pipeline cycles from the interconnect, setup/access from a bridge). The minimum is 2 cycles (the two phases, zero-wait, direct). Everything else adds: a slow slave (wait states), a deep interconnect (routing/pipeline), a bridge (APB cycles). So latency analysis is summing the cycles along the path a transfer takes — from the master, through the interconnect, to the slave, and back. So latency is the per-transfer time = the phase cycles plus all the delays on the path.

2. Why Does It Exist?

Latency exists as a distinct metric from throughput because latency-bound workloads (a single access whose result is immediately needed) are limited by how long one transfer takes — not the rate — and conflating the two leads to optimizing the wrong thing.

The latency-bound workloads drive the metric: many critical operations depend on a single access completing quickly. A CPU executing a load instruction stalls until the data returns — the latency of that load directly delays the CPU. A dependent chain (load A, then use A to compute B) is serialized by each access's latency. A pointer chase (follow a linked list) is pure latency (each step waits for the previous). For these, the rate (throughput) is irrelevant — what matters is how long each access takes. So latency is the metric for latency-bound (dependent, serialized) work. So it exists because per-transfer time matters for these workloads.

The latency ≠ throughput distinction is the key reason it's separate: throughput (chapter 14.1) is the rate — bytes/second over many transfers. Latency is the time for one. These are independent: a bus can have high throughput (many transfers/second) but high latency (each takes many cycles) — e.g. a deeply pipelined interconnect keeps many transfers in flight (high throughput) but each transfer traverses several pipeline stages (high latency, chapter 13.6). Conversely, a direct bus has low latency (2 cycles) but maybe lower throughput (no pipelining for concurrency). So latency and throughput trade off — pipelining/register stages buy throughput at a latency cost. So they're distinct, independent metrics, and optimizing one doesn't (necessarily) optimize the other. So latency exists as a separate metric because it's independent of throughput. So you must measure and optimize both, for their respective workloads.

The reason latency decomposes (into address + data + waits + path) is to locate the delay: the total latency is the sum of the cycles on the transfer's path, so decomposing it shows where the time goes — the base 2 cycles (unavoidable, the two phases), the slave's wait states (reducible with a faster slave), the interconnect's routing/pipeline cycles (a matrix's cost), the bridge's setup/access cycles (a bridge's cost). So decomposing latency locates the dominant delay, telling you what to fix (a slow slave? a deep interconnect? a bridge?). So latency exists, decomposed, to attribute the per-transfer time to its sources. So latency exists as a distinct metric because: latency-bound workloads (dependent/serialized accesses) are limited by per-transfer time, not rate (the why); latency is independent of throughput (they trade off — pipelining buys throughput at latency cost) so it must be measured separately (the distinction); and decomposing it (address + data + waits + path) locates the delay so you can reduce it (the analysis). So latency is the per-transfer-time metric, complementary to throughput, decomposed to attribute and reduce the delay — essential for the latency-sensitive parts of a system.

3. Mental Model

Model latency as the time for a single round-trip errand versus the rate of a delivery service — latency is how long one errand takes door-to-door (drive there, wait in line, drive back), while throughput is how many errands the service completes per hour; a service with many drivers (pipelining) completes lots of errands per hour (high throughput) but each individual errand might still take a long time door-to-door (high latency) — and if you're waiting on one specific errand (a dependent task), what matters is its round-trip time, not the service's overall rate.

Consider a single errand (a transfer): how long it takes door-to-door is its latency — drive to the store (the address phase), wait in line at the counter (wait states), and the result is ready (the data phase). The minimum round-trip is just there-and-back (2 cycles); a long line (slow slave) adds time; a detour through a sorting hub (an interconnect/matrix) adds time; going through a separate department (a bridge) adds time. Now contrast with a delivery service's rate — how many errands it completes per hour (throughput). A service with many drivers working in parallel (pipelining) completes lots of errands per hour (high throughput) — but each individual errand might still take a long door-to-door time (high latency), because adding drivers raises the rate without shortening any one trip (it might even lengthen each via a central depot/pipeline). So if you're personally waiting on one specific errand (a dependent task — you can't proceed until it's done), what matters is that errand's round-trip time (latency), not the service's overall throughput. Conversely, if you just need lots of stuff delivered eventually (streaming/bulk), the rate (throughput) matters.

This captures latency: one errand's door-to-door time = transfer latency; drive there + wait in line + result ready = address phase + wait states + data phase; a detour through a hub = interconnect/matrix cycles; a separate department = bridge cycles; the service's errands-per-hour = throughput; many drivers raising the rate but not shortening one trip = pipelining raising throughput at a latency cost; waiting on one specific errand = a dependent access (latency-bound). One errand's round-trip is latency; the service's rate is throughput — independent, optimize the one you need.

Watch a transfer's latency from issue to data:

The model's lesson: one errand's round-trip is latency; the service's rate is throughput — independent. In the waveform, the issue-to-data span (3 cycles here, with one wait) is the latency — distinct from how many transfers/second the bus sustains.

4. Real Hardware Perspective

In hardware, latency is the sum of cycles along the transfer's physical path — the two phases (fixed), the slave's wait states, the interconnect's routing/pipeline depth, and any bridge's protocol cycles — and reducing it means attacking the dominant term.

The fixed two-phase base: the AHB pipeline imposes a minimum of 2 cycles per transfer (address phase + data phase) — this is inherent to the protocol and can't be reduced (it's the cost of the address/data separation). So in hardware, every transfer has this 2-cycle floor. So latency starts at 2.

The slave wait states are often the dominant term: a slow slave (flash, external memory) inserts wait states (chapter 12.4), each adding a cycle of latency. A flash with, say, 3 wait states makes a transfer 2 + 3 = 5 cycles. So a slow slave can dominate latency. The hardware fix: faster slaves (fast SRAM), or caching/prefetch (so the common-case access hits the cache, avoiding the slow slave's wait states). So in hardware, slave speed is a key latency lever. So reduce wait states for latency-critical accesses.

The interconnect and bridge additions are path costs: an interconnect/matrix adds latency if it's pipelined (each register stage is +1 cycle, chapter 13.6) — a deep matrix trades latency for timing/throughput. An AHB-to-APB bridge adds the APB protocol's cycles (setup phase + access phase, plus any APB wait states) — so a peripheral access through the bridge is slower than a direct AHB access (the bridge penalty, chapter 14.7). So in hardware, the path's hops (interconnect depth, bridge) add latency. The hardware reduction: a shallower (combinational or less-pipelined) interconnect for latency-critical paths, and accepting the bridge cost for peripherals (which are usually latency-tolerant). So in hardware, latency = the fixed 2-cycle base + the slave's wait states + the interconnect's pipeline depth + the bridge's protocol cycles — and reducing it means attacking the dominant term (usually a slow slave's wait states, or a deep/bridged path). The decomposition (Figure 1) shows which term dominates, guiding the fix. So the hardware latency is the path-cycle sum, reduced by attacking the biggest contributor.

5. System Architecture Perspective

At the system level, latency is what determines the speed of dependent, serialized operations (the critical path of computation), and the latency-vs-throughput distinction is essential to optimizing a system for its actual sensitivity.

The critical-path latency: a system's compute performance often depends on a chain of dependent accesses — each must complete before the next can start (a load feeding a computation feeding another load). The total time of such a chain is the sum of each access's latency. So latency directly sets the speed of serialized, dependent work — the critical path. So for compute-bound, dependency-heavy code (pointer chasing, dependent loads), latency is the limiter, not throughput. So at the system level, latency determines critical-path speed. Reducing it (faster slaves, caches, shallow interconnect) speeds up dependent computation.

The latency vs throughput sensitivity: the crucial system insight is that different workloads are sensitive to different metrics. Latency-sensitive: a CPU's dependent loads (each stalls the CPU), interrupt response (time to first access), a pointer chase. Throughput-sensitive: a DMA streaming a buffer (rate of bulk movement), a frame buffer (sustained bandwidth). So the architect must know which the workload (or each part of it) is sensitive to — and optimize that. Optimizing throughput for a latency-bound task (e.g. adding a deep pipeline) can hurt it (raising latency). Optimizing latency for a throughput-bound task (e.g. a shallow non-pipelined path) can limit its bandwidth. So at the system level, matching the optimization to the workload's sensitivity is essential — and it's a common mistake to optimize the wrong one. So know your workload's sensitivity.

The co-existence and trade: real systems have both kinds of traffic — latency-sensitive (CPU) and throughput-sensitive (DMA) — often concurrently. The architecture must serve both: e.g. give the latency-sensitive CPU a low-latency path (fast cache/SRAM, shallow interconnect, high arbitration priority for quick grant) and the throughput-sensitive DMA a high-throughput path (bursts, wide/banked memory). A bus matrix (Module 13) helps by giving each its own path (the CPU's low-latency access to its memory isn't slowed by the DMA's bulk stream to another). So at the system level, latency determines dependent/critical-path speed, the latency-vs-throughput distinction dictates matching the optimization to each workload's sensitivity, and real systems must co-serve both (low-latency paths for dependent accesses, high-throughput paths for streaming) — often via a matrix giving each its own path. So latency and throughput together (chapters 14.1–14.2) are the two axes of AHB performance, and system-performance design is about optimizing each for the traffic that's sensitive to it. So latency is the per-transfer-time axis, complementary to throughput, and getting the sensitivity-matching right is the heart of performance architecture.

6. Engineering Tradeoffs

Latency embodies the per-transfer-time, path-sum, traded-against-throughput model.

• Latency vs throughput. Optimizing latency (fewer cycles per transfer, shallow path) speeds dependent accesses; optimizing throughput (pipelining, bursts) speeds streaming — and they trade off (pipelining raises throughput, increases latency). Optimize the metric the workload is sensitive to.
• Fast slave (low latency) vs cost. A faster slave (SRAM, cache) reduces wait states (lower latency) at the cost of the faster memory. Worth it for latency-critical accesses (CPU data).
• Shallow vs deep interconnect. A shallow (combinational/less-pipelined) interconnect has lower latency but may limit clock/throughput; a deep (pipelined) matrix has higher latency but higher clock/throughput. Choose per the path's sensitivity.
• Direct AHB vs through-bridge. A direct AHB access is low-latency; a peripheral access through the AHB-to-APB bridge adds the bridge/APB cycles (higher latency) — acceptable since peripherals are usually latency-tolerant. Keep latency-critical slaves on the fast direct path.

The throughline: latency is the time for one transfer to complete = address phase (1 cycle) + data phase (1 cycle + W wait states) + interconnect/matrix routing-pipeline cycles + AHB-to-APB bridge setup/access cycles. The base is 2 cycles (the two phases, zero-wait, direct); each wait state and each hop adds. Latency is per-transfer time, independent of throughput (the rate) — they trade off under pipelining. Latency matters for dependent/serialized accesses (critical path), throughput for streaming. Match the optimization to the workload's sensitivity; real systems co-serve both (low-latency paths for dependent accesses, high-throughput for streaming, often via a matrix).

7. Industry Example

Analyze latency for a dependent CPU access and contrast with a DMA stream.

A CPU does dependent loads; a DMA streams data — both on the same system.

• CPU dependent load (latency-bound). The CPU executes a load that feeds the next instruction — it stalls until the data returns. The latency: if the data is in fast SRAM (zero wait), 2 cycles (address + data) — fast. If it's in slow flash (3 wait states), 2 + 3 = 5 cycles — the CPU stalls 5 cycles. If the path goes through a pipelined matrix (+2 cycles) and the CPU's load is to a bridged peripheral (+ bridge cycles), the latency grows further. So the CPU's dependent-load speed is set by this latency.
• Reducing CPU latency. To speed the CPU's dependent loads: keep its data in fast SRAM or a cache (zero/low wait states), on a shallow (low-latency) interconnect path, with high arbitration priority (quick grant). This minimizes the per-load latency, speeding the dependent chain.
• DMA stream (throughput-bound). The DMA streams a buffer — it issues many transfers and cares about the rate, not any single transfer's latency. Even if each transfer has high latency (deep pipeline), the DMA's throughput is high if transfers are pipelined (many in flight) and bursted. So the DMA's speed is set by throughput, not latency.
• Optimizing the DMA. To speed the DMA: bursts (amortise address overhead, chapter 14.1), pipelining (more in flight), wide/banked memory (more bandwidth) — all throughput optimizations, unconcerned with per-transfer latency.
• The trap. If the architect added a deep pipeline to boost the DMA's throughput but routed the CPU's dependent loads through the same deep pipeline, the CPU's loads would get higher latency — slowing the latency-bound CPU. The fix: separate paths — a matrix giving the CPU a low-latency path to its SRAM and the DMA a high-throughput (pipelined) path to the streaming memory.
• Co-serving both. With a matrix, the CPU gets low latency (its own shallow path to fast SRAM) and the DMA gets high throughput (its own pipelined, bursted path) — each optimized for its sensitivity, concurrently.

The example shows latency and throughput as distinct, workload-specific concerns: the CPU's dependent loads are latency-bound (minimize per-transfer cycles), the DMA stream is throughput-bound (maximize rate), and optimizing one can hurt the other — resolved by separate, sensitivity-matched paths (a matrix). This is how latency analysis guides real performance design.

8. Common Mistakes

9. Interview Insight

Latency is a quantitative interview topic — the decomposition, the independence from throughput, and the sensitivity-matching are the signals.

The answer that lands gives the decomposition and the independence: "AHB transfer latency is the time for one transfer to complete, from issue to data. It decomposes as the address phase, one cycle, plus the data phase, one cycle plus any wait states the slave inserts. So a zero-wait transfer with no extra hops is 2 cycles, and each wait state adds one. On top of that, the path adds cycles: crossing a pipelined interconnect or bus matrix adds its routing or register-stage cycles, and going through an AHB-to-APB bridge adds the APB setup and access cycles. So you sum the cycles along the path. The crucial point is that latency is independent of throughput — latency is the time for one transfer, throughput is the rate of many. They can trade off: pipelining the interconnect raises throughput by keeping more transfers in flight, but it increases the latency of each one. So you have to optimize the metric your workload is sensitive to. Latency matters for dependent, serialized accesses — a CPU load whose result is immediately needed, a pointer chase — where each access stalls the next. Throughput matters for streaming bulk data — a DMA copy. A common mistake is to optimize throughput, like adding a deep pipeline, and inadvertently worsen the latency of a latency-bound task. Real systems serve both by giving each its own path, often via a matrix — a low-latency path for the CPU, a high-throughput path for the DMA." The decomposition, the independence-from-throughput, and the sensitivity-matching are the senior signals.

10. Practice Challenge

Compute and reason from latency.

1. Decompose. Write the latency decomposition and compute it for a transfer with 2 wait states through a 1-stage-pipelined matrix.
2. vs throughput. Explain why latency and throughput are independent and how they trade off.
3. Read the waveform. From Figure 2, identify the issue-to-data span and the wait-state contribution.
4. Sensitivity. Give a latency-bound and a throughput-bound workload, and the right optimization for each.
5. Locate the delay. Given a decomposed latency, explain which term to attack and how.

11. Key Takeaways

• Latency = address phase (1 cycle) + data phase (1 cycle + W wait states) + path (interconnect/matrix routing-pipeline cycles + AHB-to-APB bridge setup/access cycles) — the time for one transfer.
• The base is 2 cycles (the two phases, zero-wait, direct) — irreducible; each wait state and each hop adds.
• Latency is independent of throughput — latency is per-transfer time, throughput is the rate; they trade off (pipelining raises throughput, increases latency).
• Latency matters for dependent/serialized accesses (CPU loads, pointer chase — the critical path); throughput matters for streaming (DMA copy). Match the optimization to the workload's sensitivity.
• Decompose latency to locate the dominant delay — base (irreducible), wait states (fast slaves/cache), interconnect depth (shallow path), bridge (direct path) — and attack the biggest term.
• Real systems co-serve both — give latency-sensitive traffic a low-latency path and throughput-sensitive traffic a high-throughput path (often via a matrix), so one optimization doesn't hurt the other.

12. What Comes Next

You now understand latency and its relationship to throughput — the two axes of AHB performance. The next chapters quantify the specific effects:

• Pipelining Benefit (next) — quantifying the throughput gain from the address/data phase overlap.
• Wait-State Penalty, Burst Efficiency, Arbitration Overhead, Bridge Penalty, Bottleneck Analysis, Performance Debug — quantifying each effect and how to debug performance.

To revisit the pipeline that sets the 2-cycle base, see Address & Data Phase; for throughput (the complementary metric), see Throughput.