Backpressure in Streams API

TL;DR / Backpressure is the flow control mechanism in the Streams API that prevents fast producers from overwhelming slow consumers by signaling when internal queues are full.

How It Works

 ┌────────────┐        ┌────────────┐        ┌────────────┐
 │  Producer  │        │            │        │  Consumer  │
 │   (fast)   │───┐    │  Internal  │   ┌───→│   (slow)   │
 │            │   └───→│   Queue    │───┘    │            │
 └────────────┘        │  HWM = 3   │        └────────────┘
                       │            │
                       └────────────┘
   pause                                       resume
   desiredSize <= 0                            desiredSize > 0

Edit diagram

The Streams API in browsers provides ReadableStream, WritableStream, and TransformStream as primitives for processing sequential data. Backpressure is the mechanism that coordinates data flow between these primitives when they operate at different speeds.

Every stream maintains an internal queue with a high water mark (HWM) -- a threshold indicating the target maximum number of chunks (or bytes, depending on the queuing strategy) to buffer. The critical property exposed by this queue is desiredSize, calculated as HWM - queueSize. When desiredSize drops to zero or below, the stream signals that its producer should stop enqueuing data.

In a ReadableStream, backpressure flows upstream through the pull() mechanism. When you define an underlying source, the pull() callback is invoked only when the internal queue has room -- specifically, when desiredSize > 0. If the consumer (reader) is slow to call read(), chunks accumulate in the queue, desiredSize drops, and pull() stops being called. The producer is effectively paused without any explicit pause/resume API. When the consumer resumes reading and drains chunks, desiredSize climbs back above zero, and pull() is invoked again.

For WritableStream, backpressure manifests differently. The write() method on the writer returns a promise. When the internal queue is below the HWM, write() resolves immediately. When the queue is full, the promise returned by write() does not resolve until the underlying sink has consumed enough chunks to make room. The writer.ready promise provides an explicit signal -- it resolves when the stream is ready to accept more data, giving producers a clean await point.

Pipe chains are where backpressure becomes truly powerful. When you call readable.pipeTo(writable) or use pipeThrough(transform), the pipe implementation automatically manages backpressure across the entire chain. If the final writable sink is slow (disk I/O, network upload), backpressure propagates backward through every transform stream in the chain, all the way to the original readable source. No intermediate buffer overflows. No dropped data.

The queuing strategy determines how chunk sizes are counted. The default CountQueuingStrategy counts each chunk as 1 unit regardless of size. ByteLengthQueuingStrategy uses the byte length of each chunk, which is appropriate for binary data where chunks vary in size. You can also provide a custom size() function for domain-specific sizing. The HWM interacts with this: a ByteLengthQueuingStrategy with HWM of 65536 buffers up to 64KB regardless of chunk count.

TransformStream creates a matched readable/writable pair with backpressure automatically bridged between them. The transform's writable side accepts input chunks, the transform() function processes them (potentially enqueuing zero, one, or many output chunks), and the readable side exposes the results. If the downstream consumer is slow, the transform's readable queue fills, which prevents transform() from enqueuing more output, which blocks the writable side from accepting more input -- full chain backpressure.

A practical example: streaming a large file upload through an encryption transform. The file ReadableStream produces chunks at disk speed. The encryption TransformStream processes them. The upload WritableStream sends them over the network. If the network is slow, backpressure ensures the encryption transform pauses, which pauses file reading. Memory usage stays bounded at roughly 3 * HWM across the chain rather than buffering the entire file.

The ReadableStream constructor also supports a cancel() callback, invoked when a consumer aborts reading. This is distinct from backpressure -- cancellation terminates the stream entirely, while backpressure temporarily pauses it.

Gotchas

  • Default HWM is 1 for count strategy, 0 for byte strategy on readable sources -- these defaults are conservative; tune them based on your chunk sizes and latency tolerance
  • enqueue() does not throw when the queue is over HWM -- it returns undefined regardless; backpressure is advisory via desiredSize, not enforced by the API; a misbehaving source can still overwhelm the queue
  • tee() breaks independent backpressure -- teeing a readable stream creates two branches, but if one branch reads slowly, the tee buffers unboundedly for the slow branch since the fast branch keeps pulling
  • Transform streams cannot buffer internally without risk -- if transform() enqueues multiple output chunks per input chunk, it can overflow the readable side's queue; check controller.desiredSize before enqueuing
  • Piping to a closed/errored stream does not always throw synchronously -- errors propagate asynchronously through pipe chains; always attach error handlers to the pipe promise