Node streams stall at 64KB: nobody read the docs on highWaterMark and pipe backpressure

verbatim errorError: write after end at WriteStream.destroy (node:internal/streams/writable.js:384:8) # or the silent variant: the pipe just never finishes, heap grows to 4GB

Problem

A CSV export service for large files. Small files fine; anything over ~50MB either died with

Error: write after end
    at WriteStream.destroy (node:internal/streams/writable.js:384:8)

or — the nastier variant — hung forever while RSS climbed toward the container's 4GB limit. No error at all, just a slow-motion leak.

Root cause

Streams have a buffer budget, highWaterMark (default 64KB readable / 16KB writable). When a producer pushes faster than a consumer drains, write() returns false — that is backpressure — and naive code ignores it:

readable.on('data', (chunk) => {
  transform(chunk);            // sync work
  dest.write(chunk);           // returns false! ignored
});

Ignoring the false means the writable's internal buffer grows without bound — the 4GB climb. The "write after end" variant came from manual .end() calls racing with piped data: mixing pipe() with hand-rolled write/end on the same destination is a reliable way to corrupt the state machine.

fix preview — first 3 of 10 lines (ts), truncated:

import { pipeline } from 'node:stream/promises';
import { createReadStream, createWriteStream } from 'node:fs';
import { createGzip } from 'node:zlib';
… 7 more lines in the fix

🔒 the fix — including 3 code blocks — is members-only. $1/mo unlocks everything.