The limit that counted the wrong thing
The front door negotiates a maximum of a hundred concurrent streams per connection, which everybody read as a hundred concurrent requests.
A client opens a stream, the server dispatches the request to a handler, and the client cancels the stream immediately. The stream closes, the counter drops, a new stream takes its place — and the handler keeps running. The counter was accurate throughout. It was counting protocol state, and the resource is consumed by work.
Boundaries, outermost first: One connection: Streams: 100 (never exceeded), Handlers (still running; FAILED: unbounded), an empty slot for the what bounds the work, an empty slot for the the second route in Outside every boundary: One client (open, then cancel), Backend pool (exhausted) Connections: One client calls Streams: 100 — open, then cancel (step 1) Streams: 100 calls Handlers — already dispatched (step 2) Handlers calls Backend pool (step 3) what bounds the work controls Handlers (step 4) the second route in controls Streams: 100 — in and out