The message that blocked the queue
One malformed message reaches the front of the queue. The worker crashes on it, the message becomes visible again, the next worker crashes on the same message. Nothing behind it is ever processed.
The queue isn't full and the workers aren't down. One message the code can't handle is holding everything hostage — a poison pill.
Give the message somewhere to go after it has failed enough times.
Components — tap one, then tap a slot on the diagram
!One message the consumer cannot parse is at the head of the queue. Nothing behind it moves.
Outside every boundary: Worker (crashes on it; FAILED: crash-looping), Main queue (blocked; FAILED: blocked), Producer, Alerting (someone should know), an empty slot for the after N failures Connections: Producer publishes to Main queue (step 1) Main queue publishes to Worker (step 2) Worker publishes to after N failures (step 3) after N failures controls Alerting — alert on depth > 0
Workercrashes on itcrash-looping
Main queueblockedblocked
Producer
Alertingsomeone should know