wtf( )unctionsystem design, drawn
← all problemsCachingMedium

The refresh everyone waited for

The homepage is cached for sixty seconds. At second sixty-one every request in flight finds a stale entry, and every one of them waits behind the same origin fetch. Latency spikes once a minute like a metronome.

The entry is one second past its freshness lifetime. It is not wrong. It is just no longer allowed to be served.

Let the edge answer from the copy it already has while it goes and gets a new one.
Components — tap one, then tap a slot on the diagram
!Every time the homepage's cache entry expires, a hundred requests queue behind one origin fetch.

Outside every boundary: Origin (one slow render; FAILED: 100 waiting), Edge cache (entry 1s stale), Requests (100 in one second), an empty slot for the what the client gets now, an empty slot for the if the origin is down Connections: Requests calls Edge cache — all at once (step 1) Edge cache calls what the client gets now — already have it (step 2) what the client gets now calls Requests — answered now (step 3) Edge cache publishes to Origin — refresh behind it (step 4) Origin controls if the origin is down — if it 5xxs

Originone slow render100 waiting
Edge cacheentry 1s stale
Requests100 in one second