The key rotated, and every service kept the old one
The identity provider rotates its signing key on schedule, which is correct. Services fetch the published key set once at start-up and hold it, which felt responsible — the set does not change often.
It changed. Every service was holding the previous set, none of them went back for the new one, and every token signed with the new key failed verification until somebody restarted everything.
Outside every boundary: Provider (rotated its key), Service (cached at start-up; FAILED: rejects everything), Published key set (one document), A token (names its key), an empty slot for the reacting to a key it has never seen, an empty slot for the how long the retired key stays up Connections: Provider calls Published key set — publishes the set (step 1) how long the retired key stays up controls Published key set — how long it stays A token calls Service — the new key (step 2) Service controls reacting to a key it has never seen — no such key (step 3) reacting to a key it has never seen calls Published key set — one fetch, not many (step 4)