Five languages, one retry policy, and no library
Dozens of services in five languages, written by teams who will not agree on a framework. Every call between two of them has to be encrypted and mutually authenticated, every timeout and retry has to be uniform and changeable without redeploying anything, and every hop has to appear in traces.
A library per language was costed and rejected: five implementations of one policy, drifting from each other, each one needing a release to change a number.
- R1Timeouts, retries, ejection of a bad instance, encryption and tracing must all apply to service code that is not modified at all — no library, no imported client, no change to the application. Something else must terminate and originate each instance's traffic on its behalf.
- R2Each call must prove which workload is calling, using a credential belonging to that workload rather than a shared secret in configuration — and it must be short-lived and rotate without a deploy.
- R3Whatever distributes the rules must not be in the request path. If it fails, traffic that is already configured and flowing must keep flowing, and the blast radius must be that new rules stop arriving rather than that calls stop working.
Outside every boundary: Service B (another language), Service A (unmodified), Traces and metrics (from the proxies), Proxy beside B (same policy), an empty slot for the every call leaves through this, tier 1, an empty slot for the proves which workload is calling, tier 1, an empty slot for the hands out timeouts and retries, tier 2 Connections: Service A calls every call leaves through this every call leaves through this calls Proxy beside B — mutual TLS Proxy beside B calls Service B proves which workload is calling controls every call leaves through this — issues, rotates hands out timeouts and retries controls every call leaves through this — config push every call leaves through this controls Traces and metrics