Timeout & Retry Budget Planner

Every hop in a request path has its own timeout and its own retry policy, and almost nobody adds them up. Enter the chain to see the real worst-case latency, how many backend calls one client request can become, and which retries are dead code because a caller upstream gives up first.

Load a scenario

1 · Client budget

Client-facing budget (ms)

The longest a caller will wait before it gives up or a user abandons. Everything below has to fit inside this.

Per-hop processing margin (ms)

Time each hop spends on its own work — parsing, auth, serialisation — on top of waiting for the hop below.

Steady-state requests / sec

Used to show what the amplification factor means in load on the innermost dependency.

2 · The chain

Ordered outermost to innermost. Turn off the hops you do not have. A hop's timeout should always exceed the full worst case of everything below it — otherwise it abandons work that is still running.

	Hop	Timeout (ms)	Retries	Backoff (ms)

3 · Results

Worst case, hop by hop

Each block is one attempt running to its full timeout; the gaps between them are backoff. A row wider than the row above it is a hop still working on a request its caller has already abandoned.

What to change

4 · Probes & shutdown

The same budget decides how long Kubernetes should wait for a slow-starting process, how long a stall it should tolerate before restarting it, and how long to let in-flight requests finish during a rollout.

Application start time, p99 (s)

Container start to first successful health check. Measure it, do not guess — JVM apps routinely exceed a minute.

Longest tolerable stall (s)

Worst GC pause or lock contention that should not trigger a restart.

preStop drain sleep (s)

Time for the endpoint removal to propagate to every proxy before the process starts shutting down.

Configuration

Worst case is what this tool computes — the path where every attempt runs to its full timeout. Typical latency is far lower, which is exactly why these configurations survive testing and fail in an incident. Treat the amplification factor as a capacity input: your slowest dependency will receive that multiple of normal traffic at precisely the moment it is already struggling. Retries need a budget, not just a count — prefer a client-side retry budget (a cap of a few percent of requests) plus a circuit breaker over a fixed retry count at every layer.

Worst case, hop by hopApply recommended ladder

What to change

ConfigurationCopy

Worst case, hop by hop

Configuration

Timeout Retry Budget