Josh BeckmanAs we think about it this way, we realize that the closer ⍴ gets to 1, the more likely it is that an incoming item of work will find a busy server, and so will be queued. So increasing ⍴ increases queue depth, which increases latency. By a lot. In fact, it increases latency by an alarming amount as ⍴ goes it 1. One way to think about this is in terms of the number of items of work in the system, including being serviced by the server, and in the queue. For tradition’s sake, we’ll call this N and its mean (expectation) E[N].

E[N] = ⍴/(1-p) Maybe we need to draw that to show how alarming it is. Exponential curve up

brooker.co.zaLatency Sneaks Up on You - Marc's Blog

### Filed Under

### Same Source

### Same Topic

- Math notation is a powerful demonstration of how **good (and...
- it's neat how *adding* affine measures is mathematically inv...
- The student who overcomes this problem might learn the follo...
- The `io_uring` interface works through two main data structu...
- as devices get extremely fast, interrupt-driven work is no l...
- In general, the chief compliance officer at any company has ...
- The discovery of scaling laws has typically preceded a boomt...
- Use a small model to generate a 'draft' output, then use a l...