Toolstoken bucket
Eight Now, Then One a Second
A token bucket is a common way to limit how often an API can be called. A clock drops tokens into a tube that holds eight. Each request takes one token, and a request that finds the tube empty is refused.
~/tools/token-bucket
Refill:
- Tokens in the tube
- 5/8
- Served
- 0
- Refused (429)
- 0
The clock adds one token every 1/r seconds, and the tube holds at most b = 8. A token that arrives when the tube is full slides past and spills into the tray on the right, so idle time never saves up more than eight. Each request takes the bottom token and passes at once. With the tube empty the request is refused, which an HTTP API answers with 429 Too Many Requests. Over any T seconds at most about b + r·T requests pass: eight at once, then one a second. AWS API Gateway, for example, throttles with this algorithm.