How a token-bucket rate limiter works

Rate limiting is one of those "boring" things that quietly saves your production. Without it, a single bot or one runaway retry loop can flood an endpoint until it falls over. In reliakit I built the token-bucket kind. Here's how it works.

The problem

You want to allow something like: "bursty for a moment is fine, just not forever." Not a rigid "5 requests per minute" (that's a fixed window; it has burst problems at the window edge). Token bucket is smoother.

Token bucket, as simple as a bucket

Picture a bucket:

• It holds 5 tokens (the capacity).
• Each request takes 1 token. Token available → go. Empty → rejected.
• Tokens refill slowly (say, 1 token every 2 seconds).

So you can burst up to 5 at once (when the bucket is full), after which your rate is capped by the refill speed. Perfect for traffic that's naturally bursty but still needs an average ceiling.

The API

In reliakit, the rate limiter looks like this:

use reliakit::ratelimit::RateLimiter;

// capacity 5, refill 1 token every 2 (time units)
let mut limiter = RateLimiter::new(5, 1, 2);

if limiter.try_acquire_one(now) {
    // token available → go
} else {
    // bucket empty → rate limited
}

new(capacity, refill_amount, refill_interval), then try_acquire_one(now) returns a bool. That's the whole thing.

The good part: it's clock-agnostic

Notice that now is passed in by you. The limiter doesn't read the clock itself: no background thread, no SystemTime::now() buried inside. Why is that good?

Because it makes it easy to test and deterministic. You own time:

let mut rl = RateLimiter::new(2, 1, 1);

// "second" 0: 2 tokens, drain them
assert!(rl.try_acquire_one(0));
assert!(rl.try_acquire_one(0));
assert!(!rl.try_acquire_one(0)); // empty → rejected

// "second" 1: one token refilled
assert!(rl.try_acquire_one(1));

No sleep, no flaky tests. You fast-forward time however you like. Bonus: because it doesn't depend on the OS clock, it's no_std + zero-dependency, so it runs anywhere, including WASM.

Try it live in your browser

Because it runs anywhere, I compiled it to WASM and dropped it in the playground on the Projects page. Mash the button fast: the first 5 clicks pass, the rest get rejected, then tokens slowly refill. A real rate limiter, running in your browser, not a video, not a screenshot.

Wrapping up

Token bucket is simple but right: allow bursts, cap the average rate. And by making it clock-agnostic, you get the thing people usually skip: a rate limiter that's easy to test. The code is on GitHub.