Skip to content

Advanced

The rate-limiting internals. You do not need these to use the library – AlmaClient builds and drives them for you – but they are documented because the behaviour they implement is worth understanding when tuning rate_limit, or when reading the almapy.throttle log output.

See Rate limiting for the narrative version.

Token bucket

TokenBucket 

TokenBucket(rate: float)

Paces requests to a fixed rate, holding one second of burst capacity.

AlmaClient builds one from its rate_limit argument, so there is rarely a reason to construct this yourself. What it means in practice: an idle client can fire off up to rate requests at once, and is then paced at rate per second. Callers wait their turn in acquire rather than being rejected, which is why a large asyncio.gather is safe.

rate property writable 

rate: float

Current refill rate in tokens (requests) per second.

Setting it clamps the outstanding token count to the new rate, so lowering the rate takes effect immediately rather than after an accumulated burst is spent.

Raises:

Type Description
ValueError

If set to zero or a negative value.

acquire async 

acquire() -> None

Consume one token, waiting for the bucket to refill if it is empty.

Never times out – it waits as long as necessary. Use AdaptiveController.acquire for a bounded wait.

Adaptive controller

AdaptiveController 

AdaptiveController(
    bucket: TokenBucket,
    max_rate: float,
    *,
    backoff_factor: float = 0.5,
    recovery_increment: float = 1.0,
    recovery_window: float = 10.0,
    cooldown: float = 5.0,
    max_wait: float | None = None,
    min_rate: float = 1.0,
)

Adjusts a bucket's rate to match how Alma is currently behaving.

Cuts the rate sharply the moment requests start failing and restores it gradually as they succeed, so a struggling Alma is not hammered and a healthy one is not left throttled. AlmaClient builds and drives one of these for you; the tuning arguments below are exposed on its constructor as backoff_factor, recovery_increment, recovery_window, cooldown and max_wait.

Parameters:

Name Type Description Default
bucket TokenBucket

The bucket whose rate is adjusted.

 required
max_rate float

Ceiling for recovery, normally the configured rate_limit.

 required
backoff_factor float

What the rate is multiplied by on failure.

 0.5
recovery_increment float

How much is added back per successful recovery step.

 1.0
recovery_window float

Minimum interval between recovery steps, in seconds.

 10.0
cooldown float

How long after a cut further failures are ignored, in seconds.

 5.0
max_wait float | None

Bound on how long a caller will wait for a token before ThrottleTimeoutError is raised. None waits indefinitely.

 None
min_rate float

Floor the rate is never cut below.

 1.0

See Rate limiting for how this behaves in practice, and how it interacts with retries.

current_rate property 

current_rate: float

The rate the underlying bucket is currently running at, in requests/second.

Read-only, and lower than the configured maximum whenever backpressure has cut it. Useful for monitoring – logging it, or exporting it as a metric.

acquire async 

acquire() -> None

Wait for cooldown to elapse, then consume one token from the bucket.

Raises:

Type Description
ThrottleTimeoutError

If max_wait was configured and elapsed before a token became available. Subclasses TimeoutError, so except TimeoutError catches it too.

record_failure 

record_failure() -> None

Report a transient failure, cutting the rate multiplicatively.

Multiplies the rate by backoff_factor (never below min_rate) and starts a cooldown, during which further failures are ignored – a burst of concurrent failures from one incident cuts the rate once, not once per request.

Called for any failure _should_retry recognises, deliberately including those on POST and PATCH requests that will not themselves be replayed: a 5xx says something about Alma's health whichever verb provoked it.

record_success 

record_success() -> None

Report a successful request, recovering the rate additively.

Adds recovery_increment to the rate, up to max_rate. Recovery is time-gated to at most once per recovery_window and suppressed entirely during cooldown, so the rate climbs back gradually rather than jumping straight to the maximum after one success.