API

Rate Limits

Rate Limit Granularities

class freiner.limits.RateLimitItem(amount: int, multiples: int = 1, namespace: str = 'LIMITER')[source]

Bases: object

defines a Rate limited resource which contains the characteristic namespace, amount and granularity multiples of the rate limiting window.

Parameters

  • amount (int) – The amount of hits that can be sustained within the given period (e.g. X in ‘x per y seconds’).

  • multiples (int) – Multiple of the ‘per’ granularity (e.g. Y in ‘x per y seconds’).

  • namespace (str) – An arbitrary namespace for this rate limit.

Return type

None

classmethod check_granularity_string(granularity_string: str) → bool[source]

Check if this class matches a granularity string of type ‘N per hour’ etc.

Parameters

granularity_string (str) – The granularity to match against.

Returns

Does the supplied granularity identifier match this class’ granularity?

Return type

bool

get_expiry() → int[source]
Returns

The length of the expiry window in seconds.

Return type

int

key_for(*identifiers: Any) → str[source]
Parameters

identifiers (Any) – A list of arbitrary strings to append to the key.

Returns

A string key identifying this resource with each identifier appended with a ‘/’ delimiter.

Return type

str

class freiner.limits.RateLimitItemPerYear(amount: int, multiples: int = 1, namespace: str = 'LIMITER')[source]

Bases: freiner.limits.RateLimitItem

Per year rate limited resource.

Parameters

  • amount (int) –

  • multiples (int) –

  • namespace (str) –

Return type

None

class freiner.limits.RateLimitItemPerMonth(amount: int, multiples: int = 1, namespace: str = 'LIMITER')[source]

Bases: freiner.limits.RateLimitItem

Per month rate limited resource.

Parameters

  • amount (int) –

  • multiples (int) –

  • namespace (str) –

Return type

None

class freiner.limits.RateLimitItemPerDay(amount: int, multiples: int = 1, namespace: str = 'LIMITER')[source]

Bases: freiner.limits.RateLimitItem

Per day rate limited resource.

Parameters

  • amount (int) –

  • multiples (int) –

  • namespace (str) –

Return type

None

class freiner.limits.RateLimitItemPerHour(amount: int, multiples: int = 1, namespace: str = 'LIMITER')[source]

Bases: freiner.limits.RateLimitItem

Per hour rate limited resource.

Parameters

  • amount (int) –

  • multiples (int) –

  • namespace (str) –

Return type

None

class freiner.limits.RateLimitItemPerMinute(amount: int, multiples: int = 1, namespace: str = 'LIMITER')[source]

Bases: freiner.limits.RateLimitItem

Per minute rate limited resource.

Parameters

  • amount (int) –

  • multiples (int) –

  • namespace (str) –

Return type

None

class freiner.limits.RateLimitItemPerSecond(amount: int, multiples: int = 1, namespace: str = 'LIMITER')[source]

Bases: freiner.limits.RateLimitItem

Per second rate limited resource.

Parameters

  • amount (int) –

  • multiples (int) –

  • namespace (str) –

Return type

None

Utility Methods

freiner.parse(limit_string: str) → freiner.limits.RateLimitItem[source]

Parses a single rate limit in string notation (e.g. ‘1/second’ or ‘1 per second’).

# noqa: DAR402

Parameters

limit_string (str) – The rate limit string. using Rate Limit String Notation.

Raises

  • TypeError – If something other than a string was supplied.

  • ValueError – If the string notation is invalid.

Returns

An instance of freiner.limits.RateLimitItem.

Return type

freiner.limits.RateLimitItem

freiner.parse_many(limit_string: str) → Sequence[freiner.limits.RateLimitItem][source]

Parses rate limits in string notation containing multiple rate limits (e.g. ‘1/second; 5/minute’).

Parameters

limit_string (str) – The rate limit string. using Rate Limit String Notation.

Raises

  • TypeError – If something other than a string was supplied.

  • ValueError – If the string notation is invalid.

Returns

A sequence of freiner.limits.RateLimitItem instances.

Return type

Sequence[freiner.limits.RateLimitItem]

Strategies

class freiner.strategies.RateLimiter(*args, **kwargs)[source]

Bases: Protocol

hit(item: freiner.limits.RateLimitItem, *identifiers: Any) → bool[source]

Creates a hit on the rate limit and returns True if successful.

# noqa: DAR202

Parameters
Returns

True if the request was successful, or False if the rate limit had been exceeded.

Return type

bool

test(item: freiner.limits.RateLimitItem, *identifiers: Any) → bool[source]

Checks the rate limit and returns True if it is not currently exceeded.

# noqa: DAR202

Parameters
Returns

True if the rate limit has not yet been exceeded, or False if it has.

Return type

bool

get_window_stats(item: freiner.limits.RateLimitItem, *identifiers: Any) → freiner.strategies.WindowStats[source]

Retrieve statistics about the number of requests remaining within the given limit.

# noqa: DAR202

Parameters
Returns

tuple (reset time (float), remaining (int))

Return type

freiner.strategies.WindowStats

clear(item: freiner.limits.RateLimitItem, *identifiers: Any) → None[source]

Resets the request counter for a given limit to zero.

Parameters
Return type

None

class freiner.strategies.WindowStats(reset_time, remaining_count)[source]

Bases: tuple

Create new instance of WindowStats(reset_time, remaining_count)

Parameters

  • reset_time (float) –

  • remaining_count (int) –

reset_time: float

Alias for field number 0

remaining_count: int

Alias for field number 1

class freiner.strategies.fixed_window.FixedWindowRateLimiter(storage: freiner.storage.FixedWindowStorage)[source]

Bases: object

Reference: Fixed Window

Parameters

storage (freiner.storage.FixedWindowStorage) –

Return type

None

hit(item: freiner.limits.RateLimitItem, *identifiers: Any) → bool[source]

Creates a hit on the rate limit and returns True if successful.

Parameters
Returns

True if the request was successful, or False if the rate limit had been exceeded.

Return type

bool

test(item: freiner.limits.RateLimitItem, *identifiers: Any) → bool[source]

Checks the rate limit and returns True if it is not currently exceeded.

Parameters
Returns

True if the rate limit has not yet been exceeded, or False if it has.

Return type

bool

get_window_stats(item: freiner.limits.RateLimitItem, *identifiers: Any) → freiner.strategies.WindowStats[source]

Returns the number of requests remaining within this limit.

Parameters
Returns

tuple (reset time (float), remaining (int))

Return type

freiner.strategies.WindowStats

clear(item: freiner.limits.RateLimitItem, *identifiers: Any) → None[source]

Resets the request counter for a given limit to zero.

Parameters
Return type

None

class freiner.strategies.fixed_window_elastic.FixedWindowElasticExpiryRateLimiter(storage: freiner.storage.FixedWindowStorage)[source]

Bases: freiner.strategies.fixed_window.FixedWindowRateLimiter

Reference: Fixed Window with Elastic Expiry

Parameters

storage (freiner.storage.FixedWindowStorage) –

Return type

None

hit(item: freiner.limits.RateLimitItem, *identifiers: Any) → bool[source]

Creates a hit on the rate limit and returns True if successful.

Parameters
Returns

True if the request was successful, or False if the rate limit had been exceeded.

Return type

bool

class freiner.strategies.moving_window.MovingWindowRateLimiter(storage: freiner.storage.MovingWindowStorage)[source]

Bases: object

Reference: Moving Window

Parameters

storage (freiner.storage.MovingWindowStorage) –

Return type

None

hit(item: freiner.limits.RateLimitItem, *identifiers: Any) → bool[source]

Creates a hit on the rate limit and returns True if successful.

Parameters
Returns

True if the request was successful, or False if the rate limit had been exceeded.

Return type

bool

test(item: freiner.limits.RateLimitItem, *identifiers: Any) → bool[source]

Checks the rate limit and returns True if it is not currently exceeded.

Parameters
Returns

True if the rate limit has not yet been exceeded, or False if it has.

Return type

bool

get_window_stats(item: freiner.limits.RateLimitItem, *identifiers: Any) → freiner.strategies.WindowStats[source]

Returns the number of requests remaining within this limit.

Parameters
Returns

tuple (reset time (float), remaining (int))

Return type

freiner.strategies.WindowStats

clear(item: freiner.limits.RateLimitItem, *identifiers: Any) → None[source]

Resets the request counter for a given limit to zero.

Parameters
Return type

None

Storage

Storage Protocol Classes

class freiner.storage.FixedWindowStorage(*args, **kwargs)[source]

Bases: Protocol

incr(key: str, expiry: int, elastic_expiry: bool = False) → int[source]

Increments the counter for the given rate limit key.

# noqa: DAR202

Parameters

  • key (str) – The key to increment.

  • expiry (int) – Amount in seconds for the key to expire in.

  • elastic_expiry (bool) – Whether to keep extending the rate limit window every hit.

Returns

The number of hits currently on the rate limit for the given key.

Return type

int

get(key: str) → int[source]

Retrieve the current request count for the given rate limit key.

Parameters

key (str) – The key to get the counter value for.

Return type

int

get_expiry(key: str) → float[source]

Retrieve the expected expiry time for the given rate limit key.

# noqa: DAR202

Parameters

key (str) – The key to get the expiry time for.

Returns

The time at which the current rate limit for the given key ends.

Return type

float

clear(key: str) → None[source]

Resets the rate limit for the given key.

Parameters

key (str) – The key to clear rate limits for.

Return type

None

class freiner.storage.MovingWindow(start_time, acquired_count)[source]

Bases: tuple

Create new instance of MovingWindow(start_time, acquired_count)

Parameters

  • start_time (float) –

  • acquired_count (int) –

start_time: float

Alias for field number 0

acquired_count: int

Alias for field number 1

class freiner.storage.MovingWindowStorage(*args, **kwargs)[source]

Bases: Protocol

acquire_entry(key: str, limit: int, expiry: int, no_add: bool = False) → bool[source]
Parameters

  • key (str) – The rate limit key to acquire an entry in.

  • limit (int) – The total amount of entries allowed before hitting the rate limit.

  • expiry (int) – Amount in seconds for the acquired entry to expire in.

  • no_add (bool) – If False, an entry is not actually acquired but instead serves as a ‘check’.

Return type

bool

get_moving_window(key: str, limit: int, expiry: int) → freiner.storage.MovingWindow[source]

Retrieves the starting point and the number of entries in the moving window.

# noqa: DAR202

Parameters

  • key (str) – The rate limit key to retrieve statistics about.

  • limit (int) – The total amount of entries allowed before hitting the rate limit.

  • expiry (int) – Amount in seconds for the acquired entry to expire in.

Returns

(start of window, number of acquired entries)

Return type

freiner.storage.MovingWindow

clear(key: str) → None[source]

Resets the rate limit for the given key.

Parameters

key (str) – The key to clear rate limits for.

Return type

None

Backend Implementations

In-Memory

class freiner.storage.memory.MemoryStorage[source]

Bases: object

rate limit storage using collections.Counter as an in memory storage for fixed and elastic window strategies, and a simple list to implement moving window strategy.

Return type

None

incr(key: str, expiry: int, elastic_expiry: bool = False) → int[source]

Increments the counter for the given rate limit key.

Parameters

  • key (str) – The key to increment.

  • expiry (int) – Amount in seconds for the key to expire in.

  • elastic_expiry (bool) – Whether to keep extending the rate limit window every hit.

Returns

The number of hits currently on the rate limit for the given key.

Return type

int

get(key: str) → int[source]

Retrieve the current request count for the given rate limit key.

Parameters

key (str) – The key to get the counter value for.

Return type

int

clear(key: str) → None[source]

Resets the rate limit for the given key.

Parameters

key (str) – The key to clear rate limits for.

Return type

None

acquire_entry(key: str, limit: int, expiry: int, no_add: bool = False) → bool[source]
Parameters

  • key (str) – The rate limit key to acquire an entry in.

  • limit (int) – The total amount of entries allowed before hitting the rate limit.

  • expiry (int) – Amount in seconds for the acquired entry to expire in.

  • no_add (bool) – If False, an entry is not actually acquired but instead serves as a ‘check’.

Return type

bool

get_expiry(key: str) → float[source]

Retrieve the expected expiry time for the given rate limit key.

Parameters

key (str) – The key to get the expiry time for.

Returns

The time at which the current rate limit for the given key ends.

Return type

float

get_num_acquired(key: str, expiry: int) → int[source]

returns the number of entries already acquired

Parameters

  • key (str) – rate limit key to acquire an entry in

  • expiry (int) – expiry of the entry

Return type

int

get_moving_window(key: str, limit: int, expiry: int) → freiner.storage.MovingWindow[source]

Retrieves the starting point and the number of entries in the moving window.

Parameters

  • key (str) – The rate limit key to retrieve statistics about.

  • limit (int) – The total amount of entries allowed before hitting the rate limit.

  • expiry (int) – Amount in seconds for the acquired entry to expire in.

Returns

(start of window, number of acquired entries)

Return type

freiner.storage.MovingWindow

check() → bool[source]

Check if the connection to the storage backend is healthy.

Return type

bool

Redis

class freiner.storage.redis.RedisStorage(client: redis.client.Redis)[source]

Bases: freiner.storage.redis.RedisInteractor

Rate limit storage with redis as backend.

Depends on the redis library.

Parameters

client (redis.client.Redis) –

Return type

None

classmethod from_uri(uri: str, **options: Any) → freiner.storage.redis.RedisStorage[source]
Parameters

  • uri (str) – URI of the form redis://[:password]@host:port, redis://[:password]@host:port/db, rediss://[:password]@host:port, unix:///path/to/sock etc. This uri is passed directly to redis.from_url().

  • options (Any) – All remaining keyword arguments are passed directly to the constructor of redis.Redis.

Return type

freiner.storage.redis.RedisStorage

incr(key: str, expiry: int, elastic_expiry: bool = False) → int[source]

Increments the counter for the given rate limit key.

Parameters

  • key (str) – The key to increment.

  • expiry (int) – Amount in seconds for the key to expire in.

  • elastic_expiry (bool) – Whether to keep extending the rate limit window every hit.

Returns

The number of hits currently on the rate limit for the given key.

Return type

int

get(key: str) → int[source]

Retrieve the current request count for the given rate limit key.

Parameters

key (str) – The key to get the counter value for.

Return type

int

clear(key: str) → None[source]

Resets the rate limit for the given key.

Parameters

key (str) – The key to clear rate limits for.

Return type

None

acquire_entry(key: str, limit: int, expiry: int, no_add: bool = False) → bool[source]
Parameters

  • key (str) – The rate limit key to acquire an entry in.

  • limit (int) – The total amount of entries allowed before hitting the rate limit.

  • expiry (int) – Amount in seconds for the acquired entry to expire in.

  • no_add (bool) – If False, an entry is not actually acquired but instead serves as a ‘check’.

Return type

bool

get_expiry(key: str) → float[source]

Retrieve the expected expiry time for the given rate limit key.

Parameters

key (str) – The key to get the expiry time for.

Returns

The time at which the current rate limit for the given key ends.

Return type

float

check() → bool[source]

Check if the connection to the storage backend is healthy.

Return type

bool

reset() → None[source]

This function calls a Lua Script to delete keys prefixed with ‘LIMITER’ in blocks of 5000.

Warning

This operation was designed to be fast, but was not tested on a large production based system. Be careful with its usage as it could be slow on very large data sets.

Return type

None

Redis Sentinel

class freiner.storage.redis_sentinel.RedisSentinelStorage(sentinel: redis.sentinel.Sentinel, service_name: str)[source]

Bases: freiner.storage.redis.RedisStorage

Rate limit storage with redis sentinel as backend.

Depends on redis library.

Parameters
Return type

None

classmethod from_uri(uri: str, service_name: Optional[str] = None, **options: Any) → freiner.storage.redis_sentinel.RedisSentinelStorage[source]
Parameters

  • uri (str) – URI of the form redis+sentinel://host:port,host:port/service_name

  • service_name (Optional[str]) – Sentinel service name (if not provided in uri).

  • options (Any) – All remaining keyword arguments are passed directly to the constructor of redis.sentinel.Sentinel.

Raises

FreinerConfigurationError – When no service name is provided.

Return type

freiner.storage.redis_sentinel.RedisSentinelStorage

get(key: str) → int[source]

Retrieve the current request count for the given rate limit key.

Parameters

key (str) – The key to get the counter value for.

Return type

int

get_expiry(key: str) → float[source]

Retrieve the expected expiry time for the given rate limit key.

Parameters

key (str) – The key to get the expiry time for.

Returns

The time at which the current rate limit for the given key ends.

Return type

float

check() → bool[source]

Check if the connection to the storage backend is healthy.

Return type

bool

Redis Cluster

class freiner.storage.redis_cluster.RedisClusterStorage(client: redis.client.Redis)[source]

Bases: freiner.storage.redis.RedisStorage

Rate limit storage with redis cluster as backend.

Depends on redis-py-cluster library.

Parameters

client (redis.client.Redis) –

Return type

None

classmethod from_uri(uri: str, **options: Any) → freiner.storage.redis_cluster.RedisClusterStorage[source]
Parameters

  • uri (str) – URI of the form redis+cluster://[:password]@host:port,host:port

  • options (Any) – All remaining keyword arguments are passed directly to the constructor of rediscluster.RedisCluster.

Return type

freiner.storage.redis_cluster.RedisClusterStorage

reset() → None[source]

Redis Clusters are sharded and deleting across shards can’t be done atomically. Because of this, this reset loops over all keys that are prefixed with ‘LIMITER’ and calls delete on them, one at a time.

Warning

This operation was not tested with extremely large data sets. On a large production based system, care should be taken with its usage as it could be slow on very large data sets.

Return type

None

Memcached

class freiner.storage.memcached.MemcachedStorage(client: Union[pymemcache.client.base.Client, pymemcache.client.base.PooledClient, pymemcache.client.hash.HashClient])[source]

Bases: object

Rate limit storage with memcached as backend.

Depends on the pymemcache library.

Parameters

client (Union[pymemcache.client.base.Client, pymemcache.client.base.PooledClient, pymemcache.client.hash.HashClient]) –

Return type

None

classmethod from_uri(uri: str, **options: Any) → freiner.storage.memcached.MemcachedStorage[source]
Parameters

  • uri (str) – URI of the form memcached://host:port,host:port`or `memcached:///run/path/to/sock.

  • options (Any) – All remaining keyword arguments are passed directly to the constructor of pymemcache.client.base.Client.

Raises

FreinerConfigurationError – When no hosts could be parsed from the supplied URI.

Return type

freiner.storage.memcached.MemcachedStorage

get(key: str) → int[source]

Retrieve the current request count for the given rate limit key.

Parameters

key (str) – The key to get the counter value for.

Return type

int

clear(key: str) → None[source]

Resets the rate limit for the given key.

Parameters

key (str) – The key to clear rate limits for.

Return type

None

incr(key: str, expiry: int, elastic_expiry: bool = False) → int[source]

Increments the counter for the given rate limit key.

Parameters

  • key (str) – The key to increment.

  • expiry (int) – Amount in seconds for the key to expire in.

  • elastic_expiry (bool) – Whether to keep extending the rate limit window every hit.

Returns

The number of hits currently on the rate limit for the given key.

Return type

int

get_expiry(key: str) → float[source]

Retrieve the expected expiry time for the given rate limit key.

Parameters

key (str) – The key to get the expiry time for.

Returns

The time at which the current rate limit for the given key ends.

Return type

float

check() → bool[source]

Check if the connection to the storage backend is healthy.

Return type

bool

Exceptions

exception freiner.errors.FreinerConfigurationError[source]

Bases: Exception

Raised when a configuration problem is encountered.