proxywhirl.rotator.client_pool

LRU cache for httpx.Client instances with automatic eviction.

This module provides a thread-safe pool for reusing HTTP client connections, improving performance by avoiding repeated connection setup.

Classes

LRUClientPool

LRU cache for httpx.Client instances with automatic eviction.

Module Contents

class proxywhirl.rotator.client_pool.LRUClientPool(maxsize=100)

LRU cache for httpx.Client instances with automatic eviction.

When the pool reaches maxsize, the least recently used client is closed and removed to prevent unbounded memory growth.

Supports dictionary-like access for backward compatibility with tests.

Initialize LRU client pool.

Parameters:

maxsize (int) – Maximum number of clients to cache (default: 100)

clear()

Close all clients and clear the pool.

Return type:

None

get(proxy_id)

Get a client from the pool, marking it as recently used.

Parameters:

proxy_id (str) – Proxy ID to look up

Returns:

Client if found, None otherwise

Return type:

httpx.Client | None

put(proxy_id, client)

Add a client to the pool, evicting LRU client if at capacity.

Parameters:

• proxy_id (str) – Proxy ID to store under

• client (httpx.Client) – Client instance to store

Return type:

None

remove(proxy_id)

Remove and close a client from the pool.

Parameters:

proxy_id (str) – Proxy ID to remove

Return type:

None