proxywhirl.rotator.sync¶

Main proxy rotation implementation.

Classes¶

ProxyWhirl

Main class for proxy rotation with automatic failover.

Module Contents¶

class proxywhirl.rotator.sync.ProxyWhirl(proxies=None, strategy=None, config=None, retry_policy=None, rate_limiter=None, bootstrap=None)[source]¶

Bases: proxywhirl.rotator.base.ProxyRotatorBase

Main class for proxy rotation with automatic failover.

Provides HTTP methods (GET, POST, PUT, DELETE, PATCH) that automatically rotate through a pool of proxies, with intelligent failover on connection errors.

Example:

from proxywhirl import ProxyWhirl, Proxy

rotator = ProxyWhirl()
rotator.add_proxy("http://proxy1.example.com:8080")
rotator.add_proxy("http://proxy2.example.com:8080")

response = rotator.get("https://httpbin.org/ip")
print(response.json())

Initialize ProxyWhirl.

Parameters:

proxies (list[proxywhirl.models.Proxy] | None) – Initial list of proxies (optional)
strategy (proxywhirl.strategies.RotationStrategy | str | None) – Rotation strategy instance or name string. Supported names: round-robin, random, weighted, least-used, performance-based, session, geo-targeted. Defaults to RoundRobinStrategy.
config (proxywhirl.models.ProxyConfiguration | None) – Configuration settings (default: ProxyConfiguration())
retry_policy (proxywhirl.retry.RetryPolicy | None) – Retry policy configuration (default: RetryPolicy())
rate_limiter (proxywhirl.rate_limiting.SyncRateLimiter | None) – Synchronous rate limiter for controlling request rates (optional)
bootstrap (proxywhirl.models.BootstrapConfig | bool | None) – Bootstrap configuration for lazy proxy fetching. False disables auto-bootstrap (for manual proxy management). True or None uses default BootstrapConfig.

add_chain(chain)[source]¶

Add a proxy chain to the rotator.

This method registers a proxy chain for potential use in routing. The entry proxy (first proxy in the chain) is added to the pool for selection by rotation strategies.

Note: Full CONNECT tunneling implementation is not yet supported. Currently, only the entry proxy is used for routing, with chain metadata stored for future multi-hop implementation.

Parameters:: chain (proxywhirl.models.ProxyChain) – ProxyChain instance to register
Return type:: None

Example

>>> rotator = ProxyWhirl()
>>> chain = ProxyChain(
...     proxies=[
...         Proxy(url="http://proxy1.com:8080"),
...         Proxy(url="http://proxy2.com:8080"),
...     ],
...     name="my_chain"
... )
>>> rotator.add_chain(chain)

add_proxy(proxy)[source]¶

Add a proxy to the pool.

Parameters:: proxy (proxywhirl.models.Proxy | str) – Proxy instance or URL string
Return type:: None

clear_queue()[source]¶

Clear all pending requests from the queue.

Returns:: Number of requests cleared
Raises:: RuntimeError – If queue is not enabled
Return type:: int

clear_unhealthy_proxies()[source]¶

Remove all unhealthy and dead proxies from the pool.

Returns:: Number of proxies removed
Return type:: int

delete(url, **kwargs)[source]¶

Make DELETE request.

Parameters:

url (str)
kwargs (Any)

Return type:

httpx.Response

get(url, **kwargs)[source]¶

Make GET request.

Parameters:

url (str)
kwargs (Any)

Return type:

httpx.Response

get_chains()[source]¶

Get all registered proxy chains.

Returns:: List of ProxyChain instances
Return type:: list[proxywhirl.models.ProxyChain]

get_circuit_breaker_states()[source]¶

Get circuit breaker states for all proxies.

Returns:: Mapping of proxy IDs to their circuit breaker instances.
Return type:: dict[str, CircuitBreaker]

get_pool_stats()[source]¶

Get statistics about the proxy pool.

Returns:: Pool statistics including total_proxies, healthy_proxies, unhealthy_proxies, dead_proxies, total_requests, total_successes, total_failures, and average_success_rate.
Return type:: dict[str, Any]

get_queue_stats()[source]¶

Get statistics about the request queue.

Returns:: Queue statistics including enabled, size, max_size, is_full, and is_empty.
Return type:: dict[str, Any]

get_retry_metrics()[source]¶

Get retry metrics.

Returns:: RetryMetrics instance with current metrics
Return type:: proxywhirl.retry.RetryMetrics

get_statistics()[source]¶

Get comprehensive statistics including source breakdown (FR-050).

Returns:: All stats from get_pool_stats() plus source_breakdown mapping source names to proxy counts.
Return type:: dict[str, Any]

head(url, **kwargs)[source]¶

Make HEAD request.

Parameters:

url (str)
kwargs (Any)

Return type:

httpx.Response

options(url, **kwargs)[source]¶

Make OPTIONS request.

Parameters:

url (str)
kwargs (Any)

Return type:

httpx.Response

patch(url, **kwargs)[source]¶

Make PATCH request.

Parameters:

url (str)
kwargs (Any)

Return type:

httpx.Response

post(url, **kwargs)[source]¶

Make POST request.

Parameters:

url (str)
kwargs (Any)

Return type:

httpx.Response

put(url, **kwargs)[source]¶

Make PUT request.

Parameters:

url (str)
kwargs (Any)

Return type:

httpx.Response

remove_chain(chain_name)[source]¶

Remove a proxy chain by name.

Parameters:: chain_name (str) – Name of the chain to remove
Returns:: True if chain was found and removed, False otherwise
Return type:: bool

remove_proxy(proxy_id)[source]¶

Remove a proxy from the pool.

Parameters:: proxy_id (str) – UUID of proxy to remove
Return type:: None

reset_circuit_breaker(proxy_id)[source]¶

Manually reset a circuit breaker to CLOSED state.

Parameters:: proxy_id (str) – ID of the proxy whose circuit breaker to reset
Raises:: KeyError – If proxy_id not found
Return type:: None

set_strategy(strategy, *, atomic=True)[source]¶

Hot-swap the rotation strategy without restarting.

This method implements atomic strategy swapping to ensure: - New requests immediately use the new strategy - In-flight requests complete with their original strategy - No requests are dropped during the swap - Swap completes in <100ms (SC-009)

Parameters:

strategy (proxywhirl.strategies.RotationStrategy | str) – New strategy instance or name string. Supported names: round-robin, random, weighted, least-used, performance-based, session, geo-targeted.
atomic (bool) – If True (default), ensures atomic swap. If False, allows immediate replacement (faster but may affect in-flight requests)

Return type:

None

Example

>>> rotator = ProxyWhirl(strategy="round-robin")
>>> # ... after some requests ...
>>> rotator.set_strategy("performance-based")  # Hot-swap
>>> # New requests now use performance-based strategy

Thread Safety:: Thread-safe via atomic reference swap. Multiple threads can call this method safely without race conditions.
Performance:: Target: <100ms for hot-swap completion (SC-009) Typical: <10ms for strategy instance creation and assignment