proxywhirl.api.core¶

FastAPI REST API for ProxyWhirl proxy rotation service.

This module provides HTTP endpoints for: - Making proxied HTTP requests - Managing proxy pool (CRUD operations) - Monitoring health and status - Configuring runtime settings

The API uses: - FastAPI for async request handling and auto-generated OpenAPI docs - slowapi for rate limiting - Optional API key authentication - Singleton ProxyWhirl for proxy management

Classes¶

`AuditLoggingMiddleware`	Structured audit logging for API operations.
`RequestIDMiddleware`	Add request ID correlation for request tracing.
`RequestLoggingMiddleware`	Log all HTTP requests with structured JSON logging.
`SecurityHeadersMiddleware`	Add security headers to all responses.

Functions¶

`add_proxy`(request, proxy_data[, rotator, api_key])	Add a new proxy to the pool.
`delete_proxy`(proxy_id[, rotator, storage, api_key])	Remove a proxy from the pool.
`get_circuit_breaker`(proxy_id[, api_key])	Get circuit breaker state for a specific proxy.
`get_circuit_breaker_metrics`([hours, api_key])	Get circuit breaker state change events.
`get_circuit_breaker_metrics_endpoint`([format, hours, ...])	Get circuit breaker states and events in JSON or Prometheus format.
`get_config`()	Get current API configuration.
`get_configuration`([config, api_key])	Get current API configuration.
`get_proxy`(proxy_id[, rotator, api_key])	Get details of a specific proxy.
`get_rate_limit_key`(request)	Extract rate limit key from request.
`get_retry_metrics`([api_key])	Get aggregated retry metrics.
`get_retry_metrics_endpoint`([format, hours, api_key])	Get retry statistics in JSON or Prometheus format.
`get_retry_policy`([api_key])	Get the current global retry policy configuration.
`get_retry_stats_by_proxy`([hours, api_key])	Get retry statistics grouped by proxy.
`get_retry_timeseries`([hours, api_key])	Get hourly retry metrics for the specified time range.
`get_rotator`()	Get the singleton ProxyWhirl instance.
`get_stats`([rotator])	Get API performance statistics (general aggregate metrics).
`get_status`([rotator, storage, config])	Get detailed system status including pool stats.
`get_storage`()	Get the optional SQLiteStorage instance.
`health_check`([rotator])	Check API health status.
`health_check_proxies`(request_data[, rotator, api_key])	Run health checks on specified proxies.
`health_check_proxies_deprecated`(request_data[, ...])	Run health checks on specified proxies.
`internal_error_handler`(request, exc)	Handle 500 Internal Server errors.
`lifespan`(app)	Application lifespan manager for startup and shutdown.
`list_circuit_breakers`([api_key])	Get circuit breaker states for all proxies.
`list_proxies`([page, page_size, status_filter, ...])	List all proxies in the pool with pagination and filtering.
`make_proxied_request`(request[, request_data, rotator, ...])	Make an HTTP request through a rotating proxy.
`metrics`()	Expose Prometheus metrics in text format.
`not_found_handler`(request, exc)	Handle 404 Not Found errors.
`proxy_error_handler`(request, exc)	Handle ProxyWhirlError exceptions with enhanced error details.
`readiness_check`([rotator, storage])	Check if API is ready to serve requests.
`reset_circuit_breaker`(request, proxy_id[, api_key])	Manually reset a circuit breaker to CLOSED state.
`root`()	Root endpoint - redirect to docs.
`update_configuration`(request, update_data[, config, ...])	Update API configuration at runtime.
`update_prometheus_metrics`()	Update Prometheus metrics for proxy pool and circuit breakers.
`update_retry_policy`(policy_request[, api_key])	Update the global retry policy configuration.
`validate_proxied_request_url`(request_data)	Dependency to validate target URL for SSRF protection.
`validation_error_handler`(request, exc)	Handle 422 Validation errors.
`verify_api_key`([api_key])	Verify API key if authentication is required.

Module Contents¶

class proxywhirl.api.core.AuditLoggingMiddleware(app, dispatch=None)[source]¶

Bases: starlette.middleware.base.BaseHTTPMiddleware

Structured audit logging for API operations.

Provides security audit trail including: - Authentication context (API key used, redacted) - Operation classification (read, write, admin, auth) - Resource identification for mutating operations - Request body logging for writes (with sensitive data redaction)

All logs use structured JSON format for easy parsing by SIEM tools.

Parameters:

app (starlette.types.ASGIApp)
dispatch (DispatchFunction | None)

async dispatch(request, call_next)[source]¶

Process request and emit audit log for sensitive operations.

Parameters:

request (fastapi.Request)
call_next (Any)

Return type:

Any

class proxywhirl.api.core.RequestIDMiddleware(app, dispatch=None)[source]¶

Bases: starlette.middleware.base.BaseHTTPMiddleware

Add request ID correlation for request tracing.

This middleware ensures every request has a unique identifier that can be used for tracing requests through the retry/cache/circuit-breaker chain.

The request ID is: - Taken from the X-Request-ID header if provided by the client - Generated as a new UUID v4 if not provided - Added to loguru context for all downstream logging - Included in the response X-Request-ID header

Parameters:

app (starlette.types.ASGIApp)
dispatch (DispatchFunction | None)

async dispatch(request, call_next)[source]¶

Process request and add request ID correlation.

Parameters:

request (fastapi.Request)
call_next (Any)

Return type:

Any

class proxywhirl.api.core.RequestLoggingMiddleware(app, dispatch=None)[source]¶

Bases: starlette.middleware.base.BaseHTTPMiddleware

Log all HTTP requests with structured JSON logging.

Logs: - Request method, path, and query parameters (redacted) - Client IP address - Request duration in milliseconds - Response status code - Sensitive data redaction (passwords, tokens, API keys)

Parameters:

app (starlette.types.ASGIApp)
dispatch (DispatchFunction | None)

async dispatch(request, call_next)[source]¶

Process request and log details.

Parameters:

request (fastapi.Request)
call_next (Any)

Return type:

Any

class proxywhirl.api.core.SecurityHeadersMiddleware(app, dispatch=None)[source]¶

Bases: starlette.middleware.base.BaseHTTPMiddleware

Add security headers to all responses.

Parameters:

app (starlette.types.ASGIApp)
dispatch (DispatchFunction | None)

async proxywhirl.api.core.add_proxy(request, proxy_data, rotator=Depends(get_rotator), api_key=Depends(verify_api_key))[source]¶

Add a new proxy to the pool.

Parameters:

proxy_data (proxywhirl.api.models.CreateProxyRequest) – Proxy URL and optional credentials
rotator (proxywhirl.rotator.ProxyWhirl) – ProxyWhirl dependency
api_key (None) – API key verification
request (fastapi.Request)

Returns:

Created proxy resource

Return type:

proxywhirl.api.models.APIResponse[proxywhirl.api.models.ProxyResource]

async proxywhirl.api.core.delete_proxy(proxy_id, rotator=Depends(get_rotator), storage=Depends(get_storage), api_key=Depends(verify_api_key))[source]¶

Remove a proxy from the pool.

Parameters:

proxy_id (str) – Proxy identifier
rotator (proxywhirl.rotator.ProxyWhirl) – ProxyWhirl dependency
storage (proxywhirl.storage.SQLiteStorage | None) – Optional storage dependency
api_key (None) – API key verification

Return type:

None

async proxywhirl.api.core.get_circuit_breaker(proxy_id, api_key=Depends(verify_api_key))[source]¶

Get circuit breaker state for a specific proxy.

Parameters:

proxy_id (str) – Proxy ID to get circuit breaker for
api_key (None)

Returns:

Circuit breaker state

Return type:

proxywhirl.api.models.APIResponse[proxywhirl.api.models.CircuitBreakerResponse]

async proxywhirl.api.core.get_circuit_breaker_metrics(hours=24, api_key=Depends(verify_api_key))[source]¶

Get circuit breaker state change events.

Parameters:

hours (int) – Number of hours to retrieve (default: 24)
api_key (None)

Returns:

List of circuit breaker events

Return type:

proxywhirl.api.models.APIResponse[list[proxywhirl.api.models.CircuitBreakerEventResponse]]

async proxywhirl.api.core.get_circuit_breaker_metrics_endpoint(format=None, hours=24, api_key=Depends(verify_api_key))[source]¶

Get circuit breaker states and events in JSON or Prometheus format.

This endpoint provides circuit breaker information including: - Current state of all circuit breakers - State change events (history) - Failure counts and thresholds

Parameters:

format (str | None) – Output format (‘prometheus’ for Prometheus text format, default: JSON)
hours (int) – Number of hours of event history to include (default: 24)
api_key (None) – API key verification

Returns:

Circuit breaker metrics in requested format

Return type:

fastapi.Response

Example Prometheus format:

# HELP proxywhirl_circuit_breaker_state Circuit breaker state (0=closed, 1=open, 2=half_open) # TYPE proxywhirl_circuit_breaker_state gauge proxywhirl_circuit_breaker_state{proxy_id=”proxy1:8080”} 0

# HELP proxywhirl_circuit_breaker_failure_count Current failure count # TYPE proxywhirl_circuit_breaker_failure_count gauge proxywhirl_circuit_breaker_failure_count{proxy_id=”proxy1:8080”} 2

proxywhirl.api.core.get_config()[source]¶

Get current API configuration.

Returns:: Configuration dictionary
Return type:: dict[str, Any]

async proxywhirl.api.core.get_configuration(config=Depends(get_config), api_key=Depends(verify_api_key))[source]¶

Get current API configuration.

Parameters:

config (dict[str, Any]) – Configuration dependency
api_key (None) – API key verification

Returns:

Current configuration settings

Return type:

proxywhirl.api.models.APIResponse[proxywhirl.api.models.ConfigurationSettings]

async proxywhirl.api.core.get_proxy(proxy_id, rotator=Depends(get_rotator), api_key=Depends(verify_api_key))[source]¶

Get details of a specific proxy.

Parameters:

proxy_id (str) – Proxy identifier
rotator (proxywhirl.rotator.ProxyWhirl) – ProxyWhirl dependency
api_key (None) – API key verification

Returns:

Proxy resource

Return type:

proxywhirl.api.models.APIResponse[proxywhirl.api.models.ProxyResource]

proxywhirl.api.core.get_rate_limit_key(request)[source]¶

Extract rate limit key from request.

SECURITY: This function is designed to prevent rate limit bypass attacks.

For authenticated requests (with API key):

Uses hashed API key as rate limit key
This ensures rate limiting is per-API-key, not per-IP

For unauthenticated requests:

Uses ONLY direct client IP (request.client.host)
NEVER trusts X-Forwarded-For header to prevent spoofing attacks
Attackers cannot bypass rate limits by sending fake X-Forwarded-For headers

Note: If you need to trust X-Forwarded-For (e.g., behind a reverse proxy), configure your reverse proxy to set the real client IP in request.client, or use a trusted proxy middleware that validates the header chain.

Parameters:: request (fastapi.Request) – FastAPI Request object
Returns:: Rate limit key in the form apikey:{hash} or ip:{address}.
Return type:: str

async proxywhirl.api.core.get_retry_metrics(api_key=Depends(verify_api_key))[source]¶

Get aggregated retry metrics.

Returns:: Retry metrics summary
Parameters:: api_key (None)
Return type:: proxywhirl.api.models.APIResponse[proxywhirl.api.models.RetryMetricsResponse]

async proxywhirl.api.core.get_retry_metrics_endpoint(format=None, hours=24, api_key=Depends(verify_api_key))[source]¶

Get retry statistics in JSON or Prometheus format.

This endpoint provides comprehensive retry metrics including: - Total retry attempts - Success rate by attempt number - Time-series data (hourly aggregates) - Per-proxy statistics

Parameters:

format (str | None) – Output format (‘prometheus’ for Prometheus text format, default: JSON)
hours (int) – Number of hours of data to include (default: 24)
api_key (None) – API key verification

Returns:

Retry metrics in requested format

Return type:

fastapi.Response

Example Prometheus format:

# HELP proxywhirl_retry_total Total number of retry attempts # TYPE proxywhirl_retry_total counter proxywhirl_retry_total 1250

# HELP proxywhirl_retry_success_by_attempt Successful requests by attempt number # TYPE proxywhirl_retry_success_by_attempt gauge proxywhirl_retry_success_by_attempt{attempt=”0”} 850 proxywhirl_retry_success_by_attempt{attempt=”1”} 300

async proxywhirl.api.core.get_retry_policy(api_key=Depends(verify_api_key))[source]¶

Get the current global retry policy configuration.

Returns:: Current retry policy settings
Parameters:: api_key (None)
Return type:: proxywhirl.api.models.APIResponse[proxywhirl.api.models.RetryPolicyResponse]

async proxywhirl.api.core.get_retry_stats_by_proxy(hours=24, api_key=Depends(verify_api_key))[source]¶

Get retry statistics grouped by proxy.

Parameters:

hours (int) – Number of hours to analyze (default: 24)
api_key (None)

Returns:

Per-proxy retry statistics

Return type:

proxywhirl.api.models.APIResponse[proxywhirl.api.models.ProxyRetryStatsResponse]

async proxywhirl.api.core.get_retry_timeseries(hours=24, api_key=Depends(verify_api_key))[source]¶

Get hourly retry metrics for the specified time range.

Parameters:

hours (int) – Number of hours to retrieve (default: 24)
api_key (None)

Returns:

Time-series retry data

Return type:

proxywhirl.api.models.APIResponse[proxywhirl.api.models.TimeSeriesResponse]

proxywhirl.api.core.get_rotator()[source]¶

Get the singleton ProxyWhirl instance.

Returns:: ProxyWhirl instance
Raises:: HTTPException – If rotator not initialized
Return type:: proxywhirl.rotator.ProxyWhirl

async proxywhirl.api.core.get_stats(rotator=Depends(get_rotator))[source]¶

Get API performance statistics (general aggregate metrics).

This endpoint provides high-level performance statistics for the API, distinct from the detailed Prometheus metrics available at /metrics.

Parameters:: rotator (proxywhirl.rotator.ProxyWhirl) – ProxyWhirl dependency
Returns:: Performance statistics response
Return type:: proxywhirl.api.models.APIResponse[proxywhirl.api.models.MetricsResponse]

Note

This endpoint provides aggregate metrics. For detailed metrics, use: - /metrics - Prometheus format metrics - /api/v1/metrics/retries - Retry-specific metrics - /metrics/retry - Comprehensive retry metrics with JSON/Prometheus format

async proxywhirl.api.core.get_status(rotator=Depends(get_rotator), storage=Depends(get_storage), config=Depends(get_config))[source]¶

Get detailed system status including pool stats.

Parameters:

rotator (proxywhirl.rotator.ProxyWhirl) – ProxyWhirl dependency
storage (proxywhirl.storage.SQLiteStorage | None) – Optional storage dependency
config (dict[str, Any]) – Configuration dependency

Returns:

System status response

Return type:

proxywhirl.api.models.APIResponse[proxywhirl.api.models.StatusResponse]

proxywhirl.api.core.get_storage()[source]¶

Get the optional SQLiteStorage instance.

Returns:: SQLiteStorage instance or None if not configured
Return type:: proxywhirl.storage.SQLiteStorage | None

async proxywhirl.api.core.health_check(rotator=Depends(get_rotator))[source]¶

Check API health status.

Returns 200 if healthy, 503 if unhealthy.

Parameters:: rotator (proxywhirl.rotator.ProxyWhirl) – ProxyWhirl dependency
Returns:: Health status response
Return type:: fastapi.responses.JSONResponse

async proxywhirl.api.core.health_check_proxies(request_data, rotator=Depends(get_rotator), api_key=Depends(verify_api_key))[source]¶

Run health checks on specified proxies.

Parameters:

request_data (proxywhirl.api.models.HealthCheckRequest) – Optional list of proxy IDs to check
rotator (proxywhirl.rotator.ProxyWhirl) – ProxyWhirl dependency
api_key (None) – API key verification

Returns:

List of health check results

Return type:

proxywhirl.api.models.APIResponse[list[proxywhirl.api.models.HealthCheckResult]]

async proxywhirl.api.core.health_check_proxies_deprecated(request_data, rotator=Depends(get_rotator), api_key=Depends(verify_api_key))[source]¶

Run health checks on specified proxies.

DEPRECATED: Use /api/v1/proxies/health-check instead. This endpoint is kept for backward compatibility and will be removed in a future version.

Parameters:

request_data (proxywhirl.api.models.HealthCheckRequest) – Optional list of proxy IDs to check
rotator (proxywhirl.rotator.ProxyWhirl) – ProxyWhirl dependency
api_key (None) – API key verification

Returns:

List of health check results

Return type:

proxywhirl.api.models.APIResponse[list[proxywhirl.api.models.HealthCheckResult]]

async proxywhirl.api.core.internal_error_handler(request, exc)[source]¶

Handle 500 Internal Server errors.

Parameters:

request (fastapi.Request)
exc (Exception)

Return type:

fastapi.responses.JSONResponse

async proxywhirl.api.core.lifespan(app)[source]¶

Application lifespan manager for startup and shutdown.

Handles: - ProxyWhirl initialization on startup - Optional SQLiteStorage initialization - Graceful cleanup on shutdown

Parameters:: app (fastapi.FastAPI)
Return type:: collections.abc.AsyncIterator[None]

async proxywhirl.api.core.list_circuit_breakers(api_key=Depends(verify_api_key))[source]¶

Get circuit breaker states for all proxies.

Returns:: List of circuit breaker states
Parameters:: api_key (None)
Return type:: proxywhirl.api.models.APIResponse[list[proxywhirl.api.models.CircuitBreakerResponse]]

async proxywhirl.api.core.list_proxies(page=1, page_size=50, status_filter=None, rotator=Depends(get_rotator), api_key=Depends(verify_api_key))[source]¶

List all proxies in the pool with pagination and filtering.

Parameters:

page (int) – Page number (1-indexed)
page_size (int) – Number of items per page (max 100)
status_filter (str | None) – Filter by health status (optional)
rotator (proxywhirl.rotator.ProxyWhirl) – ProxyWhirl dependency
api_key (None) – API key verification

Returns:

Paginated list of proxy resources

Return type:

proxywhirl.api.models.APIResponse[proxywhirl.api.models.PaginatedResponse[proxywhirl.api.models.ProxyResource]]

async proxywhirl.api.core.make_proxied_request(request, request_data=Depends(validate_proxied_request_url), rotator=Depends(get_rotator), api_key=Depends(verify_api_key))[source]¶

Make an HTTP request through a rotating proxy.

This endpoint routes your HTTP request through the proxy pool, automatically handling rotation and failover.

SECURITY: All target URLs are validated to prevent SSRF attacks. The following are blocked by default: - Localhost and loopback addresses (127.0.0.0/8, ::1) - Private IP ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) - Link-local addresses (169.254.0.0/16) - Internal domain names (.local, .internal, .lan, .corp) - Non-HTTP/HTTPS schemes (file://, data://, etc.)

Parameters:

request_data (proxywhirl.api.models.ProxiedRequest) – Request details (URL, method, headers, body, timeout) - validated for SSRF
rotator (proxywhirl.rotator.ProxyWhirl) – ProxyWhirl dependency injection
api_key (None) – API key verification dependency
request (fastapi.Request)

Returns:

APIResponse with proxied response data

Raises:

HTTPException – For various error conditions including SSRF protection

Return type:

proxywhirl.api.models.APIResponse[proxywhirl.api.models.ProxiedResponse]

async proxywhirl.api.core.metrics()[source]¶

Expose Prometheus metrics in text format.

This endpoint returns metrics in Prometheus exposition format, including: - proxywhirl_requests_total: Total HTTP requests by endpoint, method, and status - proxywhirl_request_duration_seconds: Request duration histogram - proxywhirl_proxies_total: Total proxies in pool - proxywhirl_proxies_healthy: Number of healthy proxies - proxywhirl_circuit_breaker_state: Circuit breaker states (0=closed, 1=open, 2=half-open)

Returns:: Prometheus metrics in text format
Return type:: fastapi.Response

async proxywhirl.api.core.not_found_handler(request, exc)[source]¶

Handle 404 Not Found errors.

Parameters:

request (fastapi.Request)
exc (Any)

Return type:

fastapi.responses.JSONResponse

async proxywhirl.api.core.proxy_error_handler(request, exc)[source]¶

Handle ProxyWhirlError exceptions with enhanced error details.

Parameters:

request (fastapi.Request)
exc (proxywhirl.exceptions.ProxyWhirlError)

Return type:

fastapi.responses.JSONResponse

async proxywhirl.api.core.readiness_check(rotator=Depends(get_rotator), storage=Depends(get_storage))[source]¶

Check if API is ready to serve requests.

Returns 200 if ready, 503 if not ready.

Parameters:

rotator (proxywhirl.rotator.ProxyWhirl) – ProxyWhirl dependency
storage (proxywhirl.storage.SQLiteStorage | None) – Optional storage dependency

Returns:

Readiness status response

Return type:

fastapi.responses.JSONResponse

async proxywhirl.api.core.reset_circuit_breaker(request, proxy_id, api_key=Depends(verify_api_key))[source]¶

Manually reset a circuit breaker to CLOSED state.

Parameters:

proxy_id (str) – Proxy ID whose circuit breaker to reset
request (fastapi.Request)
api_key (None)

Returns:

Updated circuit breaker state

Return type:

proxywhirl.api.models.APIResponse[proxywhirl.api.models.CircuitBreakerResponse]

async proxywhirl.api.core.root()[source]¶

Root endpoint - redirect to docs.

Return type:: dict[str, str]

async proxywhirl.api.core.update_configuration(request, update_data, config=Depends(get_config), rotator=Depends(get_rotator), api_key=Depends(verify_api_key))[source]¶

Update API configuration at runtime.

Parameters:

update_data (proxywhirl.api.models.UpdateConfigRequest) – Configuration updates (partial)
config (dict[str, Any]) – Configuration dependency
rotator (proxywhirl.rotator.ProxyWhirl) – ProxyWhirl dependency
api_key (None) – API key verification
request (fastapi.Request)

Returns:

Updated configuration settings

Return type:

proxywhirl.api.models.APIResponse[proxywhirl.api.models.ConfigurationSettings]

proxywhirl.api.core.update_prometheus_metrics()[source]¶

Update Prometheus metrics for proxy pool and circuit breakers.

Return type:: None

async proxywhirl.api.core.update_retry_policy(policy_request, api_key=Depends(verify_api_key))[source]¶

Update the global retry policy configuration.

Parameters:

policy_request (proxywhirl.api.models.RetryPolicyRequest) – New retry policy settings
api_key (None)

Returns:

Updated retry policy

Return type:

proxywhirl.api.models.APIResponse[proxywhirl.api.models.RetryPolicyResponse]

proxywhirl.api.core.validate_proxied_request_url(request_data)[source]¶

Dependency to validate target URL for SSRF protection.

This dependency runs BEFORE other dependencies to ensure SSRF validation happens first, preventing malicious URLs from being processed.

Parameters:: request_data (proxywhirl.api.models.ProxiedRequest) – The proxied request data
Returns:: The validated request data
Raises:: HTTPException – If URL is invalid or blocked for security reasons
Return type:: proxywhirl.api.models.ProxiedRequest

async proxywhirl.api.core.validation_error_handler(request, exc)[source]¶

Handle 422 Validation errors.

Returns 400 Bad Request for client input validation failures, which is more semantically appropriate than 422 Unprocessable Entity.

Parameters:

request (fastapi.Request)
exc (Any)

Return type:

fastapi.responses.JSONResponse

proxywhirl.api.core.verify_api_key(api_key=Depends(api_key_header))[source]¶

Verify API key if authentication is required.

Parameters:: api_key (str | None) – API key from X-API-Key header
Raises:: HTTPException – If auth is required and key is invalid
Return type:: None