proxywhirl.cache.tiers

Cache tier implementations for multi-tier caching strategy.

Defines: - CacheTier: Abstract base class for cache tier implementations - MemoryCacheTier: L1 in-memory cache using OrderedDict - FileCacheTier: L2 JSONL file cache with encryption and file locking - SQLiteCacheTier: L3 database cache with full persistence

Classes

CacheTier

Abstract base class for cache tier implementations.

DiskCacheTier

L2 SQLite-based cache with encryption and indexed lookups.

JsonlCacheTier

L2 JSONL file-based cache with sharding and encryption.

MemoryCacheTier

L1 in-memory cache using OrderedDict for LRU tracking.

SQLiteCacheTier

L3 SQLite database cache with encrypted credentials.

TierType

Cache tier types.

Module Contents

class proxywhirl.cache.tiers.CacheTier(config, tier_type)[source]

Bases: abc.ABC

Abstract base class for cache tier implementations.

Defines the interface that all cache tiers (L1, L2, L3) must implement, including graceful degradation on repeated failures.

config[source]

Configuration for this tier

tier_type[source]

Type of tier (L1/L2/L3)

enabled[source]

Whether tier is operational

failure_count[source]

Consecutive failures for degradation tracking

failure_threshold[source]

Failures before auto-disabling tier

Initialize cache tier with configuration.

Parameters:
abstractmethod cleanup_expired()[source]

Remove all expired entries in bulk.

Returns:

Number of entries removed

Return type:

int

abstractmethod clear()[source]

Clear all entries, return count of removed entries.

Returns:

Number of entries removed

Return type:

int

abstractmethod delete(key)[source]

Remove entry by key, return True if existed.

Parameters:

key (str) – Cache key to delete

Returns:

True if entry existed and was deleted, False if not found

Return type:

bool

abstractmethod get(key)[source]

Retrieve entry by key, None if not found or expired.

Parameters:

key (str) – Cache key to lookup

Returns:

CacheEntry if found and valid, None otherwise

Return type:

proxywhirl.cache.models.CacheEntry | None

handle_failure(error)[source]

Handle tier operation failure for graceful degradation.

Increments failure count and disables tier if threshold exceeded. Called by implementations when operations fail.

Parameters:

error (Exception) – Exception that occurred

Return type:

None

abstractmethod keys()[source]

Return list of all keys.

Returns:

List of cache keys

Return type:

list[str]

abstractmethod put(key, entry)[source]

Store entry, return True if successful.

Parameters:
Returns:

True if stored successfully, False otherwise

Return type:

bool

reset_failures()[source]

Reset failure count on successful operation.

Re-enables tier if previously disabled and resets failure counter. Implementations should call this after successful operations.

Return type:

None

abstractmethod size()[source]

Return current number of entries.

Returns:

Number of entries in tier

Return type:

int

class proxywhirl.cache.tiers.DiskCacheTier(config, tier_type, cache_dir, encryptor=None)[source]

Bases: CacheTier

L2 SQLite-based cache with encryption and indexed lookups.

Optimized for >10K entries using SQLite with B-tree indexes instead of JSONL. Provides O(log n) lookups vs O(n) for JSONL, achieving <10ms reads for 10K+ entries.

Uses a lightweight SQLite database with: - Primary key index on cache key for fast lookups - Encrypted credentials stored as BLOB - Efficient bulk operations (cleanup, size, keys) - File-based persistence without complex sharding - Persistent connection pooling for performance (RES-005)

Thread Safety:

Uses a threading.Lock to protect connection access. The connection is created with check_same_thread=False to allow multi-threaded access.

Initialize SQLite-based L2 cache.

Parameters:
cleanup_expired()[source]

Remove all expired entries from SQLite L2 database using indexed SQL DELETE.

Performs bulk deletion of expired entries in a single SQL operation. Uses the expires_at index for efficient identification.

Returns:

Number of expired entries that were removed, 0 on error.

Return type:

int

Side Effects:

  • Acquires persistent connection via thread-safe _get_connection().

  • Calculates current timestamp in UTC.

  • Executes DELETE with WHERE clause using expires_at index.

  • Commits transaction before returning.

  • Resets failure counter on success.

Thread Safety:

Thread-safe via connection lock in _get_connection().

Performance:

O(m log n) where m is number of expired entries and n is total entries. The idx_l2_expires_at index makes this significantly faster than full table scan.

Raises:

Exception – Caught and handled via handle_failure(), returns 0.

Return type:

int

Example

>>> tier = DiskCacheTier(config, TierType.L2_FILE, cache_dir)
>>> removed = tier.cleanup_expired()
>>> print(f"Removed {removed} expired entries")
Removed 42 expired entries
clear()[source]

Clear all entries from SQLite L2 cache database.

Performs bulk deletion of all cache entries. More efficient than iterating through individual deletes.

Returns:

Number of entries that were cleared, 0 on error.

Return type:

int

Side Effects:

  • Acquires persistent connection via thread-safe _get_connection().

  • Counts total entries before deletion.

  • Executes DELETE FROM without WHERE clause (removes all rows).

  • Commits transaction before returning.

  • Resets failure counter on success.

Thread Safety:

Thread-safe via connection lock in _get_connection().

Performance:

O(n) where n is the number of entries, but executes as a single SQL operation with automatic index cleanup.

Raises:

Exception – Caught and handled via handle_failure(), returns 0.

Return type:

int

Example

>>> tier = DiskCacheTier(config, TierType.L2_FILE, cache_dir)
>>> tier.size()
1000
>>> cleared = tier.clear()
>>> print(f"Cleared {cleared} entries")
Cleared 1000 entries
>>> tier.size()
0
close()[source]

Close the persistent SQLite connection and release database resources.

Should be called when the cache tier is no longer needed to properly release database resources and file locks. Safe to call multiple times.

Side Effects:

  • Acquires connection lock to ensure thread safety.

  • Closes active SQLite connection if present.

  • Sets internal connection to None to prevent reuse.

  • Suppresses any exceptions during close to ensure cleanup completes.

Thread Safety:

Thread-safe via internal lock. Multiple threads can safely call this method concurrently.

Example

>>> tier = DiskCacheTier(config, TierType.L2_FILE, cache_dir)
>>> # ... use tier ...
>>> tier.close()  # Clean up resources
>>> tier.close()  # Safe to call again
Return type:

None

delete(key)[source]

Remove entry from SQLite database by cache key.

Uses SQL DELETE with primary key lookup for O(log n) performance.

Parameters:

key (str) – Cache key to delete.

Returns:

True if entry existed and was deleted, False if not found or on error.

Return type:

bool

Side Effects:

  • Acquires persistent connection via thread-safe _get_connection().

  • Executes DELETE query with parameterized key (SQL injection safe).

  • Commits transaction before returning.

  • Does NOT reset failure counter (only success paths do).

Thread Safety:

Thread-safe via connection lock in _get_connection().

Raises:

Exception – Caught and handled via handle_failure(), returns False.

Parameters:

key (str)

Return type:

bool

get(key)[source]

Retrieve entry from SQLite database with O(log n) indexed lookup.

Parameters:

key (str) – Cache key to lookup

Returns:

CacheEntry if found and valid, None otherwise

Return type:

proxywhirl.cache.models.CacheEntry | None

keys()[source]

Return list of all cache keys from SQLite L2 database.

Retrieves all cache keys without loading full entry data. Useful for cache inspection, debugging, and bulk operations.

Returns:

List of all cache keys in database, empty list on error.

Return type:

list[str]

Side Effects:

  • Acquires persistent connection via thread-safe _get_connection().

  • Executes SELECT key query (fetches only key column, not full rows).

  • Loads all keys into memory as a list.

  • Resets failure counter on success.

Thread Safety:

Thread-safe via connection lock in _get_connection().

Performance:

O(n) where n is number of entries. For large caches (>10K entries), consider using size() to check count before calling.

Raises:

Exception – Caught and handled via handle_failure(), returns [].

Return type:

list[str]

Warning

For very large caches (>100K entries), this may consume significant memory. Consider pagination or streaming approaches if needed.

migrate_from_jsonl(jsonl_dir=None)[source]

Migrate existing JSONL shard files to SQLite L2 cache.

This method provides a migration path from the old JSONL-based L2 cache to the new SQLite-based implementation. It reads all shard_*.jsonl files from the specified directory and imports them into the SQLite database.

Parameters:

jsonl_dir (pathlib.Path | None) – Directory containing shard_*.jsonl files. Defaults to self.cache_dir if not specified.

Returns:

Number of entries successfully migrated

Return type:

int

Example

>>> tier = DiskCacheTier(config, TierType.L2_FILE, cache_dir)
>>> migrated = tier.migrate_from_jsonl()
>>> print(f"Migrated {migrated} entries from JSONL to SQLite")
put(key, entry)[source]

Store entry in SQLite database with INSERT OR REPLACE.

Parameters:
Returns:

True if stored successfully, False otherwise

Return type:

bool

size()[source]

Return current number of entries in SQLite L2 cache database.

Uses SQL COUNT(*) for efficient O(1) size calculation via table metadata.

Returns:

Number of cache entries currently stored, 0 on error.

Return type:

int

Side Effects:

  • Acquires persistent connection via thread-safe _get_connection().

  • Executes SELECT COUNT(*) query (reads table metadata, not rows).

  • Resets failure counter on success.

Thread Safety:

Thread-safe via connection lock in _get_connection().

Performance:

O(1) - SQLite maintains row count in table metadata.

Raises:

Exception – Caught and handled via handle_failure(), returns 0.

Return type:

int

class proxywhirl.cache.tiers.JsonlCacheTier(config, tier_type, cache_dir, encryptor=None, num_shards=16)[source]

Bases: CacheTier

L2 JSONL file-based cache with sharding and encryption.

Provides persistent caching using JSONL (JSON Lines) files with: - Sharded storage for better I/O performance - File locking for concurrent access safety - Encrypted credentials at rest - Simple text format for debugging and portability

Best suited for: - Smaller cache sizes (<10K entries) - Environments where SQLite is unavailable - Cases requiring human-readable cache files - Simple deployment without database dependencies

For larger caches (>10K entries), consider DiskCacheTier (SQLite-based) which provides O(log n) lookups vs O(n) for JSONL.

Initialize JSONL-based L2 cache.

Parameters:
cleanup_expired()[source]

Remove all expired entries from all shards.

Returns:

Number of entries removed

Return type:

int

clear()[source]

Clear all JSONL shard files.

Returns:

Number of entries cleared

Return type:

int

delete(key)[source]

Remove entry from JSONL shard.

Parameters:

key (str) – Cache key to delete

Returns:

True if entry existed and was deleted, False if not found

Return type:

bool

get(key)[source]

Retrieve entry from JSONL shard.

Parameters:

key (str) – Cache key to lookup

Returns:

CacheEntry if found and valid, None otherwise

Return type:

proxywhirl.cache.models.CacheEntry | None

keys()[source]

Return list of all keys.

Returns:

List of cache keys

Return type:

list[str]

put(key, entry)[source]

Store entry in JSONL shard with encrypted credentials.

Parameters:
Returns:

True if stored successfully, False otherwise

Return type:

bool

size()[source]

Return current number of entries.

Returns:

Number of entries in index

Return type:

int

class proxywhirl.cache.tiers.MemoryCacheTier(config, tier_type, on_evict=None)[source]

Bases: CacheTier

L1 in-memory cache using OrderedDict for LRU tracking.

Provides O(1) lookups with automatic LRU eviction when max_entries exceeded.

Initialize memory cache with LRU tracking.

Parameters:
cleanup_expired()[source]

Remove all expired entries from memory cache in bulk.

Returns:

Number of expired entries that were removed.

Return type:

int

Side Effects:

Deletes all entries where is_expired property is True.

clear()[source]

Clear all entries from memory cache.

Returns:

Number of entries that were removed.

Return type:

int

Side Effects:

Empties the OrderedDict, releasing all cached entries.

delete(key)[source]

Remove entry from memory cache by key.

Parameters:

key (str) – Cache key to delete.

Returns:

True if entry existed and was deleted, False if not found.

Return type:

bool

Side Effects:

Removes entry from OrderedDict if present.

get(key)[source]

Retrieve entry from memory cache, updating LRU order.

Parameters:

key (str) – Cache key to lookup.

Returns:

CacheEntry if found, None otherwise. Updates LRU order on hit by moving the accessed entry to the end of the OrderedDict.

Return type:

proxywhirl.cache.models.CacheEntry | None

Side Effects:

Moves accessed entry to end of LRU queue (most recently used position).

keys()[source]

Return list of all cache keys in memory tier.

Returns:

List of cache keys in LRU order (oldest to newest).

Return type:

list[str]

put(key, entry)[source]

Store entry in memory cache with automatic LRU eviction.

Parameters:
Returns:

True if stored successfully, False on error.

Return type:

bool

Side Effects:

  • Removes existing entry if key already exists (update operation).

  • Evicts least recently used entry if max_entries exceeded.

  • Calls on_evict callback if provided when eviction occurs.

  • Resets failure counter on success.

Raises:

Exception – Caught and handled via handle_failure(), returns False.

Parameters:
Return type:

bool

size()[source]

Return current number of entries in memory cache.

Returns:

Count of entries currently stored in the OrderedDict.

Return type:

int

class proxywhirl.cache.tiers.SQLiteCacheTier(config, tier_type, db_path, encryptor=None)[source]

Bases: CacheTier

L3 SQLite database cache with encrypted credentials.

Provides durable persistence with SQL indexing for fast lookups.

Initialize SQLite-based L3 cache with health monitoring.

Parameters:
Side Effects:

  • Creates parent directories for database if they don’t exist.

  • Initializes database schema with cache_entries and health_history tables.

  • Creates indexes for expires_at, source, health_status, and last_accessed.

cleanup_expired()[source]

Remove all expired entries in bulk using SQL DELETE.

This is significantly more efficient than iterating through all entries, reducing cleanup from O(n) to O(1) for expired entries.

Returns:

Number of entries removed

Return type:

int

clear()[source]

Clear all entries from SQLite database.

Returns:

Number of entries that were deleted.

Return type:

int

Side Effects:

  • Opens new SQLite connection for operation.

  • Deletes all rows from cache_entries table.

  • Cascades to delete all health_history records via FOREIGN KEY.

  • Commits transaction before returning.

  • Resets failure counter on success.

Raises:

Exception – Caught and handled via handle_failure(), returns 0.

Return type:

int

delete(key)[source]

Remove entry from SQLite database by key.

Parameters:

key (str) – Cache key to delete.

Returns:

True if entry existed and was deleted, False if not found.

Return type:

bool

Side Effects:

  • Opens new SQLite connection for deletion.

  • Cascades to delete related health_history records via FOREIGN KEY.

  • Commits transaction before returning.

Raises:

Exception – Caught and handled via handle_failure(), returns False.

Parameters:

key (str)

Return type:

bool

get(key)[source]

Retrieve entry from SQLite database with decrypted credentials.

Parameters:

key (str) – Cache key to lookup.

Returns:

CacheEntry if found with decrypted username/password, None otherwise.

Return type:

proxywhirl.cache.models.CacheEntry | None

Side Effects:

  • Opens new SQLite connection for query.

  • Decrypts username_encrypted and password_encrypted BLOBs.

  • Converts UNIX timestamps to datetime objects.

  • Resets failure counter on success.

Raises:

Exception – Caught and handled via handle_failure(), returns None.

Parameters:

key (str)

Return type:

proxywhirl.cache.models.CacheEntry | None

keys()[source]

Return all cache keys from SQLite database.

Returns:

List of all cache keys, empty list on error.

Return type:

list[str]

Side Effects:

  • Opens new SQLite connection for query.

  • Resets failure counter on success.

Raises:

Exception – Caught and handled via handle_failure(), returns [].

Return type:

list[str]

put(key, entry)[source]

Store entry in SQLite database with encrypted credentials.

Parameters:
Returns:

True if stored successfully, False on error.

Return type:

bool

Side Effects:

  • Opens new SQLite connection for write.

  • Encrypts username and password fields as BLOBs.

  • Uses INSERT OR REPLACE (upsert) to handle updates.

  • Sets created_at and updated_at to current timestamp.

  • Commits transaction before returning.

  • Resets failure counter on success.

Raises:

Exception – Caught and handled via handle_failure(), returns False.

Parameters:
Return type:

bool

size()[source]

Return current number of entries in SQLite database.

Returns:

Count of entries in cache_entries table, 0 on error.

Return type:

int

Side Effects:

  • Opens new SQLite connection for query.

  • Resets failure counter on success.

Raises:

Exception – Caught and handled via handle_failure(), returns 0.

Return type:

int

class proxywhirl.cache.tiers.TierType[source]

Bases: str, enum.Enum

Cache tier types.

Initialize self. See help(type(self)) for accurate signature.