Class CacheConfig

Namespace

AiDotNet.Deployment.Configuration

Assembly

AiDotNet.dll

Configuration for model caching - storing loaded models in memory to avoid repeated loading.

public class CacheConfig

Inheritance

object

CacheConfig

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

For Beginners: Loading an AI model from disk takes time. Caching keeps recently-used models in memory so they can be used again instantly, like keeping your frequently-used apps open on your phone instead of closing and reopening them.

Benefits:

• Much faster inference (no model loading time)
• Better throughput when serving multiple requests
• Reduces disk I/O

Considerations:

• Uses memory (RAM) to store models
• Limited cache size - old models get evicted when full

Eviction Policies (what to remove when cache is full):

• LRU (Least Recently Used): Removes models you haven't used in a while (recommended)
• LFU (Least Frequently Used): Removes models used least often
• FIFO: Removes oldest models first
• Random: Removes random models (simple but unpredictable)

For most applications, LRU with a moderate max size works well.

Properties

DefaultTTL

Gets or sets the cache entry time-to-live as a TimeSpan (default: 1 hour).

public TimeSpan DefaultTTL { get; set; }

Property Value

TimeSpan

Remarks

For Beginners: A more convenient way to specify the cache duration. This is equivalent to TimeToLiveSeconds but uses the TimeSpan type for clarity.

Enabled

Gets or sets whether caching is enabled (default: true).

public bool Enabled { get; set; }

Property Value

bool

Remarks

For Beginners: Set to true to enable caching, false to disable it entirely. Caching is recommended for production systems to improve performance.

EvictionPolicy

Gets or sets the cache eviction policy (default: LRU).

public CacheEvictionPolicy EvictionPolicy { get; set; }

Property Value

CacheEvictionPolicy

Remarks

For Beginners: Determines which model to remove when cache is full. LRU (Least Recently Used) is recommended - it removes models you haven't used recently.

MaxCacheSize

Gets or sets the maximum number of models to cache (default: 10).

public int MaxCacheSize { get; set; }

Property Value

int

Remarks

For Beginners: How many models to keep in memory simultaneously. Higher values use more memory but reduce cache misses. 10 is a good default for most cases.

MaxSizeMB

Gets or sets the maximum cache size in megabytes (default: 100.0 MB).

public double MaxSizeMB { get; set; }

Property Value

double

Remarks

For Beginners: Memory-based limit for the cache. When total cached model size exceeds this, older models are evicted. Use in addition to MaxCacheSize for more precise memory control.

PreloadModels

Gets or sets whether to preload models on startup (default: false).

public bool PreloadModels { get; set; }

Property Value

bool

Remarks

For Beginners: If true, frequently-used models are loaded into cache at startup. This eliminates first-request latency but increases startup time. Use for production servers.

TimeToLiveSeconds

Gets or sets the cache entry time-to-live in seconds (default: 3600 = 1 hour).

public int TimeToLiveSeconds { get; set; }

Property Value

int

Remarks

For Beginners: How long unused models stay in cache before removal. Default is 1 hour. Set to 0 to disable TTL (models only removed when cache is full).

TrackStatistics

Gets or sets whether to track cache hit/miss statistics (default: true).

public bool TrackStatistics { get; set; }

Property Value

bool

Remarks

For Beginners: Tracks how often models are found in cache (hits) vs loaded from disk (misses). Useful for monitoring and optimization but has tiny performance overhead. Recommended.