Enum SpeculationPolicy

Namespace

AiDotNet.Configuration

Assembly

AiDotNet.dll

Policies for enabling/disabling speculative decoding at runtime.

public enum SpeculationPolicy

Fields

Auto = 0

Automatically decide based on runtime conditions (recommended).

ForceOff = 2

Always disable speculative decoding even if enabled in config.

ForceOn = 1

Always enable speculative decoding when configured.

LatencyFirst = 3

Prefer speculative decoding to reduce latency, even under moderate load.

ThroughputFirst = 4

Prefer throughput and stability: use speculative decoding only when conditions are ideal.