Class CQLOptions<T>

Namespace

AiDotNet.Models.Options

Assembly

AiDotNet.dll

Configuration options for Conservative Q-Learning (CQL) agent.

public class CQLOptions<T>

Type Parameters

T

The numeric type used for calculations.

Inheritance

object

CQLOptions<T>

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

CQL is an offline RL algorithm that learns from fixed datasets without environment interaction. It addresses overestimation by adding a conservative penalty to Q-values.

For Beginners: CQL is designed for learning from logged data without trying new actions. This is useful when you have historical data but can't experiment in the real environment (e.g., medical treatment, autonomous driving).

Key innovation:

• Conservative Q-Learning: Penalizes Q-values for unseen actions to prevent overoptimistic estimates
• Offline Learning: No environment interaction during training

Think of it like learning to drive from dashcam footage - you can't try new maneuvers, so you need to be conservative about what you haven't seen.

Based on SAC architecture with conservative regularization.

Constructors

CQLOptions()

public CQLOptions()

Properties

ActionSize

public int ActionSize { get; set; }

Property Value

int

AlphaLearningRate

public T AlphaLearningRate { get; set; }

Property Value

T

AutoTuneTemperature

public bool AutoTuneTemperature { get; set; }

Property Value

bool

BatchSize

public int BatchSize { get; set; }

Property Value

int

BufferSize

public int BufferSize { get; set; }

Property Value

int

CQLAlpha

public T CQLAlpha { get; set; }

Property Value

T

CQLLagrange

public bool CQLLagrange { get; set; }

Property Value

bool

CQLNumActions

public int CQLNumActions { get; set; }

Property Value

int

CQLTargetActionGap

public T CQLTargetActionGap { get; set; }

Property Value

T

DiscountFactor

public T DiscountFactor { get; set; }

Property Value

T

GradientSteps

public int GradientSteps { get; set; }

Property Value

int

InitialTemperature

public T InitialTemperature { get; set; }

Property Value

T

PolicyHiddenLayers

public List<int> PolicyHiddenLayers { get; set; }

Property Value

List<int>

PolicyLearningRate

public T PolicyLearningRate { get; set; }

Property Value

T

QHiddenLayers

public List<int> QHiddenLayers { get; set; }

Property Value

List<int>

QLearningRate

public T QLearningRate { get; set; }

Property Value

T

QLossFunction

public ILossFunction<T> QLossFunction { get; set; }

Property Value

ILossFunction<T>

Seed

public int? Seed { get; set; }

Property Value

int?

StateSize

public int StateSize { get; set; }

Property Value

int

TargetEntropy

public T? TargetEntropy { get; set; }

Property Value

T

TargetUpdateTau

public T TargetUpdateTau { get; set; }

Property Value

T