Class DecisionTransformerOptions<T>

Namespace

AiDotNet.Models.Options

Assembly

AiDotNet.dll

Configuration options for Decision Transformer agents.

public class DecisionTransformerOptions<T> : ReinforcementLearningOptions<T>

Type Parameters

T

The numeric type used for calculations.

Inheritance

object

ReinforcementLearningOptions<T>

DecisionTransformerOptions<T>

Inherited Members

ReinforcementLearningOptions<T>.LearningRate

ReinforcementLearningOptions<T>.DiscountFactor

ReinforcementLearningOptions<T>.LossFunction

ReinforcementLearningOptions<T>.Seed

ReinforcementLearningOptions<T>.BatchSize

ReinforcementLearningOptions<T>.ReplayBufferSize

ReinforcementLearningOptions<T>.TargetUpdateFrequency

ReinforcementLearningOptions<T>.UsePrioritizedReplay

ReinforcementLearningOptions<T>.EpsilonStart

ReinforcementLearningOptions<T>.EpsilonEnd

ReinforcementLearningOptions<T>.EpsilonDecay

ReinforcementLearningOptions<T>.WarmupSteps

ReinforcementLearningOptions<T>.MaxGradientNorm

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

Decision Transformer treats RL as sequence modeling, using transformer architecture to model trajectories conditioned on desired returns.

For Beginners: Decision Transformer is a radically different approach to RL. Instead of learning "what action is best", it learns "what action was taken when the outcome was X". Then at test time, you tell it "I want outcome X" and it generates actions.

Key innovation:

• Sequence Modeling: Uses transformers (like GPT) instead of RL algorithms
• Return Conditioning: Specify desired return, get action sequence
• Offline-Friendly: Works excellently with fixed datasets
• No Value Functions: No Q-networks or critics needed

Think of it like: "Show me examples of successful chess games, and I'll learn to play moves that lead to success."

Famous for: Berkeley/Meta research showing transformers can replace RL algorithms

Properties

ActionSize

public int ActionSize { get; init; }

Property Value

int

BufferSize

public int BufferSize { get; init; }

Property Value

int

ContextLength

public int ContextLength { get; init; }

Property Value

int

DropoutRate

public double DropoutRate { get; init; }

Property Value

double

EmbeddingDim

public int EmbeddingDim { get; init; }

Property Value

int

NumHeads

public int NumHeads { get; init; }

Property Value

int

NumLayers

public int NumLayers { get; init; }

Property Value

int

Optimizer

The optimizer used for updating network parameters. If null, Adam optimizer will be used by default.

public IOptimizer<T, Vector<T>, Vector<T>>? Optimizer { get; init; }

Property Value

IOptimizer<T, Vector<T>, Vector<T>>

StateSize

public int StateSize { get; init; }

Property Value

int