Class OrnsteinUhlenbeckNoise<T>

Namespace

AiDotNet.ReinforcementLearning.Policies.Exploration

Assembly

AiDotNet.dll

Ornstein-Uhlenbeck process noise for temporally correlated exploration. Commonly used in DDPG and other continuous control algorithms. Process equation: dx = θ(μ - x)dt + σdW

public class OrnsteinUhlenbeckNoise<T> : ExplorationStrategyBase<T>, IExplorationStrategy<T>

Type Parameters

T

The numeric type used for calculations.

Inheritance

object

ExplorationStrategyBase<T>

OrnsteinUhlenbeckNoise<T>

Implements

IExplorationStrategy<T>

Inherited Members

ExplorationStrategyBase<T>.NumOps

ExplorationStrategyBase<T>.BoxMullerSample(Random)

ExplorationStrategyBase<T>.ClampAction(Vector<T>, double, double)

ExplorationStrategyBase<T>.ValidateActionSize(int, int, string)

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Constructors

OrnsteinUhlenbeckNoise(int, double, double, double, double)

Initializes a new instance of the Ornstein-Uhlenbeck noise exploration.

public OrnsteinUhlenbeckNoise(int actionSize, double theta = 0.15, double sigma = 0.2, double mu = 0, double dt = 0.01)

Parameters

actionSize int

The size of the action space.

theta double

Mean reversion rate (default: 0.15).

sigma double

Volatility/noise scale (default: 0.2).

mu double

Long-term mean (default: 0.0).

dt double

Time step (default: 0.01).

Methods

GetExplorationAction(Vector<T>, Vector<T>, int, Random)

Applies Ornstein-Uhlenbeck noise to the policy action.

public override Vector<T> GetExplorationAction(Vector<T> state, Vector<T> policyAction, int actionSpaceSize, Random random)

Parameters

state Vector<T>

policyAction Vector<T>

actionSpaceSize int

random Random

Returns

Vector<T>

Reset()

Resets the noise state to zero.

public override void Reset()

Update()

Updates internal parameters (no-op for OU noise as it self-regulates).

public override void Update()