Class StepLRScheduler

Namespace

AiDotNet.LearningRateSchedulers

Assembly

AiDotNet.dll

Decays the learning rate by a factor (gamma) every specified number of steps.

public class StepLRScheduler : LearningRateSchedulerBase, ILearningRateScheduler

Inheritance

object

LearningRateSchedulerBase

StepLRScheduler

Implements

ILearningRateScheduler

Inherited Members

LearningRateSchedulerBase._baseLearningRate

LearningRateSchedulerBase._currentLearningRate

LearningRateSchedulerBase._currentStep

LearningRateSchedulerBase._minLearningRate

LearningRateSchedulerBase.CurrentLearningRate

LearningRateSchedulerBase.BaseLearningRate

LearningRateSchedulerBase.CurrentStep

LearningRateSchedulerBase.Step()

LearningRateSchedulerBase.GetLearningRateAtStep(int)

LearningRateSchedulerBase.Reset()

LearningRateSchedulerBase.LoadState(Dictionary<string, object>)

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Examples

// Reduce LR by 10x every 30 epochs
var scheduler = new StepLRScheduler(
    baseLearningRate: 0.1,
    stepSize: 30,
    gamma: 0.1
);

for (int epoch = 0; epoch < 100; epoch++)
{
    Train(model, scheduler.CurrentLearningRate);
    scheduler.Step();
}

Remarks

StepLR is one of the simplest and most commonly used learning rate schedulers. It multiplies the learning rate by gamma every step_size epochs/steps.

For Beginners: This scheduler reduces the learning rate by a fixed amount at regular intervals. For example, you might reduce the learning rate by 10x every 30 epochs. This is like slowing down periodically as you get closer to your destination, making your adjustments more precise as training progresses.

Formula: lr = base_lr * gamma^(floor(step / step_size))

Constructors

StepLRScheduler(double, int, double, double)

Initializes a new instance of the StepLRScheduler class.

public StepLRScheduler(double baseLearningRate, int stepSize, double gamma = 0.1, double minLearningRate = 0)

Parameters

baseLearningRate double

The initial learning rate.

stepSize int

Period of learning rate decay (number of steps between each decay).

gamma double

Multiplicative factor of learning rate decay. Default: 0.1

minLearningRate double

Minimum learning rate floor. Default: 0

Exceptions

ArgumentException

Thrown when stepSize is not positive or gamma is not in (0, 1].

Properties

Gamma

Gets the multiplicative factor of learning rate decay.

public double Gamma { get; }

Property Value

double

StepSize

Gets the step size (period of learning rate decay).

public int StepSize { get; }

Property Value

int

Methods

ComputeLearningRate(int)

Computes the learning rate for a given step.

protected override double ComputeLearningRate(int step)

Parameters

step int

The step number.

Returns

double

The computed learning rate.

GetState()

Gets the scheduler state for serialization/checkpointing.

public override Dictionary<string, object> GetState()

Returns

Dictionary<string, object>

A dictionary containing the scheduler state.