Class SequentialLRScheduler

Namespace

AiDotNet.LearningRateSchedulers

Assembly

AiDotNet.dll

Chains multiple learning rate schedulers together in sequence.

public class SequentialLRScheduler : LearningRateSchedulerBase, ILearningRateScheduler

Inheritance

object

LearningRateSchedulerBase

SequentialLRScheduler

Implements

ILearningRateScheduler

Inherited Members

LearningRateSchedulerBase._baseLearningRate

LearningRateSchedulerBase._currentLearningRate

LearningRateSchedulerBase._currentStep

LearningRateSchedulerBase._minLearningRate

LearningRateSchedulerBase.CurrentLearningRate

LearningRateSchedulerBase.BaseLearningRate

LearningRateSchedulerBase.CurrentStep

LearningRateSchedulerBase.GetLearningRateAtStep(int)

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Examples

// Warmup for 1000 steps, then cosine annealing for 9000 steps
var schedulers = new List<ILearningRateScheduler>
{
    new LinearWarmupScheduler(0.001, 1000, 1000),
    new CosineAnnealingLRScheduler(0.001, 9000)
};
var milestones = new[] { 1000 };  // Switch after step 1000
var scheduler = new SequentialLRScheduler(schedulers, milestones);

Remarks

SequentialLR allows you to compose multiple schedulers, each running for a specified number of steps. This is useful for complex training schedules that combine different strategies at different phases of training.

For Beginners: Sometimes you want different learning rate strategies at different points in training. For example, you might want linear warmup for the first 1000 steps, then cosine annealing for the next 9000 steps. This scheduler lets you chain multiple schedulers together, specifying when to switch from one to the next.

Constructors

SequentialLRScheduler(IList<ILearningRateScheduler>, int[])

Initializes a new instance of the SequentialLRScheduler class.

public SequentialLRScheduler(IList<ILearningRateScheduler> schedulers, int[] milestones)

Parameters

schedulers IList<ILearningRateScheduler>

List of schedulers to chain together.

milestones int[]

Steps at which to switch to the next scheduler.

Exceptions

ArgumentException

Thrown when parameters are invalid.

Properties

CurrentScheduler

Gets the current active scheduler.

public ILearningRateScheduler CurrentScheduler { get; }

Property Value

ILearningRateScheduler

CurrentSchedulerIndex

Gets the current active scheduler index.

public int CurrentSchedulerIndex { get; }

Property Value

int

Methods

ComputeLearningRate(int)

Computes the learning rate for a given step.

protected override double ComputeLearningRate(int step)

Parameters

step int

The step number.

Returns

double

The computed learning rate.

GetState()

Gets the scheduler state for serialization/checkpointing.

public override Dictionary<string, object> GetState()

Returns

Dictionary<string, object>

A dictionary containing the scheduler state.

LoadState(Dictionary<string, object>)

Loads the scheduler state from a checkpoint.

public override void LoadState(Dictionary<string, object> state)

Parameters

state Dictionary<string, object>

The state dictionary to load from.

Reset()

Resets the scheduler to its initial state.

public override void Reset()

Step()

Advances the scheduler by one step and returns the new learning rate.

public override double Step()

Returns

double

The updated learning rate for the next step.