Enum MetaSGDLearningRateInitialization

Namespace

AiDotNet.MetaLearning.Options

Assembly

AiDotNet.dll

Learning rate initialization strategies for Meta-SGD.

public enum MetaSGDLearningRateInitialization

Fields

LayerBased = 3

Initialize learning rates based on layer depth. Deeper layers get smaller learning rates.

MagnitudeBased = 2

Initialize learning rates based on parameter magnitudes. Larger parameters get smaller learning rates.

Random = 1

Initialize learning rates uniformly at random within LearningRateInitRange.

Uniform = 0

Initialize all learning rates to the same value (InnerLearningRate).

Xavier = 4

Initialize learning rates using Xavier-style initialization. Based on fan-in and fan-out of each layer.

Remarks

Different initialization strategies can affect how quickly Meta-SGD converges and the quality of the learned per-parameter optimizers.