Class InitializationStrategyBase<T>

Namespace

AiDotNet.Initialization

Assembly

AiDotNet.dll

Base class for initialization strategies providing common functionality.

public abstract class InitializationStrategyBase<T> : IInitializationStrategy<T>

Type Parameters

T

The numeric type used for calculations.

Inheritance

object

InitializationStrategyBase<T>

Implements

IInitializationStrategy<T>

Derived

EagerInitializationStrategy<T>

FromFileInitializationStrategy<T>

LazyInitializationStrategy<T>

ZeroInitializationStrategy<T>

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

This abstract base class provides shared implementation for all initialization strategies, including common helper methods for weight initialization patterns like Xavier/Glorot and He initialization.

For Beginners: This base class contains the shared code that all initialization strategies need, avoiding duplication and ensuring consistent behavior across different initialization methods.

Constructors

InitializationStrategyBase()

Initializes a new instance of the InitializationStrategyBase<T> class.

protected InitializationStrategyBase()

Fields

NumOps

The numeric operations helper for type T.

protected readonly INumericOperations<T> NumOps

Field Value

INumericOperations<T>

Random

Thread-safe random number generator.

protected readonly Random Random

Field Value

Random

Properties

IsLazy

Gets a value indicating whether this strategy defers initialization until first use.

public abstract bool IsLazy { get; }

Property Value

bool

true if initialization is deferred until first Forward() call; false if initialization happens immediately.

LoadFromExternal

Gets a value indicating whether weights should be loaded from an external source.

public abstract bool LoadFromExternal { get; }

Property Value

bool

true if weights should be loaded from file or other external source; false if weights should be randomly initialized.

Methods

HeNormalInitialize(Tensor<T>, int)

Initializes weights using He/Kaiming normal initialization.

protected void HeNormalInitialize(Tensor<T> weights, int fanIn)

Parameters

weights Tensor<T>

The weights tensor to initialize.

fanIn int

The number of input units (fan-in).

Remarks

He normal initialization samples from a normal distribution with variance 2/fan_in.

Formula: W ~ N(0, sqrt(2/fan_in))

HeUniformInitialize(Tensor<T>, int)

Initializes weights using He/Kaiming uniform initialization.

protected void HeUniformInitialize(Tensor<T> weights, int fanIn)

Parameters

weights Tensor<T>

The weights tensor to initialize.

fanIn int

The number of input units (fan-in).

Remarks

He initialization is designed for ReLU and its variants. It accounts for the fact that ReLU zeros out half of the values, requiring larger initial weights.

Formula: W ~ U(-sqrt(6/fan_in), sqrt(6/fan_in))

InitializeBiases(Tensor<T>)

Initializes the biases tensor with appropriate values.

public abstract void InitializeBiases(Tensor<T> biases)

Parameters

biases Tensor<T>

The biases tensor to initialize.

InitializeWeights(Tensor<T>, int, int)

Initializes the weights tensor with appropriate values.

public abstract void InitializeWeights(Tensor<T> weights, int inputSize, int outputSize)

Parameters

weights Tensor<T>

The weights tensor to initialize.

inputSize int

The number of input features.

outputSize int

The number of output features.

SampleGaussian(double, double)

Samples a value from a Gaussian (normal) distribution using the Box-Muller transform.

protected double SampleGaussian(double mean, double stddev)

Parameters

mean double

The mean of the distribution.

stddev double

The standard deviation of the distribution.

Returns

double

A sample from the specified Gaussian distribution.

XavierNormalInitialize(Tensor<T>, int, int)

Initializes weights using Xavier/Glorot normal initialization.

protected void XavierNormalInitialize(Tensor<T> weights, int fanIn, int fanOut)

Parameters

weights Tensor<T>

The weights tensor to initialize.

fanIn int

The number of input units (fan-in).

fanOut int

The number of output units (fan-out).

Remarks

Similar to Xavier uniform but samples from a normal distribution instead.

Formula: W ~ N(0, sqrt(2/(fan_in + fan_out)))

XavierUniformInitialize(Tensor<T>, int, int)

Initializes weights using Xavier/Glorot uniform initialization.

protected void XavierUniformInitialize(Tensor<T> weights, int fanIn, int fanOut)

Parameters

weights Tensor<T>

The weights tensor to initialize.

fanIn int

The number of input units (fan-in).

fanOut int

The number of output units (fan-out).

Remarks

Xavier initialization is designed to keep the variance of activations roughly the same across layers during forward propagation. It works well with sigmoid and tanh activations.

Formula: W ~ U(-sqrt(6/(fan_in + fan_out)), sqrt(6/(fan_in + fan_out)))

ZeroInitializeBiases(Tensor<T>)

Initializes biases to zero (common default).

protected void ZeroInitializeBiases(Tensor<T> biases)

Parameters

biases Tensor<T>

The biases tensor to initialize.