Interface IGpuOptimizerConfig

Namespace

AiDotNet.Interfaces

Assembly

AiDotNet.dll

Configuration for GPU-resident optimizer updates.

public interface IGpuOptimizerConfig

Remarks

This interface allows layers to receive optimizer-specific configuration for GPU parameter updates. Different optimizer types (SGD, Adam, etc.) have different implementations with their specific hyperparameters.

For Beginners: When training on GPU, the weights need to be updated using an optimizer (like SGD or Adam). This configuration tells the GPU exactly how to update the weights - with what learning rate, momentum, etc.

Properties

LearningRate

Gets the learning rate for parameter updates.

float LearningRate { get; }

Property Value

float

OptimizerType

Gets the type of optimizer (SGD, Adam, AdamW, etc.).

GpuOptimizerType OptimizerType { get; }

Property Value

GpuOptimizerType

Step

Gets the current optimization step (used for bias correction in Adam-family optimizers).

int Step { get; }

Property Value

int

WeightDecay

Gets the weight decay (L2 regularization) coefficient.

float WeightDecay { get; }

Property Value

float

Methods

ApplyUpdate(IDirectGpuBackend, IGpuBuffer, IGpuBuffer, GpuOptimizerState, int)

Applies the optimizer update to the given parameter buffer using its gradient.

void ApplyUpdate(IDirectGpuBackend backend, IGpuBuffer param, IGpuBuffer gradient, GpuOptimizerState state, int size)

Parameters

backend IDirectGpuBackend

The GPU backend to execute the update.

param IGpuBuffer

Buffer containing the parameters to update (modified in-place).

gradient IGpuBuffer

Buffer containing the gradients.

state GpuOptimizerState

Optimizer state buffers (momentum, squared gradients, etc.).

size int

Number of parameters to update.

Remarks

For Beginners: This method applies the optimizer's update rule directly on the GPU. Each optimizer type (SGD, Adam, etc.) implements its own update logic using GPU kernels. The state parameter contains any auxiliary buffers needed (like velocity for SGD with momentum, or m/v buffers for Adam).

Design Note: Following the Open/Closed Principle, each optimizer config knows how to apply its own update, so adding new optimizers doesn't require modifying layer code.