Class iMAMLOptions<T, TInput, TOutput>

Namespace

AiDotNet.MetaLearning.Options

Assembly

AiDotNet.dll

Configuration options for iMAML (Implicit Model-Agnostic Meta-Learning) algorithm.

public class iMAMLOptions<T, TInput, TOutput> : IMetaLearnerOptions<T>

Type Parameters

T

The numeric data type used for calculations (e.g., float, double).

TInput

The input data type (e.g., Matrix<T>, Tensor<T>).

TOutput

The output data type (e.g., Vector<T>, Tensor<T>).

Inheritance

object

iMAMLOptions<T, TInput, TOutput>

Implements

IMetaLearnerOptions<T>

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

iMAML extends MAML by using implicit differentiation to compute meta-gradients, which allows for many more adaptation steps without increased memory usage.

For Beginners: iMAML is a more memory-efficient version of MAML. Regular MAML needs to remember every adaptation step (expensive!). iMAML uses a mathematical trick to get the same result with constant memory.

Key parameters specific to iMAML: - LambdaRegularization: Controls stability of implicit gradient computation - ConjugateGradientIterations: How many CG steps to solve the implicit equation - ConjugateGradientTolerance: When to stop CG early if converged

Constructors

iMAMLOptions(IFullModel<T, TInput, TOutput>)

Initializes a new instance of the iMAMLOptions class with the required meta-model.

public iMAMLOptions(IFullModel<T, TInput, TOutput> metaModel)

Parameters

metaModel IFullModel<T, TInput, TOutput>

The meta-model to be trained (required).

Exceptions

ArgumentNullException

Thrown when metaModel is null.

Properties

AdaptationSteps

Gets or sets the number of gradient steps to take during inner loop adaptation.

public int AdaptationSteps { get; set; }

Property Value

int

Default: 10 (iMAML can use more steps than MAML due to constant memory cost).

Remarks

One of iMAML's key advantages: because memory usage is constant regardless of adaptation steps, you can use 10, 20, or even more steps without running out of memory. More steps often lead to better task adaptation.

For Beginners: Unlike MAML where more steps = more memory, iMAML can use many steps freely. Try 10-20 steps for better performance.

CheckpointFrequency

Gets or sets how often to save checkpoints.

public int CheckpointFrequency { get; set; }

Property Value

int

Default: 500.

ConjugateGradientIterations

Gets or sets the maximum number of Conjugate Gradient iterations.

public int ConjugateGradientIterations { get; set; }

Property Value

int

Default: 10 (usually sufficient for convergence).

Remarks

The Conjugate Gradient (CG) method is used to solve the linear system that arises in implicit differentiation. More iterations give more accurate solutions but take longer.

For Beginners: This is how many steps to take when solving the implicit equation. 10-20 is usually enough.

ConjugateGradientTolerance

Gets or sets the convergence tolerance for Conjugate Gradient.

public double ConjugateGradientTolerance { get; set; }

Property Value

double

Default: 1e-10 (very precise convergence).

Remarks

CG stops early if the residual falls below this threshold. Lower values are more precise but may require more iterations.

DataLoader

Gets or sets the episodic data loader for sampling tasks.

public IEpisodicDataLoader<T, TInput, TOutput>? DataLoader { get; set; }

Property Value

IEpisodicDataLoader<T, TInput, TOutput>

Default: null (tasks must be provided manually to MetaTrain).

EnableCheckpointing

Gets or sets whether to save checkpoints during training.

public bool EnableCheckpointing { get; set; }

Property Value

bool

Default: false.

EvaluationFrequency

Gets or sets how often to evaluate during meta-training.

public int EvaluationFrequency { get; set; }

Property Value

int

Default: 100 (evaluate every 100 iterations).

EvaluationTasks

Gets or sets the number of tasks to use for evaluation.

public int EvaluationTasks { get; set; }

Property Value

int

Default: 100.

GradientClipThreshold

Gets or sets the maximum gradient norm for gradient clipping.

public double? GradientClipThreshold { get; set; }

Property Value

double?

Default: 10.0 (prevents exploding gradients).

InnerLearningRate

Gets or sets the learning rate for the inner loop (task adaptation).

public double InnerLearningRate { get; set; }

Property Value

double

Default: 0.01 (standard for meta-learning inner loops).

Remarks

In iMAML, the inner learning rate controls how quickly the model adapts during the K gradient steps. Unlike MAML, iMAML can use many more steps without memory penalty, so this rate can be tuned more conservatively.

InnerOptimizer

Gets or sets the optimizer for inner-loop adaptation.

public IGradientBasedOptimizer<T, TInput, TOutput>? InnerOptimizer { get; set; }

Property Value

IGradientBasedOptimizer<T, TInput, TOutput>

Default: null (uses built-in SGD optimizer with InnerLearningRate).

LambdaRegularization

Gets or sets the regularization strength for implicit gradient computation.

public double LambdaRegularization { get; set; }

Property Value

double

Default: 1.0 (balances stability and accuracy).

Remarks

This parameter (often called lambda) controls the regularization in the implicit equation: (I + lambda * H)^(-1) * g, where H is the Hessian.

For Beginners: This controls how "smooth" the implicit gradient is: - Higher values (e.g., 2.0, 5.0): More stable but less accurate gradients - Lower values (e.g., 0.5, 0.1): More accurate but potentially unstable - 1.0 is a good default that balances both concerns

LossFunction

Gets or sets the loss function for training.

public ILossFunction<T>? LossFunction { get; set; }

Property Value

ILossFunction<T>

Default: null (uses model's default loss function if available).

MetaBatchSize

Gets or sets the number of tasks to sample per meta-training iteration.

public int MetaBatchSize { get; set; }

Property Value

int

Default: 4 (typical meta-batch size).

MetaModel

Gets or sets the meta-model to be trained. This is the only required property.

public IFullModel<T, TInput, TOutput> MetaModel { get; set; }

Property Value

IFullModel<T, TInput, TOutput>

Remarks

The model must implement IFullModel to support parameter getting/setting and gradient computation required for iMAML's implicit differentiation.

MetaOptimizer

Gets or sets the optimizer for meta-parameter updates (outer loop).

public IGradientBasedOptimizer<T, TInput, TOutput>? MetaOptimizer { get; set; }

Property Value

IGradientBasedOptimizer<T, TInput, TOutput>

Default: null (uses built-in Adam optimizer with OuterLearningRate).

NeumannSeriesTerms

Gets or sets the number of terms in the Neumann series approximation.

public int NeumannSeriesTerms { get; set; }

Property Value

int

Default: 5 (sufficient for most cases).

Remarks

Only used when UseNeumannApproximation is true. More terms give better approximations but take longer to compute.

NumMetaIterations

Gets or sets the total number of meta-training iterations to perform.

public int NumMetaIterations { get; set; }

Property Value

int

Default: 1000 (typical meta-training length).

OuterLearningRate

Gets or sets the learning rate for the outer loop (meta-optimization).

public double OuterLearningRate { get; set; }

Property Value

double

Default: 0.001 (typically 10x smaller than inner rate).

Remarks

Controls how the meta-parameters are updated after computing implicit gradients. The implicit gradient computation in iMAML often provides more accurate gradients than FOMAML, so this rate can sometimes be set slightly higher.

RandomSeed

Gets or sets the random seed for reproducibility.

public int? RandomSeed { get; set; }

Property Value

int?

Default: null (non-deterministic).

UseFirstOrder

Gets or sets whether to use first-order approximation.

public bool UseFirstOrder { get; set; }

Property Value

bool

Default: false (use full implicit differentiation).

Remarks

When true, falls back to a first-order approximation similar to FOMAML. This is faster but loses some of the benefits of implicit differentiation. Only set to true if you're having stability issues or need maximum speed.

UseNeumannApproximation

Gets or sets whether to use the Neumann series approximation for implicit gradients.

public bool UseNeumannApproximation { get; set; }

Property Value

bool

Default: false (use full CG solver).

Remarks

The Neumann series provides a faster but approximate solution to the implicit equation. Set to true for faster training at the cost of some gradient accuracy.

Methods

Clone()

Creates a deep copy of the iMAML options.

public IMetaLearnerOptions<T> Clone()

Returns

IMetaLearnerOptions<T>

A new iMAMLOptions instance with the same configuration values.

IsValid()

Validates that all iMAML configuration options are properly set.

public bool IsValid()

Returns

bool

True if the configuration is valid for iMAML training; otherwise, false.

Remarks

Checks all required hyperparameters including iMAML-specific ones: - Standard meta-learning parameters (learning rates, steps, etc.) - Lambda regularization must be non-negative - CG parameters must be positive