Class MAMLOptions<T, TInput, TOutput>

Namespace

AiDotNet.MetaLearning.Options

Assembly

AiDotNet.dll

Configuration options for MAML (Model-Agnostic Meta-Learning) algorithm.

public class MAMLOptions<T, TInput, TOutput> : IMetaLearnerOptions<T>

Type Parameters

T

The numeric data type used for calculations (e.g., float, double).

TInput

The input data type (e.g., Matrix<T>, Tensor<T>).

TOutput

The output data type (e.g., Vector<T>, Tensor<T>).

Inheritance

object

MAMLOptions<T, TInput, TOutput>

Implements

IMetaLearnerOptions<T>

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

MAML learns an initialization that can be quickly fine-tuned to new tasks. These options control both the inner loop (task adaptation) and outer loop (meta-optimization).

For Beginners: MAML has two learning loops: - Inner loop: Fast adaptation to a specific task (uses InnerLearningRate, AdaptationSteps) - Outer loop: Slow learning of good initialization (uses OuterLearningRate)

Constructors

MAMLOptions(IFullModel<T, TInput, TOutput>)

Initializes a new instance of the MAMLOptions class with the required meta-model.

public MAMLOptions(IFullModel<T, TInput, TOutput> metaModel)

Parameters

metaModel IFullModel<T, TInput, TOutput>

The meta-model to be trained (required).

Properties

AdaptationSteps

Gets or sets the number of gradient steps to take during inner loop adaptation.

public int AdaptationSteps { get; set; }

Property Value

int

The number of adaptation steps. Default is 5.

Remarks

This is the K in "K-shot learning." MAML performs K gradient descent steps on the support set of each task during the inner loop. The original MAML paper uses 1-10 steps. More steps allow better adaptation but increase computation and memory usage.

For Beginners: How many times the model updates itself when learning a new task. 1 step = very fast but rough adaptation; 10 steps = slower but more refined.

CheckpointFrequency

Gets or sets how often (in meta-iterations) to save checkpoints.

public int CheckpointFrequency { get; set; }

Property Value

int

The checkpoint frequency. Default is 500.

Remarks

When checkpointing is enabled, MAML saves the meta-parameters every CheckpointFrequency iterations. More frequent saves use more disk space but reduce potential data loss on interruption.

DataLoader

Gets or sets the episodic data loader for sampling tasks. Default: null (tasks must be provided manually to MetaTrain).

public IEpisodicDataLoader<T, TInput, TOutput>? DataLoader { get; set; }

Property Value

IEpisodicDataLoader<T, TInput, TOutput>

EnableCheckpointing

Gets or sets whether to save model checkpoints during meta-training.

public bool EnableCheckpointing { get; set; }

Property Value

bool

True to enable checkpointing; false to disable. Default is false.

Remarks

When enabled, MAML will save the meta-parameters at regular intervals. This allows resuming training after interruption and keeping the best model.

EvaluationFrequency

Gets or sets how often (in meta-iterations) to evaluate the meta-learner.

public int EvaluationFrequency { get; set; }

Property Value

int

The evaluation frequency. Default is 100.

Remarks

Every EvaluationFrequency meta-iterations, MAML will evaluate on EvaluationTasks tasks to track training progress. More frequent evaluation provides better visibility but slows down training.

EvaluationTasks

Gets or sets the number of tasks to use for evaluation during meta-training.

public int EvaluationTasks { get; set; }

Property Value

int

The number of evaluation tasks. Default is 100.

Remarks

Periodically, MAML evaluates its generalization by adapting to and evaluating on this many held-out tasks. More tasks provide more reliable evaluation metrics.

GradientClipThreshold

Gets or sets the maximum gradient norm for gradient clipping.

public double? GradientClipThreshold { get; set; }

Property Value

double?

The gradient clip threshold, or null to disable. Default is 10.0.

Remarks

Gradient clipping prevents training instability by limiting the magnitude of gradients. In MAML, gradients can become large due to the nested optimization structure, making clipping especially important.

For Beginners: Prevents the model from taking steps that are too large, which could break training. Set to null to disable.

InnerLearningRate

Gets or sets the learning rate for the inner loop (task adaptation).

public double InnerLearningRate { get; set; }

Property Value

double

The inner learning rate. Default is 0.01.

Remarks

In MAML, the inner learning rate controls how quickly the model adapts to a specific task during the K gradient steps of the inner loop. This rate is typically larger than the outer learning rate because adaptation needs to happen quickly with few examples.

For Beginners: This is how fast the model learns when it sees a new task. A higher value means faster but potentially less stable adaptation.

InnerOptimizer

Gets or sets the optimizer for inner-loop adaptation. Default: null (uses built-in SGD optimizer with InnerLearningRate).

public IGradientBasedOptimizer<T, TInput, TOutput>? InnerOptimizer { get; set; }

Property Value

IGradientBasedOptimizer<T, TInput, TOutput>

LossFunction

Gets or sets the loss function for training. Default: null (uses model's default loss function if available).

public ILossFunction<T>? LossFunction { get; set; }

Property Value

ILossFunction<T>

MetaBatchSize

Gets or sets the number of tasks to sample per meta-training iteration.

public int MetaBatchSize { get; set; }

Property Value

int

The meta-batch size. Default is 4.

Remarks

Each meta-training step samples this many tasks, adapts to each one, and averages the meta-gradients. Larger batch sizes provide more stable gradient estimates but require more memory and computation per iteration.

For Beginners: How many different practice tasks to learn from before updating the model's "learning strategy."

MetaModel

Gets or sets the meta-model to be trained. This is the only required property.

public IFullModel<T, TInput, TOutput> MetaModel { get; set; }

Property Value

IFullModel<T, TInput, TOutput>

MetaOptimizer

Gets or sets the optimizer for meta-parameter updates (outer loop). Default: null (uses built-in Adam optimizer with OuterLearningRate).

public IGradientBasedOptimizer<T, TInput, TOutput>? MetaOptimizer { get; set; }

Property Value

IGradientBasedOptimizer<T, TInput, TOutput>

NumMetaIterations

Gets or sets the total number of meta-training iterations to perform.

public int NumMetaIterations { get; set; }

Property Value

int

The number of meta-iterations. Default is 1000.

Remarks

The total number of outer loop updates. Each iteration processes MetaBatchSize tasks. The original MAML paper uses 60,000 iterations for Omniglot and 15,000 for MiniImageNet.

For Beginners: How long to train the meta-learner. More iterations = better learning but longer training time.

OuterLearningRate

Gets or sets the learning rate for the outer loop (meta-optimization).

public double OuterLearningRate { get; set; }

Property Value

double

The outer learning rate. Default is 0.001.

Remarks

The outer learning rate controls how the meta-parameters (initial weights) are updated after evaluating adaptation performance across a batch of tasks. This rate is typically 10x smaller than the inner rate because meta-learning requires careful, gradual updates.

For Beginners: This is how fast the model learns "how to learn." Too high can cause instability; too low means slow meta-training.

RandomSeed

Gets or sets the random seed for reproducibility.

public int? RandomSeed { get; set; }

Property Value

int?

The random seed, or null for non-deterministic behavior. Default is null.

Remarks

Setting a seed ensures reproducible task sampling and weight initialization in MAML. This is useful for debugging and comparing experiments.

UseFirstOrder

Gets or sets whether to use first-order approximation (FOMAML) instead of full MAML.

public bool UseFirstOrder { get; set; }

Property Value

bool

True to use FOMAML; false for full second-order MAML. Default is false.

Remarks

First-order MAML (FOMAML) ignores the second-order derivatives that arise from differentiating through the inner loop adaptation. This dramatically reduces memory usage and computation time while maintaining competitive performance.

For Beginners: A simpler, faster version of MAML that works almost as well. Set to true if you're running out of memory or training is too slow.

UseFirstOrderApproximation

Gets or sets whether to use first-order approximation (FOMAML).

public bool UseFirstOrderApproximation { get; set; }

Property Value

bool

Remarks

When true, ignores second-order derivatives during meta-gradient computation. This is faster and uses less memory, with minimal performance impact.

Default: false (alias for UseFirstOrder property)

Methods

Clone()

Creates a deep copy of the MAML options.

public IMetaLearnerOptions<T> Clone()

Returns

IMetaLearnerOptions<T>

A new MAMLOptions instance with the same configuration values.

Remarks

Note: The MetaModel reference is shared (not deep-cloned) because models are typically expensive to copy and MAML manages its own model cloning internally during the meta-training process.

IsValid()

Validates that all MAML configuration options are properly set.

public bool IsValid()

Returns

bool

True if the configuration is valid for MAML training; otherwise, false.

Remarks

This method checks all critical hyperparameters required for MAML to function correctly: - MetaModel must be set (the neural network to meta-train) - Learning rates must be positive (both inner and outer loops) - Adaptation steps must be at least 1 (K gradient steps in inner loop) - Batch sizes and iteration counts must be positive

For Beginners: Call this before training to ensure your settings make sense. If it returns false, check that all required values are set and positive.