Class ANILOptions<T, TInput, TOutput>

Namespace

AiDotNet.MetaLearning.Options

Assembly

AiDotNet.dll

Configuration options for Almost No Inner Loop (ANIL) algorithm.

public class ANILOptions<T, TInput, TOutput> : IMetaLearnerOptions<T>

Type Parameters

T

The numeric data type used for calculations (e.g., float, double).

TInput

The input data type (e.g., Matrix<T>, Tensor<T>).

TOutput

The output data type (e.g., Vector<T>, Tensor<T>).

Inheritance

object

ANILOptions<T, TInput, TOutput>

Implements

IMetaLearnerOptions<T>

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

ANIL is a simplified version of MAML that only adapts the classification head during inner-loop adaptation, while keeping the feature extractor frozen. This significantly reduces computation while maintaining competitive performance.

For Beginners: ANIL splits a neural network into two parts:

1. Body (Feature Extractor): Learns general features shared across tasks (FROZEN during adaptation)
2. Head (Classifier): Task-specific layer that is adapted for each new task

Key insight: Most of the "learning to learn" happens in the feature extractor, which doesn't need to be adapted per-task. Only the small classifier head needs to change for each new task.

Benefits:

• Much faster than MAML (fewer parameters to adapt)
• Less memory usage (no need to store gradients for body)
• Often performs as well as full MAML

Constructors

ANILOptions(IFullModel<T, TInput, TOutput>)

Initializes a new instance of the ANILOptions class with the required meta-model.

public ANILOptions(IFullModel<T, TInput, TOutput> metaModel)

Parameters

metaModel IFullModel<T, TInput, TOutput>

The neural network to be trained (required).

Exceptions

ArgumentNullException

Thrown when metaModel is null.

Properties

AdaptationSteps

Gets or sets the number of adaptation steps (gradient steps on support set).

public int AdaptationSteps { get; set; }

Property Value

int

Default is 5.

CheckpointFrequency

Gets or sets how often to save checkpoints.

public int CheckpointFrequency { get; set; }

Property Value

int

DataLoader

Gets or sets the episodic data loader for sampling tasks. Default: null (tasks must be provided manually to MetaTrain).

public IEpisodicDataLoader<T, TInput, TOutput>? DataLoader { get; set; }

Property Value

IEpisodicDataLoader<T, TInput, TOutput>

EnableCheckpointing

Gets or sets whether to save model checkpoints.

public bool EnableCheckpointing { get; set; }

Property Value

bool

EvaluationFrequency

Gets or sets how often to evaluate the meta-learner.

public int EvaluationFrequency { get; set; }

Property Value

int

EvaluationTasks

Gets or sets the number of tasks to use for evaluation.

public int EvaluationTasks { get; set; }

Property Value

int

FeatureDimension

Gets or sets the dimension of the final feature representation (before head).

public int FeatureDimension { get; set; }

Property Value

int

Default is 512.

GradientClipThreshold

Gets or sets the maximum gradient norm for gradient clipping.

public double? GradientClipThreshold { get; set; }

Property Value

double?

Default is 10.0.

HeadL2Regularization

Gets or sets the L2 regularization strength for the head.

public double HeadL2Regularization { get; set; }

Property Value

double

Default is 0.0 (no regularization).

InnerLearningRate

Gets or sets the learning rate for the inner loop (head adaptation).

public double InnerLearningRate { get; set; }

Property Value

double

Default is 0.01.

InnerOptimizer

Gets or sets the optimizer for inner loop updates (head only). Default: null (uses SGD with InnerLearningRate).

public IGradientBasedOptimizer<T, TInput, TOutput>? InnerOptimizer { get; set; }

Property Value

IGradientBasedOptimizer<T, TInput, TOutput>

LossFunction

Gets or sets the loss function for training. Default: null (uses cross-entropy loss internally).

public ILossFunction<T>? LossFunction { get; set; }

Property Value

ILossFunction<T>

MetaBatchSize

Gets or sets the number of tasks to sample per meta-training iteration.

public int MetaBatchSize { get; set; }

Property Value

int

Default is 4.

MetaModel

Gets or sets the meta-model to be trained. This is the only required property.

public IFullModel<T, TInput, TOutput> MetaModel { get; set; }

Property Value

IFullModel<T, TInput, TOutput>

Remarks

The meta-model should be a neural network with a separable body (feature extractor) and head (classifier). Only the head will be adapted during the inner loop.

MetaOptimizer

Gets or sets the optimizer for meta-parameter (outer loop) updates. Default: null (uses built-in Adam optimizer with OuterLearningRate).

public IGradientBasedOptimizer<T, TInput, TOutput>? MetaOptimizer { get; set; }

Property Value

IGradientBasedOptimizer<T, TInput, TOutput>

NumClasses

Gets or sets the number of output classes.

public int NumClasses { get; set; }

Property Value

int

Default is 5.

NumMetaIterations

Gets or sets the total number of meta-training iterations.

public int NumMetaIterations { get; set; }

Property Value

int

Default is 1000.

OuterLearningRate

Gets or sets the learning rate for the outer loop (meta-update).

public double OuterLearningRate { get; set; }

Property Value

double

Default is 0.001.

RandomSeed

Gets or sets the random seed for reproducibility.

public int? RandomSeed { get; set; }

Property Value

int?

ReinitializeHead

Gets or sets whether to reinitialize the head for each task.

public bool ReinitializeHead { get; set; }

Property Value

bool

Default is false (use meta-learned head initialization).

Remarks

For Beginners: If true, the classifier head starts fresh for each task. If false, it starts from a meta-learned initialization that is optimized across tasks.

UseFirstOrder

Gets or sets whether to use first-order approximation.

public bool UseFirstOrder { get; set; }

Property Value

bool

Default is true (ANIL typically uses first-order for efficiency).

UseHeadBias

Gets or sets whether to use a bias term in the classification head.

public bool UseHeadBias { get; set; }

Property Value

bool

Default is true.

Methods

Clone()

Creates a deep copy of the ANIL options.

public IMetaLearnerOptions<T> Clone()

Returns

IMetaLearnerOptions<T>

A new ANILOptions instance with the same configuration.

IsValid()

Validates that all ANIL configuration options are properly set.

public bool IsValid()

Returns

bool

True if the configuration is valid; otherwise, false.