Class MetaLearnerBase<T, TInput, TOutput>

Namespace

AiDotNet.MetaLearning

Assembly

AiDotNet.dll

Unified base class for all meta-learning algorithms, providing both training infrastructure and shared algorithm utilities.

public abstract class MetaLearnerBase<T, TInput, TOutput> : IMetaLearner<T, TInput, TOutput>

Type Parameters

T

The numeric data type used for calculations (e.g., float, double).

TInput

The input data type (e.g., Matrix<T>, Tensor<T>).

TOutput

The output data type (e.g., Vector<T>, Tensor<T>).

Inheritance

object

MetaLearnerBase<T, TInput, TOutput>

Implements

IMetaLearner<T, TInput, TOutput>

Derived

ANILAlgorithm<T, TInput, TOutput>

BOILAlgorithm<T, TInput, TOutput>

CNAPAlgorithm<T, TInput, TOutput>

GNNMetaAlgorithm<T, TInput, TOutput>

LEOAlgorithm<T, TInput, TOutput>

MAMLAlgorithm<T, TInput, TOutput>

MANNAlgorithm<T, TInput, TOutput>

MatchingNetworksAlgorithm<T, TInput, TOutput>

MetaOptNetAlgorithm<T, TInput, TOutput>

MetaSGDAlgorithm<T, TInput, TOutput>

NTMAlgorithm<T, TInput, TOutput>

ProtoNetsAlgorithm<T, TInput, TOutput>

RelationNetworkAlgorithm<T, TInput, TOutput>

ReptileAlgorithm<T, TInput, TOutput>

SEALAlgorithm<T, TInput, TOutput>

TADAMAlgorithm<T, TInput, TOutput>

iMAMLAlgorithm<T, TInput, TOutput>

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

This base class follows the TimeSeriesModelBase pattern where the base class does heavy lifting and concrete algorithm implementations override with their custom logic. It provides:

• Configuration management and validation
• Loss and accuracy computation
• Model saving/loading with serialization
• Training loop orchestration
• Evaluation on multiple tasks
• Gradient computation, clipping, and application utilities
• Built-in inner/outer optimizer support with Adam defaults

For Algorithm Implementers: To create a new meta-learning algorithm: 1. Extend this base class 2. Set AlgorithmType in constructor 3. Override MetaTrainCore() with your algorithm's meta-update logic 4. Override AdaptCore() with your task adaptation strategy 5. All shared functionality (metrics, saving, evaluation) is handled automatically

Constructors

MetaLearnerBase(IFullModel<T, TInput, TOutput>, ILossFunction<T>, IMetaLearnerOptions<T>, IEpisodicDataLoader<T, TInput, TOutput>?, IGradientBasedOptimizer<T, TInput, TOutput>?, IGradientBasedOptimizer<T, TInput, TOutput>?)

Initializes a new instance of the MetaLearnerBase class.

protected MetaLearnerBase(IFullModel<T, TInput, TOutput> metaModel, ILossFunction<T> lossFunction, IMetaLearnerOptions<T> options, IEpisodicDataLoader<T, TInput, TOutput>? dataLoader = null, IGradientBasedOptimizer<T, TInput, TOutput>? metaOptimizer = null, IGradientBasedOptimizer<T, TInput, TOutput>? innerOptimizer = null)

Parameters

metaModel IFullModel<T, TInput, TOutput>

The model to meta-train.

lossFunction ILossFunction<T>

Loss function for evaluation.

options IMetaLearnerOptions<T>

Configuration options with all hyperparameters.

dataLoader IEpisodicDataLoader<T, TInput, TOutput>

Optional episodic data loader for sampling tasks.

metaOptimizer IGradientBasedOptimizer<T, TInput, TOutput>

Optional optimizer for meta-updates. If null, gradient updates use manual SGD with OuterLearningRate.

innerOptimizer IGradientBasedOptimizer<T, TInput, TOutput>

Optional optimizer for inner-loop. If null, gradient updates use manual SGD with InnerLearningRate.

Exceptions

ArgumentNullException

Thrown when metaModel, lossFunction, or options is null.

ArgumentException

Thrown when configuration validation fails.

Fields

DataLoader

Episodic data loader for sampling meta-learning tasks.

protected readonly IEpisodicDataLoader<T, TInput, TOutput>? DataLoader

Field Value

IEpisodicDataLoader<T, TInput, TOutput>

InnerOptimizer

Optimizer for task adaptation (inner loop).

protected readonly IGradientBasedOptimizer<T, TInput, TOutput>? InnerOptimizer

Field Value

IGradientBasedOptimizer<T, TInput, TOutput>

LossFunction

The loss function used to evaluate task performance.

protected readonly ILossFunction<T> LossFunction

Field Value

ILossFunction<T>

MetaModel

The model being meta-trained.

protected IFullModel<T, TInput, TOutput> MetaModel

Field Value

IFullModel<T, TInput, TOutput>

MetaOptimizer

Optimizer for meta-parameter updates (outer loop).

protected readonly IGradientBasedOptimizer<T, TInput, TOutput>? MetaOptimizer

Field Value

IGradientBasedOptimizer<T, TInput, TOutput>

NumOps

Numeric operations for type T.

protected static readonly INumericOperations<T> NumOps

Field Value

INumericOperations<T>

RandomGenerator

Random number generator for stochastic operations.

protected Random RandomGenerator

Field Value

Random

_currentIteration

Current meta-training iteration count.

protected int _currentIteration

Field Value

int

_options

Configuration options for meta-learning.

protected readonly IMetaLearnerOptions<T> _options

Field Value

IMetaLearnerOptions<T>

Properties

AdaptationSteps

Gets the number of adaptation steps to perform during task adaptation (inner loop).

public int AdaptationSteps { get; }

Property Value

int

AlgorithmType

Gets the type of meta-learning algorithm.

public abstract MetaLearningAlgorithmType AlgorithmType { get; }

Property Value

MetaLearningAlgorithmType

BaseModel

Gets the base model being meta-trained.

public IFullModel<T, TInput, TOutput> BaseModel { get; }

Property Value

IFullModel<T, TInput, TOutput>

CurrentIteration

Gets the current meta-training iteration count.

public int CurrentIteration { get; }

Property Value

int

Engine

Gets the global execution engine for vectorized operations.

protected IEngine Engine { get; }

Property Value

IEngine

Remarks

The execution engine provides CPU/GPU-accelerated vector and matrix operations. All vectorized computations should use this engine rather than manual loops for optimal performance and hardware acceleration.

For Beginners: This engine automatically chooses the best way to run math operations - on your CPU with SIMD optimization, or on your GPU for massive parallelism. You don't need to change your code to switch between them.

InnerLearningRate

Gets the learning rate used for task adaptation (inner loop).

public double InnerLearningRate { get; }

Property Value

double

Options

Gets the meta-learner options (configuration).

public IMetaLearnerOptions<T> Options { get; }

Property Value

IMetaLearnerOptions<T>

OuterLearningRate

Gets the learning rate used for meta-learning (outer loop).

public double OuterLearningRate { get; }

Property Value

double

Methods

Adapt(IMetaLearningTask<T, TInput, TOutput>)

Adapts the model to a new task using its support set.

public abstract IModel<TInput, TOutput, ModelMetadata<T>> Adapt(IMetaLearningTask<T, TInput, TOutput> task)

Parameters

task IMetaLearningTask<T, TInput, TOutput>

The task to adapt to.

Returns

IModel<TInput, TOutput, ModelMetadata<T>>

A new model instance adapted to the task.

Remarks

For Beginners: This is where the "quick learning" happens. Given a new task with just a few examples (the support set), this method creates a new model that's specialized for that specific task.

AdaptAndEvaluate(MetaLearningTask<T, TInput, TOutput>)

Adapts the model to a specific task and evaluates adaptation quality.

public virtual MetaAdaptationResult<T> AdaptAndEvaluate(MetaLearningTask<T, TInput, TOutput> task)

Parameters

task MetaLearningTask<T, TInput, TOutput>

Meta-learning task with support set (for adaptation) and query set (for evaluation).

Returns

MetaAdaptationResult<T>

Detailed metrics about adaptation performance and timing.

ApplyGradients(Vector<T>, Vector<T>, double)

Applies gradients to update model parameters using simple SGD.

protected virtual Vector<T> ApplyGradients(Vector<T> parameters, Vector<T> gradients, double learningRate)

Parameters

parameters Vector<T>

Current parameters.

gradients Vector<T>

Gradients to apply.

learningRate double

Learning rate.

Returns

Vector<T>

Updated parameters.

Remarks

This method performs vanilla SGD: θ = θ - lr * gradients. It does NOT use the stored MetaOptimizer or InnerOptimizer because those optimizers expect to work with the full model, not just parameter vectors.

For Advanced Optimizers: If you need momentum, Adam, or other adaptive methods, override this method in your algorithm implementation or use the stored optimizers directly in your MetaTrain(TaskBatch<T, TInput, TOutput>) and Adapt(IMetaLearningTask<T, TInput, TOutput>) implementations.

ClipGradients(Vector<T>, double?)

Clips gradients to prevent exploding gradients.

protected virtual Vector<T> ClipGradients(Vector<T> gradients, double? threshold = null)

Parameters

gradients Vector<T>

Gradients to clip.

threshold double?

Maximum gradient norm. If null, uses options value.

Returns

Vector<T>

Clipped gradients.

CloneModel()

Clones the meta-model for task-specific adaptation.

protected virtual IFullModel<T, TInput, TOutput> CloneModel()

Returns

IFullModel<T, TInput, TOutput>

Exceptions

InvalidOperationException

Thrown when the model does not implement ICloneable. Meta-learning requires model cloning to prevent parameter corruption during parallel task adaptation.

ComputeAccuracy(TOutput, TOutput)

Computes accuracy for classification tasks.

protected virtual double ComputeAccuracy(TOutput predictions, TOutput labels)

Parameters

predictions TOutput

Model predictions (logits or probabilities per class).

labels TOutput

Ground truth labels (one-hot encoded or class indices).

Returns

double

Accuracy as a value between 0.0 and 1.0, or 0.0 if computation is not possible.

Remarks

This method computes classification accuracy by comparing predicted class indices (argmax of predictions) against true class indices (argmax of labels for one-hot, or direct value for class indices).

For Beginners: Accuracy is simply the fraction of samples where the model's predicted class matches the true class. For example, 0.85 means 85% correct predictions.

ComputeGradients(IFullModel<T, TInput, TOutput>, TInput, TOutput)

Computes gradients of the loss with respect to model parameters using the model's built-in gradient computation via IGradientComputable<T, TInput, TOutput>.

protected virtual Vector<T> ComputeGradients(IFullModel<T, TInput, TOutput> model, TInput input, TOutput expectedOutput)

Parameters

model IFullModel<T, TInput, TOutput>

The model to compute gradients for.

input TInput

Input data.

expectedOutput TOutput

Expected output.

Returns

Vector<T>

Gradient vector with respect to all model parameters.

Remarks

This method leverages the model's own gradient computation capabilities, which typically use: - Automatic differentiation (backpropagation for neural networks) - GPU acceleration via the Engine system when available - Optimized JIT-compiled gradient kernels

Production Implementation: Models that implement IGradientComputable<T, TInput, TOutput> provide their own efficient gradient computation. This is the preferred path for production use.

For Beginners: Gradients tell us how to adjust each parameter to reduce the loss. This method asks the model to compute those gradients using its internal backpropagation mechanism, which is much more accurate and efficient than numerical approximations.

Exceptions

InvalidOperationException

Thrown when the model does not implement IGradientComputable<T, TInput, TOutput> and no fallback computation is possible.

ComputeLossFromOutput(TOutput, TOutput)

Computes loss from TOutput by converting to Vector<T> if needed.

protected virtual T ComputeLossFromOutput(TOutput predictions, TOutput expectedOutput)

Parameters

predictions TOutput

The predictions from the model.

expectedOutput TOutput

The expected output.

Returns

T

The computed loss value.

ComputeMean(List<T>)

Computes mean of a list of values.

protected T ComputeMean(List<T> values)

Parameters

values List<T>

Returns

T

ComputeSecondOrderGradients(IFullModel<T, TInput, TOutput>, List<(TInput input, TOutput target)>, TInput, TOutput, T)

Computes second-order gradients for full MAML by backpropagating through the adaptation process.

protected virtual Vector<T> ComputeSecondOrderGradients(IFullModel<T, TInput, TOutput> model, List<(TInput input, TOutput target)> adaptationSteps, TInput queryInput, TOutput queryTarget, T innerLearningRate)

Parameters

model IFullModel<T, TInput, TOutput>

The model to compute second-order gradients for.

adaptationSteps List<(TInput input, TOutput target)>

Sequence of (input, target) pairs used during inner loop adaptation.

queryInput TInput

Query input for meta-gradient computation.

queryTarget TOutput

Query target for meta-gradient computation.

innerLearningRate T

Learning rate used during inner loop adaptation.

Returns

Vector<T>

Meta-gradients that account for how changing initial parameters affects learning trajectory.

Remarks

Second-order gradients are required for full MAML (Model-Agnostic Meta-Learning). They capture how changes to initial parameters affect the entire adaptation trajectory, not just the final adapted parameters.

Computational Cost: O(K * P^2) where K is adaptation steps and P is parameter count. For large models, consider using first-order MAML (FOMAML) approximation instead.

For Beginners: Normal gradients ask "how should I change parameters to improve?" Second-order gradients ask "how should I change my starting point so that learning itself becomes more effective?"

Exceptions

InvalidOperationException

Thrown when the model doesn't implement ISecondOrderGradientComputable<T, TInput, TOutput> and second-order gradients are required.

ConvertToVector(TOutput)

Converts TOutput to Vector<T> if possible.

protected virtual Vector<T>? ConvertToVector(TOutput output)

Parameters

output TOutput

The output to convert.

Returns

Vector<T>

The converted vector, or null if conversion is not possible.

CreateTaskBatch(IReadOnlyList<MetaLearningTask<T, TInput, TOutput>>)

Creates a TaskBatch from a list of MetaLearningTasks.

protected TaskBatch<T, TInput, TOutput> CreateTaskBatch(IReadOnlyList<MetaLearningTask<T, TInput, TOutput>> tasks)

Parameters

tasks IReadOnlyList<MetaLearningTask<T, TInput, TOutput>>

Returns

TaskBatch<T, TInput, TOutput>

Evaluate(TaskBatch<T, TInput, TOutput>)

Evaluates the meta-learning algorithm on a batch of tasks.

public virtual T Evaluate(TaskBatch<T, TInput, TOutput> taskBatch)

Parameters

taskBatch TaskBatch<T, TInput, TOutput>

The batch of tasks to evaluate on.

Returns

T

The average evaluation loss across all tasks.

Remarks

For Beginners: This checks how well the meta-learning algorithm performs. For each task, it adapts using the support set and then tests on the query set. The returned value is the average loss across all tasks - lower means better performance.

Evaluate(int)

Evaluates meta-learning performance on multiple held-out tasks.

public virtual MetaEvaluationResult<T> Evaluate(int numTasks)

Parameters

numTasks int

Number of tasks to evaluate (100-1000 recommended for statistics).

Returns

MetaEvaluationResult<T>

Comprehensive metrics including mean accuracy, confidence intervals, and per-task statistics.

Remarks

Uses the episodic data loader configured during construction to sample evaluation tasks.

GetMetaModel()

Gets the current meta-model.

public IFullModel<T, TInput, TOutput> GetMetaModel()

Returns

IFullModel<T, TInput, TOutput>

The current meta-model.

Remarks

For Beginners: This returns the "meta-learned" model that has been trained on many tasks. This model itself may not be very good at any specific task, but it's excellent as a starting point for quickly adapting to new tasks.

Load(string)

Loads a previously meta-trained model from disk.

public virtual void Load(string filePath)

Parameters

filePath string

File path to the saved model.

Remarks

This method loads the meta-model parameters. Note that optimizer state is not restored and will be reset after loading. Call Reset() after loading if you want to ensure a clean training state.

MetaTrain(TaskBatch<T, TInput, TOutput>)

Performs one meta-training step on a batch of tasks.

public abstract T MetaTrain(TaskBatch<T, TInput, TOutput> taskBatch)

Parameters

taskBatch TaskBatch<T, TInput, TOutput>

The batch of tasks to train on.

Returns

T

The meta-training loss for this batch.

Remarks

For Beginners: This method updates the model by training on multiple tasks at once. Each task teaches the model something about how to learn quickly. The returned loss value indicates how well the model is doing - lower is better.

MetaTrainStep(int)

Performs one meta-training step (outer loop update) using the episodic data loader.

public virtual MetaTrainingStepResult<T> MetaTrainStep(int batchSize)

Parameters

batchSize int

Number of tasks to sample for this meta-update.

Returns

MetaTrainingStepResult<T>

Metrics including meta-loss, task loss, accuracy, and timing information.

Remarks

Uses the episodic data loader configured during construction to sample tasks for this meta-update.

Reset()

Resets the meta-learner to initial untrained state.

public virtual void Reset()

Save(string)

Saves the meta-trained model to disk for later deployment.

public virtual void Save(string filePath)

Parameters

filePath string

File path where model should be saved.

Remarks

This method saves the meta-model parameters. Note that optimizer state (momentum, velocity) is not persisted and will be reset when loading. For production use, consider implementing a custom serialization scheme that saves:

• Model parameters (saved here)
• Optimizer state (momentum, velocity vectors)
• Current iteration count
• Learning rate schedules

SetMetaModel(IFullModel<T, TInput, TOutput>)

Sets the base model for this meta-learning algorithm.

public void SetMetaModel(IFullModel<T, TInput, TOutput> model)

Parameters

model IFullModel<T, TInput, TOutput>

The model to use as the base.

ToMetaLearningTask(MetaLearningTask<T, TInput, TOutput>)

Converts a MetaLearningTask to IMetaLearningTask.

protected IMetaLearningTask<T, TInput, TOutput> ToMetaLearningTask(MetaLearningTask<T, TInput, TOutput> task)

Parameters

task MetaLearningTask<T, TInput, TOutput>

Returns

IMetaLearningTask<T, TInput, TOutput>

Train()

Trains the meta-learner using the configuration specified during construction.

public virtual MetaTrainingResult<T> Train()

Returns

MetaTrainingResult<T>

Complete training history with loss/accuracy progression and timing information.

Remarks

This method performs the complete outer-loop meta-training process, repeatedly calling MetaTrainStep and collecting metrics across all iterations. All training parameters are specified in the IMetaLearnerOptions provided during construction.

For Beginners: This is the main training method for meta-learning. Unlike traditional training where you train once on a dataset, this trains your model across many different tasks so it learns how to quickly adapt to new tasks.