Class NTMOptions<T, TInput, TOutput>

Namespace

AiDotNet.MetaLearning.Options

Assembly

AiDotNet.dll

Configuration options for Neural Turing Machine (NTM) algorithm.

public class NTMOptions<T, TInput, TOutput> : IMetaLearnerOptions<T>

Type Parameters

T

The numeric data type used for calculations (e.g., float, double).

TInput

The input data type (e.g., Matrix<T>, Tensor<T>).

TOutput

The output data type (e.g., Vector<T>, Tensor<T>).

Inheritance

object

NTMOptions<T, TInput, TOutput>

Implements

IMetaLearnerOptions<T>

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

Neural Turing Machines augment neural networks with an external memory matrix and differentiable attention mechanisms for reading and writing. This enables algorithms to be learned and executed within the neural network itself.

For Beginners: NTM is like a neural computer with RAM:

1. Controller network processes inputs like a CPU
2. Generates read/write keys for memory access
3. Attention mechanism determines where to read/write
4. External memory stores information persistently
5. Differentiable operations allow end-to-end learning

This allows learning algorithms like sorting, copying, and associative recall!

Constructors

NTMOptions(IFullModel<T, TInput, TOutput>)

Initializes a new instance of the NTMOptions class with the required meta-model.

public NTMOptions(IFullModel<T, TInput, TOutput> metaModel)

Parameters

metaModel IFullModel<T, TInput, TOutput>

The controller network to be trained (required).

Exceptions

ArgumentNullException

Thrown when metaModel is null.

Properties

AdaptationSteps

Gets or sets the number of adaptation steps.

public int AdaptationSteps { get; set; }

Property Value

int

CheckpointFrequency

Gets or sets how often to save checkpoints.

public int CheckpointFrequency { get; set; }

Property Value

int

ControllerHiddenSize

Gets or sets the hidden size of the controller.

public int ControllerHiddenSize { get; set; }

Property Value

int

Default is 100.

ControllerType

Gets or sets the type of controller to use.

public NTMControllerType ControllerType { get; set; }

Property Value

NTMControllerType

Default is LSTM.

DataLoader

Gets or sets the episodic data loader for sampling tasks. Default: null (tasks must be provided manually to MetaTrain).

public IEpisodicDataLoader<T, TInput, TOutput>? DataLoader { get; set; }

Property Value

IEpisodicDataLoader<T, TInput, TOutput>

EnableCheckpointing

Gets or sets whether to save model checkpoints.

public bool EnableCheckpointing { get; set; }

Property Value

bool

EvaluationFrequency

Gets or sets how often to evaluate the meta-learner.

public int EvaluationFrequency { get; set; }

Property Value

int

EvaluationTasks

Gets or sets the number of tasks to use for evaluation.

public int EvaluationTasks { get; set; }

Property Value

int

GradientClipThreshold

Gets or sets the maximum gradient norm for gradient clipping.

public double? GradientClipThreshold { get; set; }

Property Value

double?

InitializeMemory

Gets or sets whether to initialize memory at start of episodes.

public bool InitializeMemory { get; set; }

Property Value

bool

Default is true.

InnerLearningRate

Gets or sets the learning rate for the inner loop (not used in NTM).

public double InnerLearningRate { get; set; }

Property Value

double

InnerOptimizer

Gets or sets the optimizer for inner loop updates. Default: null (NTM uses memory-based adaptation, not gradient-based inner loop).

public IGradientBasedOptimizer<T, TInput, TOutput>? InnerOptimizer { get; set; }

Property Value

IGradientBasedOptimizer<T, TInput, TOutput>

LossFunction

Gets or sets the loss function for training. Default: null (uses cross-entropy loss internally).

public ILossFunction<T>? LossFunction { get; set; }

Property Value

ILossFunction<T>

MemoryInitialization

Gets or sets how to initialize memory.

public NTMMemoryInitialization MemoryInitialization { get; set; }

Property Value

NTMMemoryInitialization

Default is Zeros.

MemorySharpnessRegularization

Gets or sets the memory sharpness regularization strength.

public double MemorySharpnessRegularization { get; set; }

Property Value

double

Default is 0.0 (no regularization).

MemorySize

Gets or sets the number of memory slots.

public int MemorySize { get; set; }

Property Value

int

Default is 128.

MemoryUsageRegularization

Gets or sets the memory usage regularization strength.

public double MemoryUsageRegularization { get; set; }

Property Value

double

Default is 0.0 (no regularization).

MemoryWidth

Gets or sets the dimension of each memory slot.

public int MemoryWidth { get; set; }

Property Value

int

Default is 20.

MetaBatchSize

Gets or sets the number of tasks to sample per meta-training iteration.

public int MetaBatchSize { get; set; }

Property Value

int

MetaModel

Gets or sets the meta-model (controller network) to be trained. This is the only required property.

public IFullModel<T, TInput, TOutput> MetaModel { get; set; }

Property Value

IFullModel<T, TInput, TOutput>

Remarks

The meta-model acts as the controller that processes inputs and generates read/write keys for memory operations.

MetaOptimizer

Gets or sets the optimizer for network updates. Default: null (uses built-in Adam optimizer with OuterLearningRate).

public IGradientBasedOptimizer<T, TInput, TOutput>? MetaOptimizer { get; set; }

Property Value

IGradientBasedOptimizer<T, TInput, TOutput>

NumClasses

Gets or sets the number of output classes.

public int NumClasses { get; set; }

Property Value

int

Default is 5.

NumMetaIterations

Gets or sets the total number of meta-training iterations.

public int NumMetaIterations { get; set; }

Property Value

int

NumReadHeads

Gets or sets the number of read heads.

public int NumReadHeads { get; set; }

Property Value

int

Default is 1.

NumWriteHeads

Gets or sets the number of write heads.

public int NumWriteHeads { get; set; }

Property Value

int

Default is 1.

OuterLearningRate

Gets or sets the learning rate for the outer loop (controller training).

public double OuterLearningRate { get; set; }

Property Value

double

RandomSeed

Gets or sets the random seed for reproducibility.

public int? RandomSeed { get; set; }

Property Value

int?

UseFirstOrder

Gets or sets whether to use first-order approximation.

public bool UseFirstOrder { get; set; }

Property Value

bool

Methods

Clone()

Creates a deep copy of the NTM options.

public IMetaLearnerOptions<T> Clone()

Returns

IMetaLearnerOptions<T>

IsValid()

Validates that all NTM configuration options are properly set.

public bool IsValid()

Returns

bool