Table of Contents

Class HTMNetwork<T>

Namespace
AiDotNet.NeuralNetworks
Assembly
AiDotNet.dll

Represents a Hierarchical Temporal Memory (HTM) network, a biologically-inspired sequence learning algorithm.

public class HTMNetwork<T> : NeuralNetworkBase<T>, INeuralNetworkModel<T>, INeuralNetwork<T>, IFullModel<T, Tensor<T>, Tensor<T>>, IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>, IModelSerializer, ICheckpointableModel, IParameterizable<T, Tensor<T>, Tensor<T>>, IFeatureAware, IFeatureImportance<T>, ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>, IGradientComputable<T, Tensor<T>, Tensor<T>>, IJitCompilable<T>, IInterpretableModel<T>, IInputGradientComputable<T>, IDisposable

Type Parameters

T

The numeric type used for calculations, typically float or double.

Inheritance
HTMNetwork<T>
Implements
IFullModel<T, Tensor<T>, Tensor<T>>
IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>
IParameterizable<T, Tensor<T>, Tensor<T>>
ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>
IGradientComputable<T, Tensor<T>, Tensor<T>>
Inherited Members
Extension Methods

Remarks

Hierarchical Temporal Memory (HTM) is a machine learning model that mimics the structural and algorithmic properties of the neocortex. It is particularly designed for sequence learning, prediction, and anomaly detection in time-series data. HTM networks consist of two main components: a Spatial Pooler that creates sparse distributed representations of inputs, and a Temporal Memory that learns sequences of these representations.

For Beginners: HTM is a special type of neural network inspired by how the human brain works.

Think of HTM like a system that learns patterns over time:

  • It's similar to how your brain recognizes songs or predicts what comes next in a familiar sequence
  • It's especially good at learning from time-series data (information that changes over time)
  • It can recognize patterns even when they contain noise or slight variations

HTM networks have two main parts:

  • Spatial Pooler: This converts incoming data into a special format that highlights important patterns (like how your brain might focus on the melody of a song rather than background noise)
  • Temporal Memory: This learns sequences and can predict what might come next (like how you can anticipate the next note in a familiar song)

HTM is particularly useful for:

  • Anomaly detection (finding unusual patterns)
  • Sequence prediction (guessing what comes next)
  • Pattern recognition in noisy data

Constructors

HTMNetwork(NeuralNetworkArchitecture<T>, int, int, double, ILossFunction<T>?)

Initializes a new instance of the HTMNetwork<T> class with the specified architecture and parameters.

public HTMNetwork(NeuralNetworkArchitecture<T> architecture, int columnCount = 2048, int cellsPerColumn = 32, double sparsityThreshold = 0.02, ILossFunction<T>? lossFunction = null)

Parameters

architecture NeuralNetworkArchitecture<T>

The neural network architecture defining the structure of the network.

columnCount int

The number of columns in the spatial pooler. Default is 2048.

cellsPerColumn int

The number of cells per column in the temporal memory. Default is 32.

sparsityThreshold double

The target sparsity for the spatial pooler output. Default is 0.02 (2%).

lossFunction ILossFunction<T>

Remarks

This constructor creates an HTM network with the specified architecture and parameters. It initializes the spatial pooler and temporal memory components based on the provided configuration values. The default parameters are based on common values used in HTM research and applications.

For Beginners: This creates a new HTM network with your chosen settings.

When creating an HTM network, you can customize several important aspects:

  1. Architecture: The basic structure of your network

  2. Column Count: How many pattern detectors (default is 2048)

    • More columns = more capacity to learn different patterns
    • Fewer columns = faster but less capable

  3. Cells Per Column: How much context the network can understand (default is 32)

    • More cells = better at understanding complex sequences
    • Fewer cells = simpler but may confuse similar sequences

  4. Sparsity Threshold: How selective the network is (default is 0.02 or 2%)

    • Lower values = more selective, focuses on fewer elements
    • Higher values = less selective, considers more elements important

The default values work well for many applications, but you can adjust them based on your specific needs.

Exceptions

InvalidOperationException

Thrown when the input shape is not specified in the architecture.

Methods

CreateNewInstance()

Creates a new instance of the HTM Network with the same architecture and configuration.

protected override IFullModel<T, Tensor<T>, Tensor<T>> CreateNewInstance()

Returns

IFullModel<T, Tensor<T>, Tensor<T>>

A new HTM Network instance with the same architecture and configuration.

Remarks

This method creates a new instance of the HTM Network with the same architecture and HTM-specific parameters as the current instance. It's used in scenarios where a fresh copy of the model is needed while maintaining the same configuration.

For Beginners: This method creates a brand new copy of the HTM network with the same setup.

Think of it like creating a clone of the network:

  • The new network has the same architecture (structure)
  • It has the same number of columns, cells per column, and sparsity threshold
  • But it's a completely separate instance with its own state
  • It starts with clean internal memory and connections

This is useful when you want to:

  • Train the same network design on different datasets
  • Compare how the same network structure learns from different sequences
  • Start with a fresh network that has the same configuration but no learned patterns

DeserializeNetworkSpecificData(BinaryReader)

Deserializes HTM-specific data from a binary reader.

protected override void DeserializeNetworkSpecificData(BinaryReader reader)

Parameters

reader BinaryReader

The binary reader to read from.

GetModelMetadata()

Gets metadata about the HTM network.

public override ModelMetadata<T> GetModelMetadata()

Returns

ModelMetadata<T>

A ModelMetaData object containing information about the HTM network.

Remarks

This method returns comprehensive metadata about the HTM network, including its architecture, layer configuration, and HTM-specific parameters.

For Beginners: This provides detailed information about your HTM network.

The metadata includes:

  • What kind of network this is (HTM)
  • Details about its structure (columns, cells, etc.)
  • Configuration settings
  • Basic statistics about the network

This is useful for documentation, comparison with other models, and keeping track of different network configurations.

InitializeLayers()

Initializes the layers of the HTM network based on the provided architecture.

protected override void InitializeLayers()

Remarks

This method sets up the layers of the HTM network. If the architecture provides specific layers, those are used directly. Otherwise, default layers appropriate for an HTM network are created, including a spatial pooler layer and a temporal memory layer configured with the specified parameters.

For Beginners: This method sets up the building blocks of your HTM network.

When initializing the network:

  • If you provided specific layers in the architecture, those are used
  • If not, the network creates standard HTM layers automatically

The standard HTM layers typically include:

  1. Spatial Pooler: Converts input to sparse distributed representations
  2. Temporal Memory: Learns sequences and makes predictions
  3. Optional additional layers: May include encoder/decoder layers or other processing

This process is like assembling all the components before the network starts learning.

Learn(Vector<T>)

Processes an input vector through the network and updates the network's internal state based on the input.

public void Learn(Vector<T> input)

Parameters

input Vector<T>

The input vector to learn from.

Remarks

This method implements the learning process in an HTM network. It first propagates the input through the spatial pooler and temporal memory layers. Then it performs learning operations on both components, allowing the network to adapt to the input sequence. The temporal memory layer's state is updated to maintain the sequence context for the next input.

For Beginners: This method both predicts AND learns from new data.

When you call this method with new input data:

  1. The network first processes the input through the Spatial Pooler

    • Converts your input to a sparse representation
    • Highlights the most important patterns

  2. Then it processes the result through the Temporal Memory

    • Makes predictions based on what it has learned so far
    • Creates a representation of the current state

  3. The Temporal Memory learns from this new information

    • Updates its understanding of sequences
    • Adjusts its predictions for the future

  4. The network stores the current state to use as context for the next input

    • This is how it maintains an understanding of sequences over time

  5. The Spatial Pooler also learns from the input

    • Improves its ability to identify important patterns
    • Adapts to changing input statistics

This continuous learning process allows the network to improve over time as it sees more data.

Exceptions

ArgumentException

Thrown when the input size doesn't match the expected size.

InvalidOperationException

Thrown when the network doesn't contain the expected layer types.

Predict(Tensor<T>)

Makes a prediction using the HTM network.

public override Tensor<T> Predict(Tensor<T> input)

Parameters

input Tensor<T>

The input tensor to process.

Returns

Tensor<T>

A tensor containing the network's prediction.

SerializeNetworkSpecificData(BinaryWriter)

Serializes HTM-specific data to a binary writer.

protected override void SerializeNetworkSpecificData(BinaryWriter writer)

Parameters

writer BinaryWriter

The binary writer to write to.

Train(Tensor<T>, Tensor<T>)

Trains the HTM network on a sequence of inputs.

public override void Train(Tensor<T> input, Tensor<T> expectedOutput)

Parameters

input Tensor<T>

The input tensor, which may contain a batch of inputs or a single input.

expectedOutput Tensor<T>

Not used in HTM networks as they are self-supervised.

Remarks

HTM networks are self-supervised and learn to predict future inputs based on past patterns. This method processes inputs sequentially through the network, allowing it to build a model of the temporal patterns in the data.

For Beginners: This method teaches the network to recognize patterns in sequential data.

Unlike traditional neural networks, HTMs:

  • Don't need labels or expected outputs
  • Learn by trying to predict what comes next in a sequence
  • Build their own understanding of the patterns in your data

This method processes each input in sequence, helping the network learn temporal patterns and relationships in your data over time.

UpdateParameters(Vector<T>)

Updates the parameters of all layers in the network using the provided parameter vector.

public override void UpdateParameters(Vector<T> parameters)

Parameters

parameters Vector<T>

A vector containing updated parameters for all layers.

Remarks

This method distributes the provided parameter values to each layer in the network. It extracts the appropriate segment of the parameter vector for each layer based on the layer's parameter count. For HTM networks, this is typically used for fine-tuning parameters after initial learning.

For Beginners: This method updates all the configuration values in the network.

While HTM networks primarily learn through the specific Learning methods in each layer, this method allows you to directly update the parameters of all layers at once:

  1. It takes a long list of parameter values
  2. Determines which values belong to which layers
  3. Updates each layer with its corresponding values

This might be used for:

  • Fine-tuning a network after initial training
  • Applying optimized parameters found through experimentation
  • Resetting certain aspects of the network

This method is less commonly used in HTM networks compared to other types of neural networks, as HTMs typically learn through their specialized learning mechanisms.

Edit this page