Table of Contents

Class SpatialPoolerLayer<T>

Namespace
AiDotNet.NeuralNetworks.Layers
Assembly
AiDotNet.dll

Represents a spatial pooler layer inspired by hierarchical temporal memory (HTM) principles.

public class SpatialPoolerLayer<T> : LayerBase<T>, ILayer<T>, IJitCompilable<T>, IDiagnosticsProvider, IWeightLoadable<T>, IDisposable

Type Parameters

T

The numeric type used for calculations, typically float or double.

Inheritance
SpatialPoolerLayer<T>
Implements
Inherited Members

Remarks

A spatial pooler is a key component in HTM systems that converts input patterns into sparse distributed representations (SDRs). It maps input space to a new representation that captures the spatial structure of the input while maintaining semantic similarity. The spatial pooler creates these representations by selecting a small subset of active columns based on their connection strengths to the input.

For Beginners: This layer helps convert input data into a format that's easier for neural networks to learn from.

Think of it like a translator that:

  • Takes dense input information (where many values can be active)
  • Converts it to a sparse representation (where only a few values are active)
  • Preserves the important patterns and relationships in the data

Benefits include:

  • Better handling of noisy or incomplete data
  • More efficient representation of information
  • Improved ability to recognize patterns

For example, when processing images, a spatial pooler might identify the most important features while ignoring background noise or variations that don't matter for classification.

Constructors

SpatialPoolerLayer(int, int, double)

Initializes a new instance of the SpatialPoolerLayer<T> class.

public SpatialPoolerLayer(int inputSize, int columnCount, double sparsityThreshold)

Parameters

inputSize int

The size of the input vector.

columnCount int

The number of columns in the spatial pooler.

sparsityThreshold double

The threshold that determines the sparsity of the output.

Remarks

This constructor creates a spatial pooler layer with the specified input size, column count, and sparsity threshold. It initializes the connection matrix with random values and sets up the input and output shapes for the layer.

For Beginners: This sets up a new spatial pooler layer with your specified settings.

When creating a spatial pooler, you need to specify:

  • inputSize: How many values are in each input (e.g., number of pixels in an image)
  • columnCount: How many feature detectors to create (more means more detailed representation)
  • sparsityThreshold: How selective the layer should be (higher means fewer active columns)

The constructor automatically initializes the connections between inputs and columns with random values that will be adjusted during learning.

Properties

SupportsGpuExecution

Gets a value indicating whether this layer supports GPU execution.

protected override bool SupportsGpuExecution { get; }

Property Value

bool

SupportsJitCompilation

Gets a value indicating whether this layer supports JIT compilation.

public override bool SupportsJitCompilation { get; }

Property Value

bool

Always true. SpatialPoolerLayer uses straight-through estimator for JIT compilation.

Remarks

JIT compilation for SpatialPooler uses a straight-through estimator for the threshold operation. The forward pass produces sparse binary activations (0 or 1), but gradients pass through unchanged during backpropagation. This enables differentiable training while maintaining the sparse output characteristics.

SupportsTraining

Gets a value indicating whether this layer supports training through backpropagation.

public override bool SupportsTraining { get; }

Property Value

bool

Always returns true as spatial pooler layers have trainable parameters.

Remarks

This property indicates that the spatial pooler layer can be trained. The layer contains trainable parameters (connection strengths) that are updated during the training process.

For Beginners: This property tells you that the layer can learn from data.

A value of true means:

  • The layer contains numbers (parameters) that can be adjusted during training
  • It will improve its performance as it sees more examples
  • It participates in the learning process of the neural network

Methods

Backward(Tensor<T>)

Performs the backward pass of the spatial pooler layer.

public override Tensor<T> Backward(Tensor<T> outputGradient)

Parameters

outputGradient Tensor<T>

The gradient of the loss with respect to the layer's output.

Returns

Tensor<T>

The gradient of the loss with respect to the layer's input.

Remarks

This method implements the backward pass of the spatial pooler layer, which is used during training to propagate error gradients back through the network. It computes the gradient of the loss with respect to the input by propagating the output gradient through the connection matrix.

For Beginners: This method is used during training to calculate how the layer's input should change to reduce errors.

During the backward pass:

  • The layer receives gradients (error signals) from the next layer
  • It uses the connection strengths to determine how these errors should be distributed to the input
  • Each input element receives a weighted sum of gradients based on its connections to the columns

This process allows error information to flow backward through the network during training, enabling all layers to learn from the overall network performance.

ExportComputationGraph(List<ComputationNode<T>>)

Exports the layer's computation graph for JIT compilation.

public override ComputationNode<T> ExportComputationGraph(List<ComputationNode<T>> inputNodes)

Parameters

inputNodes List<ComputationNode<T>>

List to populate with input computation nodes.

Returns

ComputationNode<T>

The output computation node representing the layer's operation.

Remarks

This method constructs a computation graph representation of the layer's forward pass that can be JIT compiled for faster inference. All layers MUST implement this method to support JIT compilation.

For Beginners: JIT (Just-In-Time) compilation converts the layer's operations into optimized native code for 5-10x faster inference.

To support JIT compilation, a layer must:

  1. Implement this method to export its computation graph
  2. Set SupportsJitCompilation to true
  3. Use ComputationNode and TensorOperations to build the graph

All layers are required to implement this method, even if they set SupportsJitCompilation = false.

Forward(Tensor<T>)

Performs the forward pass of the spatial pooler layer.

public override Tensor<T> Forward(Tensor<T> input)

Parameters

input Tensor<T>

The input tensor to process.

Returns

Tensor<T>

The output tensor after processing through the spatial pooler.

Remarks

This method implements the forward pass of the spatial pooler layer. It computes the overlap between the input and each column's connections, and then activates columns whose overlap exceeds the sparsity threshold. The result is a sparse representation where active columns are set to 1 and inactive columns are set to 0.

For Beginners: This method processes data through the spatial pooler.

During the forward pass:

  • The input tensor is converted to a vector for easier processing
  • For each column, the layer calculates how strongly it responds to the input
  • Columns with a strong enough response (above the sparsity threshold) are activated (set to 1)
  • All other columns remain inactive (set to 0)
  • The result is a sparse output where only a small percentage of columns are active

This sparse representation helps the network focus on the most important features and ignore noise or irrelevant details.

ForwardGpu(params IGpuTensor<T>[])

Performs the GPU-accelerated forward pass for the spatial pooler layer.

public override IGpuTensor<T> ForwardGpu(params IGpuTensor<T>[] inputs)

Parameters

inputs IGpuTensor<T>[]

The GPU tensor inputs. First element is the input activation.

Returns

IGpuTensor<T>

A GPU tensor containing the sparse binary output.

Remarks

The spatial pooler converts input patterns into sparse distributed representations. This method processes the input using GPU operations for matrix multiplication and thresholding.

GetParameters()

Gets all trainable parameters of the layer as a single vector.

public override Vector<T> GetParameters()

Returns

Vector<T>

A vector containing all trainable parameters.

Remarks

This method retrieves all trainable parameters of the layer (connection strengths) and combines them into a single vector. This is useful for optimization algorithms that operate on all parameters at once, or for saving and loading model weights.

For Beginners: This method collects all the learnable values from the layer into a single list.

The parameters:

  • Are the connection strengths between input elements and columns
  • Are converted from a matrix to a single long list (vector)
  • Can be used to save the state of the layer or apply optimization techniques

This method is useful for:

  • Saving the model to disk
  • Loading parameters from a previously trained model
  • Advanced optimization techniques that need access to all parameters

Learn(Vector<T>)

Performs a learning step on the spatial pooler using the provided input.

public void Learn(Vector<T> input)

Parameters

input Vector<T>

The input vector to learn from.

Remarks

This method implements the learning algorithm for the spatial pooler. It updates the connection strengths based on the current input and output. For active columns, connections to active input elements are strengthened and connections to inactive input elements are weakened. For inactive columns, a small boost is applied to help them become active for other patterns.

For Beginners: This method teaches the spatial pooler to recognize patterns.

During learning:

  1. The input is processed to determine which columns activate
  2. For active columns:
    • Connections to active input elements are strengthened
    • Connections to inactive input elements are weakened
  3. For inactive columns:
    • A small boost is applied to connections to active input elements
    • This gives inactive columns a better chance of activating in the future
  4. All connections are normalized to keep their values balanced

Over time, this process helps the spatial pooler learn to recognize important patterns in the input data and represent them effectively.

ResetState()

Resets the internal state of the spatial pooler layer.

public override void ResetState()

Remarks

This method resets the internal state of the spatial pooler layer, including the cached inputs and outputs. This is useful when starting to process a new batch or when implementing stateful networks that need to be reset between sequences.

For Beginners: This method clears the layer's memory to start fresh.

When resetting the state:

  • Stored inputs and outputs from previous passes are cleared
  • The connection strengths are not affected, so the layer retains what it has learned

This is important for:

  • Processing a new batch of unrelated data
  • Preventing information from one batch affecting another
  • Starting a new training episode

Think of it like clearing your short-term memory while keeping your long-term knowledge intact.

SetParameters(Vector<T>)

Sets the trainable parameters of the layer from a single vector.

public override void SetParameters(Vector<T> parameters)

Parameters

parameters Vector<T>

A vector containing all parameters to set.

Remarks

This method sets the trainable parameters of the layer (connection strengths) from a single vector. It expects the vector to contain the parameters in the same order as they are retrieved by GetParameters(). This is useful for loading saved model weights or for implementing optimization algorithms that operate on all parameters at once.

For Beginners: This method updates all the connections in the layer from a single list.

When setting parameters:

  • The input must be a vector with exactly the right number of values
  • The values are distributed back into the connection matrix
  • The order must match how they were stored in GetParameters()

This method is useful for:

  • Loading a previously saved model
  • Transferring parameters from another model
  • Testing different parameter values

An error is thrown if the input vector doesn't have the expected number of parameters.

Exceptions

ArgumentException

Thrown when the parameters vector has incorrect length.

UpdateParameters(T)

Updates the parameters of the layer using the calculated gradients.

public override void UpdateParameters(T learningRate)

Parameters

learningRate T

The learning rate to use for the parameter updates.

Remarks

This method updates the connection strengths based on the most recent input and output from a forward pass. It strengthens connections between active input elements and active columns, then normalizes the connections to maintain balanced total connection strength.

For Beginners: This method updates the layer's connections during standard neural network training.

When updating parameters:

  • If there's no record of previous input/output, the method does nothing
  • Otherwise, it strengthens connections between active inputs and active columns
  • The learning rate controls how big each update step is
  • Finally, connections are normalized to keep their values balanced

This method is similar to the Learn method but is designed to work within the standard neural network training framework, where gradient information flows through the network.

Edit this page