Table of Contents

Class RBFLayer<T>

Namespace
AiDotNet.NeuralNetworks.Layers
Assembly
AiDotNet.dll

Represents a Radial Basis Function (RBF) layer for neural networks.

public class RBFLayer<T> : LayerBase<T>, ILayer<T>, IJitCompilable<T>, IDiagnosticsProvider, IWeightLoadable<T>, IDisposable

Type Parameters

T

The numeric type used for calculations, typically float or double.

Inheritance
RBFLayer<T>
Implements
Inherited Members

Remarks

The RBF layer implements a type of artificial neural network that uses radial basis functions as activation functions. Each neuron in this layer has a center point in the input space and responds most strongly to inputs near that center. The response decreases as the distance from the center increases, controlled by the width parameter of each neuron.

For Beginners: This layer works like a collection of specialized detectors.

Think of each neuron in this layer as a spotlight:

  • Each spotlight has a specific location (center) in the input space
  • Each spotlight has a certain brightness range (width)
  • When input comes in, spotlights that are close to that input light up brightly
  • Spotlights far from the input barely light up at all

For example, if you're recognizing handwritten digits:

  • One spotlight might be positioned to detect curved lines (like in "8")
  • Another might detect vertical lines (like in "1")
  • When a "3" comes in, the spotlights for curves light up strongly, while others stay dim

This layer is particularly good at classification problems and function approximation where the relationship between inputs and outputs is complex or non-linear.

Constructors

RBFLayer(int, int, IRadialBasisFunction<T>)

Initializes a new instance of the RBFLayer<T> class with specified dimensions and radial basis function.

public RBFLayer(int inputSize, int outputSize, IRadialBasisFunction<T> rbf)

Parameters

inputSize int

The size of the input to the layer.

outputSize int

The size of the output from the layer (number of RBF neurons).

rbf IRadialBasisFunction<T>

The radial basis function to use for computing neuron activations.

Remarks

This constructor creates a new RBF layer with the specified dimensions and radial basis function. The centers are initialized randomly around the origin, and the widths are initialized with random values between 0 and 1. The scale of the random initialization for centers depends on the layer dimensions.

For Beginners: This creates a new RBF layer for your neural network.

When you create this layer, you specify:

  • inputSize: How many numbers come into the layer (features of your data)
  • outputSize: How many RBF neurons to create (more neurons can capture more complex patterns)
  • rbf: The specific mathematical function that determines how neurons respond to input

For example, if you're analyzing images with 784 pixels and want 100 different pattern detectors, you might use inputSize=784 and outputSize=100.

Common radial basis functions include Gaussian (bell-shaped), Multiquadric, and Inverse Quadratic. Each creates a different pattern of responsiveness around the neuron centers.

Properties

SupportsGpuExecution

Gets whether this layer has a GPU execution implementation for inference.

protected override bool SupportsGpuExecution { get; }

Property Value

bool

Remarks

Override this to return true when the layer implements ForwardGpu(params IGpuTensor<T>[]). The actual CanExecuteOnGpu property combines this with engine availability.

For Beginners: This flag indicates if the layer has GPU code for the forward pass. Set this to true in derived classes that implement ForwardGpu.

SupportsJitCompilation

Gets whether this layer supports JIT compilation.

public override bool SupportsJitCompilation { get; }

Property Value

bool

True if the layer can be JIT compiled, false otherwise.

Remarks

This property indicates whether the layer has implemented ExportComputationGraph() and can benefit from JIT compilation. All layers MUST implement this property.

For Beginners: JIT compilation can make inference 5-10x faster by converting the layer's operations into optimized native code.

Layers should return false if they:

  • Have not yet implemented a working ExportComputationGraph()
  • Use dynamic operations that change based on input data
  • Are too simple to benefit from JIT compilation

When false, the layer will use the standard Forward() method instead.

SupportsTraining

Gets a value indicating whether this layer supports training.

public override bool SupportsTraining { get; }

Property Value

bool

Always true for RBF layers, indicating that the layer can be trained through backpropagation.

Remarks

This property indicates that the RBF layer has trainable parameters (centers and widths) that can be optimized during the training process using backpropagation. The gradients of these parameters are calculated during the backward pass and used to update the parameters.

For Beginners: This property tells you if the layer can learn from data.

A value of true means:

  • The layer has values (centers and widths) that can be adjusted during training
  • It will improve its performance as it sees more data
  • It participates in the learning process of the neural network

When you train a neural network containing this layer, the centers and widths will automatically adjust to better match the patterns in your specific data.

Methods

Backward(Tensor<T>)

Performs the backward pass of the RBF layer.

public override Tensor<T> Backward(Tensor<T> outputGradient)

Parameters

outputGradient Tensor<T>

The gradient of the loss with respect to the layer's output.

Returns

Tensor<T>

The gradient of the loss with respect to the layer's input.

Remarks

This method implements the backward pass of the RBF layer, which is used during training to propagate error gradients back through the network. It calculates the gradients of the loss with respect to the centers and widths (to update the layer's parameters) and with respect to the input (to propagate back to previous layers).

For Beginners: This method is used during training to calculate how the layer should change to reduce errors.

During the backward pass:

  1. The error gradient from the next layer is received
  2. The layer calculates how each center should move to reduce the error
  3. The layer calculates how each width should change to reduce the error
  4. The layer calculates how the previous layer's output should change

This is like saying "Based on the mistakes we made, how should we adjust our pattern detectors to be more accurate next time?" The gradients tell us both how to update this layer and how to guide the previous layers.

Exceptions

InvalidOperationException

Thrown when backward is called before forward.

ExportComputationGraph(List<ComputationNode<T>>)

Exports the layer's computation graph for JIT compilation.

public override ComputationNode<T> ExportComputationGraph(List<ComputationNode<T>> inputNodes)

Parameters

inputNodes List<ComputationNode<T>>

List to populate with input computation nodes.

Returns

ComputationNode<T>

The output computation node representing the layer's operation.

Remarks

This method constructs a computation graph representation of the layer's forward pass that can be JIT compiled for faster inference. All layers MUST implement this method to support JIT compilation.

For Beginners: JIT (Just-In-Time) compilation converts the layer's operations into optimized native code for 5-10x faster inference.

To support JIT compilation, a layer must:

  1. Implement this method to export its computation graph
  2. Set SupportsJitCompilation to true
  3. Use ComputationNode and TensorOperations to build the graph

All layers are required to implement this method, even if they set SupportsJitCompilation = false.

Forward(Tensor<T>)

Performs the forward pass of the RBF layer.

public override Tensor<T> Forward(Tensor<T> input)

Parameters

input Tensor<T>

The input tensor to process.

Returns

Tensor<T>

The output tensor after RBF processing.

Remarks

This method implements the forward pass of the RBF layer. For each neuron (center), it calculates the Euclidean distance between the input vector and the center, then applies the radial basis function to this distance to produce the neuron's activation. The input and output are cached for use during the backward pass.

For Beginners: This method processes your data through the RBF neurons.

During the forward pass:

  1. For each input example, the layer measures how far it is from each neuron's center
  2. The distance is plugged into the radial basis function (like a mathematical formula)
  3. The result determines how strongly each neuron activates
  4. Neurons with centers close to the input activate strongly; distant ones activate weakly

This is like asking "How similar is this input to each of my known patterns?" The output tells you the similarity scores for each pattern.

The layer saves the input and output for later use during training.

ForwardGpu(params IGpuTensor<T>[])

Performs the forward pass of the layer on GPU.

public override IGpuTensor<T> ForwardGpu(params IGpuTensor<T>[] inputs)

Parameters

inputs IGpuTensor<T>[]

The GPU-resident input tensor(s).

Returns

IGpuTensor<T>

The GPU-resident output tensor.

Remarks

This method performs the layer's forward computation entirely on GPU. The input and output tensors remain in GPU memory, avoiding expensive CPU-GPU transfers.

For Beginners: This is like Forward() but runs on the graphics card.

The key difference:

  • Forward() uses CPU tensors that may be copied to/from GPU
  • ForwardGpu() keeps everything on GPU the whole time

Override this in derived classes that support GPU acceleration.

Exceptions

NotSupportedException

Thrown when the layer does not support GPU execution.

GetParameters()

Gets all trainable parameters of the RBF layer as a single vector.

public override Vector<T> GetParameters()

Returns

Vector<T>

A vector containing all trainable parameters (centers and widths).

Remarks

This method retrieves all trainable parameters (centers and widths) of the RBF layer as a single vector. The centers are stored first, followed by the widths. This is useful for optimization algorithms that operate on all parameters at once, or for saving and loading model weights.

For Beginners: This method collects all the learnable values from the RBF layer.

The parameters:

  • Are the centers and widths that the RBF layer learns during training
  • Control where and how widely each neuron responds to inputs
  • Are returned as a single list (vector)

This is useful for:

  • Saving the model to disk
  • Loading parameters from a previously trained model
  • Advanced optimization techniques that need access to all parameters

The centers are stored first in the vector, followed by all the width values.

ResetState()

Resets the internal state of the RBF layer.

public override void ResetState()

Remarks

This method resets the internal state of the RBF layer, including the cached inputs and outputs from the forward pass, and the gradients from the backward pass. This is useful when starting to process a new sequence or batch of data.

For Beginners: This method clears the layer's memory to start fresh.

When resetting the state:

  • Stored inputs and outputs from previous calculations are cleared
  • Calculated gradients are cleared
  • The layer forgets any information from previous batches

This is important for:

  • Processing a new, unrelated batch of data
  • Preventing information from one batch affecting another
  • Starting a new training episode

The centers and widths (the learned parameters) are not reset, only the temporary state information.

SetParameters(Vector<T>)

Sets the trainable parameters of the RBF layer.

public override void SetParameters(Vector<T> parameters)

Parameters

parameters Vector<T>

A vector containing all parameters (centers and widths) to set.

Remarks

This method sets the trainable parameters (centers and widths) of the RBF layer from a single vector. The vector should contain the center values first, followed by the width values. This is useful for loading saved model weights or for implementing optimization algorithms that operate on all parameters at once.

For Beginners: This method updates all the centers and widths in the RBF layer.

When setting parameters:

  • The input must be a vector with the correct total length
  • The first part of the vector is used for the centers (positions of the neurons)
  • The second part of the vector is used for the widths (how broadly each neuron responds)

This is useful for:

  • Loading a previously saved model
  • Transferring parameters from another model
  • Testing different parameter values

An error is thrown if the input vector doesn't have the expected number of parameters.

Exceptions

ArgumentException

Thrown when the parameters vector has incorrect length.

UpdateParameters(T)

Updates the parameters of the RBF layer using the calculated gradients.

public override void UpdateParameters(T learningRate)

Parameters

learningRate T

The learning rate to use for the parameter updates.

Remarks

This method updates the centers and widths of the RBF layer based on the gradients calculated during the backward pass. The learning rate controls the size of the parameter updates. This method should be called after the backward pass to apply the calculated updates.

For Beginners: This method updates the layer's internal values during training.

When updating parameters:

  1. The center positions are adjusted based on their gradients
  2. The widths are adjusted based on their gradients
  3. The learning rate controls how big each update step is

Imagine each RBF neuron as a spotlight:

  • Updating the centers moves where the spotlight is pointing
  • Updating the widths changes how broad or narrow the spotlight beam is

Smaller learning rates mean slower but more stable learning, while larger learning rates mean faster but potentially unstable learning.

Exceptions

InvalidOperationException

Thrown when UpdateParameters is called before Backward.

Edit this page