Table of Contents

Class RecurrentNeuralNetwork<T>

Namespace
AiDotNet.NeuralNetworks
Assembly
AiDotNet.dll

Represents a Recurrent Neural Network, which is a type of neural network designed to process sequential data by maintaining an internal state.

public class RecurrentNeuralNetwork<T> : NeuralNetworkBase<T>, INeuralNetworkModel<T>, INeuralNetwork<T>, IFullModel<T, Tensor<T>, Tensor<T>>, IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>, IModelSerializer, ICheckpointableModel, IParameterizable<T, Tensor<T>, Tensor<T>>, IFeatureAware, IFeatureImportance<T>, ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>, IGradientComputable<T, Tensor<T>, Tensor<T>>, IJitCompilable<T>, IInterpretableModel<T>, IInputGradientComputable<T>, IDisposable

Type Parameters

T

The numeric type used for calculations, typically float or double.

Inheritance
RecurrentNeuralNetwork<T>
Implements
IFullModel<T, Tensor<T>, Tensor<T>>
IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>
IParameterizable<T, Tensor<T>, Tensor<T>>
ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>
IGradientComputable<T, Tensor<T>, Tensor<T>>
Inherited Members
Extension Methods

Remarks

A Recurrent Neural Network (RNN) is a class of neural networks that can process sequential data by maintaining an internal state (memory) that captures information about previous inputs. Unlike traditional feedforward neural networks, RNNs have connections that form directed cycles, allowing information to persist from one step to the next. This architectural feature makes RNNs particularly well-suited for tasks involving sequential data, such as time series prediction, natural language processing, and speech recognition.

For Beginners: A Recurrent Neural Network is a type of neural network that has memory.

Think of it like reading a book:

  • A standard neural network looks at each word in isolation
  • An RNN remembers what it read earlier to understand the current word

For example, in the sentence "The clouds were dark, so I took my umbrella," an RNN understands that "I took my umbrella" is related to "The clouds were dark" because it remembers the earlier part of the sentence.

This memory makes RNNs good at:

  • Processing sequences like sentences, time series, or music
  • Understanding context and relationships over time
  • Predicting what comes next in a sequence

Common applications include text generation, translation, speech recognition, and stock price prediction - all tasks where what happened before affects how you interpret what's happening now.

Constructors

RecurrentNeuralNetwork(NeuralNetworkArchitecture<T>, double, ILossFunction<T>?)

Initializes a new instance of the RecurrentNeuralNetwork<T> class with the specified architecture.

public RecurrentNeuralNetwork(NeuralNetworkArchitecture<T> architecture, double learningRate = 0.01, ILossFunction<T>? lossFunction = null)

Parameters

architecture NeuralNetworkArchitecture<T>

The neural network architecture to use for the RNN.

learningRate double
lossFunction ILossFunction<T>

Remarks

This constructor creates a new Recurrent Neural Network with the specified architecture. It initializes the network layers based on the architecture, or creates default RNN layers if no specific layers are provided.

For Beginners: This sets up the Recurrent Neural Network with its basic structure.

When creating a new RNN:

  • The architecture defines what the network looks like - how many neurons it has, how they're connected, etc.
  • The constructor prepares the network by either:
    • Using the specific layers provided in the architecture, or
    • Creating default layers designed for processing sequences if none are specified

This is like setting up a specialized calculator before you start using it. The architecture determines what kind of calculations it can perform and how it will process information.

Methods

CreateNewInstance()

Creates a new instance of the recurrent neural network with the same configuration.

protected override IFullModel<T, Tensor<T>, Tensor<T>> CreateNewInstance()

Returns

IFullModel<T, Tensor<T>, Tensor<T>>

A new instance of RecurrentNeuralNetwork<T> with the same configuration as the current instance.

Remarks

This method creates a new recurrent neural network that has the same configuration as the current instance. It's used for model persistence, cloning, and transferring the model's configuration to new instances. The new instance will have the same architecture and learning rate as the original, but will not share parameter values unless they are explicitly copied after creation.

For Beginners: This method makes a fresh copy of the current model with the same settings.

It's like creating a blueprint copy of your network that can be used to:

  • Save your model's settings
  • Create a new identical model
  • Transfer your model's configuration to another system

This is useful when you want to:

  • Create multiple similar recurrent neural networks
  • Save a model's configuration for later use
  • Reset a model while keeping its settings

Note that while the settings are copied, the learned parameters (like the weights that determine how the network processes sequences) are not automatically transferred, so the new instance will need training or parameter copying to match the performance of the original.

DeserializeNetworkSpecificData(BinaryReader)

Deserializes network-specific data for the Recurrent Neural Network.

protected override void DeserializeNetworkSpecificData(BinaryReader reader)

Parameters

reader BinaryReader

The BinaryReader to read the data from.

Remarks

This method reads the specific configuration and state of the RNN from a binary stream. It reconstructs the RNN-specific parameters to match the state of the network when it was serialized.

For Beginners: This method loads the unique settings of your RNN.

It reads:

  • The size of the hidden state (which determines the network's memory capacity)
  • The length of sequences the network is designed to handle
  • Any other RNN-specific parameters

Loading these details allows you to recreate the exact same network structure that was previously saved. It's like following a recipe to recreate a dish exactly as it was made before.

GetModelMetadata()

Gets metadata about the Recurrent Neural Network model.

public override ModelMetadata<T> GetModelMetadata()

Returns

ModelMetadata<T>

A ModelMetaData object containing information about the model.

Remarks

This method returns metadata that describes the RNN, including its type, architecture details, and other relevant information. This metadata can be useful for model management, documentation, and versioning.

For Beginners: This provides a summary of your network's setup and characteristics.

The metadata includes:

  • The type of model (Recurrent Neural Network)
  • How many inputs and outputs the network has
  • A description of the network's structure
  • Additional information specific to RNNs

This is useful for:

  • Keeping track of different versions of your model
  • Comparing different network configurations
  • Documenting your model's setup for future reference

Think of it like a spec sheet for a car, listing all its important features and capabilities.

InitializeLayers()

Initializes the neural network layers based on the provided architecture or default configuration.

protected override void InitializeLayers()

Remarks

This method sets up the neural network layers for the Recurrent Neural Network. If the architecture provides specific layers, those are used. Otherwise, a default configuration optimized for RNNs is created. In a typical RNN, this involves creating recurrent layers that maintain state between processing steps.

For Beginners: This method sets up the building blocks of the neural network.

When initializing layers:

  • If the user provided specific layers, those are used
  • Otherwise, default layers suitable for recurrent networks are created automatically
  • The system checks that any custom layers will work properly with the RNN

A typical RNN has specialized layers that include:

  • Input layers that accept sequences
  • Recurrent layers that maintain memory of previous inputs
  • Output layers that produce predictions

These layers work together to process sequential data while maintaining context from previous steps.

Predict(Tensor<T>)

Makes a prediction using the current state of the Recurrent Neural Network.

public override Tensor<T> Predict(Tensor<T> input)

Parameters

input Tensor<T>

The input tensor to make a prediction for.

Returns

Tensor<T>

The predicted output tensor after passing through all layers of the network.

Remarks

This method performs a forward pass through the network, transforming the input data through each layer to produce a final prediction. It includes input validation to ensure the provided tensor matches the expected input shape of the network.

For Beginners: This is how the network processes new data to make predictions.

The prediction process:

  1. Checks if the input data is valid and has the correct shape
  2. Passes the input through each layer of the network
  3. Each layer transforms the data, with recurrent layers using their internal state
  4. The final layer produces the network's prediction

Think of it like a game of telephone, where each person (layer) passes along a message, but also remembers previous messages to provide context.

Exceptions

ArgumentNullException

Thrown when the input tensor is null.

ArgumentException

Thrown when the input tensor has incorrect dimensions.

SerializeNetworkSpecificData(BinaryWriter)

Serializes network-specific data for the Recurrent Neural Network.

protected override void SerializeNetworkSpecificData(BinaryWriter writer)

Parameters

writer BinaryWriter

The BinaryWriter to write the data to.

Remarks

This method writes the specific configuration and state of the RNN to a binary stream. It includes RNN-specific parameters that are essential for later reconstruction of the network.

For Beginners: This method saves the unique settings of your RNN.

It writes:

  • The size of the hidden state (which determines the network's memory capacity)
  • The length of sequences the network is designed to handle
  • Any other RNN-specific parameters

Saving these details allows you to recreate the exact same network structure later. It's like writing down a recipe so you can make the same dish again in the future.

Train(Tensor<T>, Tensor<T>)

Trains the Recurrent Neural Network using the provided input and expected output.

public override void Train(Tensor<T> input, Tensor<T> expectedOutput)

Parameters

input Tensor<T>

The input tensor used for training.

expectedOutput Tensor<T>

The expected output tensor for the given input.

Remarks

This method implements the training process for the RNN. It performs a forward pass, calculates the error between the network's prediction and the expected output, and then backpropagates this error to adjust the network's parameters.

For Beginners: This is how the network learns from examples.

The training process:

  1. Takes a sequence of inputs and their correct answers (expected outputs)
  2. Makes predictions using the current network state
  3. Compares the predictions to the correct answers to calculate the error
  4. Uses this error to adjust the network's internal settings (backpropagation through time)

It's like learning to predict weather patterns:

  • The network looks at a series of past weather conditions (input)
  • It tries to predict the next day's weather (output)
  • It compares its prediction to what actually happened
  • It adjusts its understanding based on its mistakes

Exceptions

ArgumentNullException

Thrown when either input or expectedOutput is null.

UpdateParameters(Vector<T>)

Updates the parameters of the recurrent neural network layers.

public override void UpdateParameters(Vector<T> parameters)

Parameters

parameters Vector<T>

The vector of parameter updates to apply.

Remarks

This method updates the parameters of each layer in the recurrent neural network based on the provided parameter updates. The parameters vector is divided into segments corresponding to each layer's parameter count, and each segment is applied to its respective layer. In an RNN, these parameters typically include weights for both the input connections and the recurrent connections that maintain state.

For Beginners: This method updates how the RNN makes decisions based on training.

During training:

  • The network learns by adjusting its internal parameters
  • This method applies those adjustments
  • Each layer gets the portion of updates meant specifically for it

For an RNN, these adjustments might include:

  • How much attention to pay to the current input
  • How much to rely on memory of previous inputs
  • How to combine different pieces of information

These parameter updates help the network learn to:

  • Recognize important patterns in sequences
  • Decide what information is worth remembering
  • Make better predictions based on both current and past information
Edit this page