Table of Contents

Class ExtremeLearningMachine<T>

Namespace
AiDotNet.NeuralNetworks
Assembly
AiDotNet.dll

Represents an Extreme Learning Machine (ELM), a type of feedforward neural network with a unique training approach.

public class ExtremeLearningMachine<T> : NeuralNetworkBase<T>, INeuralNetworkModel<T>, INeuralNetwork<T>, IFullModel<T, Tensor<T>, Tensor<T>>, IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>, IModelSerializer, ICheckpointableModel, IParameterizable<T, Tensor<T>, Tensor<T>>, IFeatureAware, IFeatureImportance<T>, ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>, IGradientComputable<T, Tensor<T>, Tensor<T>>, IJitCompilable<T>, IInterpretableModel<T>, IInputGradientComputable<T>, IDisposable

Type Parameters

T

The numeric type used for calculations, typically float or double.

Inheritance
ExtremeLearningMachine<T>
Implements
IFullModel<T, Tensor<T>, Tensor<T>>
IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>
IParameterizable<T, Tensor<T>, Tensor<T>>
ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>
IGradientComputable<T, Tensor<T>, Tensor<T>>
Inherited Members
Extension Methods

Remarks

An Extreme Learning Machine is a special type of single-hidden-layer feedforward neural network that uses a non-iterative training approach. Unlike traditional neural networks that use backpropagation to adjust all weights, ELMs randomly assign the weights between the input and hidden layer and only train the weights between the hidden and output layer. This is done analytically using a pseudo-inverse operation rather than through iterative optimization, resulting in extremely fast training times while maintaining good generalization performance.

For Beginners: An Extreme Learning Machine is like a neural network on fast-forward.

Think of it like building a bridge:

  • Traditional neural networks carefully adjust every piece of the bridge (slow but thorough)
  • ELMs randomly set up most of the bridge, then only carefully adjust the final section
  • This approach is much faster but can still create a surprisingly strong bridge

The "extreme" part refers to its extremely fast training time. While traditional networks might take hours or days to train, ELMs can often be trained in seconds or minutes, making them useful for applications where training speed is critical.

Constructors

ExtremeLearningMachine(NeuralNetworkArchitecture<T>, int, ILossFunction<T>?)

Initializes a new instance of the ExtremeLearningMachine<T> class with the specified architecture and hidden layer size.

public ExtremeLearningMachine(NeuralNetworkArchitecture<T> architecture, int hiddenLayerSize, ILossFunction<T>? lossFunction = null)

Parameters

architecture NeuralNetworkArchitecture<T>

The neural network architecture configuration.

hiddenLayerSize int

The number of neurons in the hidden layer.

lossFunction ILossFunction<T>

Remarks

This constructor creates a new Extreme Learning Machine with the specified architecture and hidden layer size. The hidden layer size is a key parameter that determines the capacity and learning ability of the ELM. After setting this parameter, the constructor initializes the network layers.

For Beginners: This sets up a new ELM with a specific number of pattern detectors.

When creating a new ELM:

  • The architecture defines the overall structure (input and output sizes)
  • The hiddenLayerSize determines how many pattern detectors the network will have
  • The constructor sets up the initial structure, but doesn't train the network yet

Think of it like assembling a team of a specific size to look for patterns, where each team member will be randomly assigned what to look for.

Methods

CreateNewInstance()

Creates a new instance of the ExtremeLearningMachine with the same configuration as the current instance.

protected override IFullModel<T, Tensor<T>, Tensor<T>> CreateNewInstance()

Returns

IFullModel<T, Tensor<T>, Tensor<T>>

A new ExtremeLearningMachine instance with the same architecture and hidden layer size as the current instance.

Remarks

This method creates a new instance of the ExtremeLearningMachine with the same architecture and hidden layer size as the current instance. This is useful for model cloning, ensemble methods, or cross-validation scenarios where multiple instances of the same model with identical configurations are needed.

For Beginners: This method creates a fresh copy of the ELM's blueprint.

When you need multiple versions of the same type of ELM with identical settings:

  • This method creates a new, empty ELM with the same configuration
  • It's like making a copy of a recipe before you start cooking
  • The new ELM has the same structure but no trained data
  • This is useful for techniques that need multiple models, like ensemble methods

For example, when training multiple ELMs on different subsets of data, you'd want each one to have the same architecture and hidden layer size.

DeserializeNetworkSpecificData(BinaryReader)

Deserializes Extreme Learning Machine-specific data from a binary reader.

protected override void DeserializeNetworkSpecificData(BinaryReader reader)

Parameters

reader BinaryReader

The BinaryReader to read the data from.

Remarks

This method reads ELM-specific configuration data from a binary stream. It retrieves properties such as the hidden layer size and the weights of all layers. After reading this data, the ELM's state is fully restored to what it was when saved.

For Beginners: This restores the special configuration of your ELM from saved data.

It's like following the recipe to rebuild your ELM exactly as it was:

  • Setting the hidden layer to the right size
  • Restoring the random weights for the input-to-hidden connections
  • Restoring the trained weights for the hidden-to-output connections

By reading these details, the ELM can be reconstructed exactly as it was when it was saved, preserving all its behavior and learned patterns.

GetModelMetadata()

Gets metadata about the Extreme Learning Machine model.

public override ModelMetadata<T> GetModelMetadata()

Returns

ModelMetadata<T>

A ModelMetaData object containing information about the model.

Remarks

This method returns metadata about the ELM, including its model type, hidden layer size, and additional configuration information. This metadata is useful for model management and for generating reports about the model's structure and configuration.

For Beginners: This provides a summary of your ELM's configuration.

The metadata includes:

  • The type of model (Extreme Learning Machine)
  • The size of the hidden layer
  • Information about the layers and their configurations
  • Serialized data that can be used to save and reload the model

Think of it like a label that describes the specific type and characteristics of your neural network. This is useful for organizing and managing multiple models.

InitializeLayers()

Initializes the layers of the Extreme Learning Machine based on the architecture.

protected override void InitializeLayers()

Remarks

This method sets up the layers of the Extreme Learning Machine. If custom layers are provided in the architecture, those layers are used. Otherwise, default layers are created based on the architecture's specifications and the specified hidden layer size. A typical ELM consists of an input layer, a hidden layer with random weights, and an output layer.

For Beginners: This builds the structure of the neural network.

When initializing the layers:

  • If you've specified your own custom layers, the network will use those
  • If not, the network will create a standard set of layers suitable for an ELM:
    1. A random input-to-hidden layer with fixed weights
    2. A non-linear activation function (typically sigmoid or tanh)
    3. A trainable hidden-to-output layer

The most important part is that only the final layer (hidden-to-output) will be trained - the other layers will keep their random weights.

Predict(Tensor<T>)

Makes a prediction using the Extreme Learning Machine.

public override Tensor<T> Predict(Tensor<T> input)

Parameters

input Tensor<T>

The input tensor to process.

Returns

Tensor<T>

The output tensor containing the prediction.

Remarks

This method passes the input data through the Extreme Learning Machine network to make a prediction. It sequentially processes the input through all layers of the network, which typically includes a fixed random projection from the input layer to the hidden layer, followed by a nonlinear activation function, and finally the analytically trained weights from the hidden layer to the output layer.

For Beginners: This is how the ELM processes new data to make predictions.

The prediction process works in three stages:

  1. Input data is projected through random weights to the hidden layer
  2. A nonlinear activation function (like sigmoid) is applied to these projections
  3. The resulting hidden layer activations are multiplied by the trained output weights

Unlike traditional neural networks, the first step uses fixed random weights that were never trained, which is part of what makes ELMs so unique and fast.

SerializeNetworkSpecificData(BinaryWriter)

Serializes Extreme Learning Machine-specific data to a binary writer.

protected override void SerializeNetworkSpecificData(BinaryWriter writer)

Parameters

writer BinaryWriter

The BinaryWriter to write the data to.

Remarks

This method writes ELM-specific configuration data to a binary stream. It includes properties such as the hidden layer size and the weights of all layers. This data is needed to reconstruct the ELM when deserializing.

For Beginners: This saves the special configuration of your ELM.

It's like writing down the recipe for how your specific ELM was built:

  • How many hidden neurons it has
  • The random weights used in the input-to-hidden connections
  • The trained weights used in the hidden-to-output connections

This allows you to save the model and reload it later, without having to retrain it.

Train(Tensor<T>, Tensor<T>)

Trains the Extreme Learning Machine on a single batch of data.

public override void Train(Tensor<T> input, Tensor<T> expectedOutput)

Parameters

input Tensor<T>

The input tensor for training.

expectedOutput Tensor<T>

The expected output tensor for the given input.

Remarks

This method implements the ELM training algorithm, which is fundamentally different from traditional neural network training. Instead of using iterative optimization with backpropagation, ELM fixes the weights from input to hidden layer with random values and analytically calculates the optimal weights from hidden to output layer using a Moore-Penrose pseudo-inverse operation. This allows for extremely fast training compared to traditional neural networks.

For Beginners: This trains the ELM in one single step, which is much faster than traditional neural networks.

The ELM training process is unique:

  1. The weights from input to hidden layer stay fixed at their random initial values
  2. The input data is projected through these fixed random weights to get hidden layer activations
  3. The optimal output weights are calculated using linear algebra (pseudo-inverse) in one step

This is like solving a system of equations directly rather than making many small adjustments, which is why ELMs can train thousands of times faster than traditional neural networks.

TrainWithRegularization(Tensor<T>, Tensor<T>, double)

Trains the ELM using regularized least squares for improved generalization.

public void TrainWithRegularization(Tensor<T> input, Tensor<T> expectedOutput, double regularizationFactor = 0.01)

Parameters

input Tensor<T>

The input tensor for training.

expectedOutput Tensor<T>

The expected output tensor for the given input.

regularizationFactor double

The regularization factor (lambda) to use. Default is 0.01.

Remarks

This method implements a regularized version of the ELM training algorithm. It adds a regularization term to the pseudoinverse calculation, which helps prevent overfitting. The formula becomes: OutputWeights = (H^T * H + λI)^(-1) * H^T * T, where λ is the regularization factor, I is the identity matrix, H is the hidden layer activations, and T is the target output.

For Beginners: This is a more robust training method that helps prevent overfitting.

Regularization is like adding a penalty for overly complex solutions:

  • It keeps the weights from becoming too large or specialized to the training data
  • It helps the model generalize better to new, unseen data
  • The regularization factor controls how strong this penalty is

This method still trains in one fast step (unlike traditional neural networks), but produces weights that typically work better on new data.

UpdateParameters(Vector<T>)

Updates the parameters of all layers in the Extreme Learning Machine.

public override void UpdateParameters(Vector<T> parameters)

Parameters

parameters Vector<T>

A vector containing the parameters to update all layers with.

Remarks

This method is not implemented for Extreme Learning Machines because they do not use traditional parameter updates. In an ELM, the input-to-hidden weights are randomly generated and remain fixed, while the hidden-to-output weights are calculated analytically in a single step rather than through iterative updates.

For Beginners: This method always throws an error because ELMs don't train like regular neural networks.

Extreme Learning Machines are different from standard neural networks:

  • They don't use backpropagation or gradient descent
  • Most of their weights stay fixed (unchangeable) after random initialization
  • The output weights are calculated in one step, not iteratively updated

If you try to update parameters like in a regular neural network, you'll get an error because this isn't how ELMs work.

Exceptions

NotImplementedException

Always thrown because ELM does not support traditional parameter updates.

Edit this page