Table of Contents

Class OctonionLinearLayer<T>

Namespace
AiDotNet.NeuralNetworks.Layers
Assembly
AiDotNet.dll

Represents a fully connected layer using octonion-valued weights and inputs.

public class OctonionLinearLayer<T> : LayerBase<T>, ILayer<T>, IJitCompilable<T>, IDiagnosticsProvider, IWeightLoadable<T>, IDisposable

Type Parameters

T

The numeric type used for calculations (float or double).

Inheritance
OctonionLinearLayer<T>
Implements
Inherited Members

Remarks

An octonion linear layer performs matrix-vector multiplication in the 8-dimensional octonion algebra. Each weight and input is an octonion (8 real components), enabling the layer to capture more complex relationships than real-valued layers.

For Beginners: This layer is like a regular dense layer, but it uses 8-dimensional numbers (octonions) instead of regular numbers.

Benefits of octonion layers:

  • Can model more complex relationships with fewer parameters
  • Useful for certain types of image and signal processing
  • Better at capturing rotational relationships in data

The tradeoff is that computations are more expensive per parameter.

Constructors

OctonionLinearLayer(int, int, IActivationFunction<T>?)

Initializes a new instance of the OctonionLinearLayer.

public OctonionLinearLayer(int inputFeatures, int outputFeatures, IActivationFunction<T>? activationFunction = null)

Parameters

inputFeatures int

Number of input features (octonion-valued).

outputFeatures int

Number of output features (octonion-valued).

activationFunction IActivationFunction<T>

Optional activation function.

Properties

InputFeatures

Gets the number of input features (octonion-valued).

public int InputFeatures { get; }

Property Value

int

OutputFeatures

Gets the number of output features (octonion-valued).

public int OutputFeatures { get; }

Property Value

int

ParameterCount

Gets the total number of trainable parameters.

public override int ParameterCount { get; }

Property Value

int

Remarks

Each octonion has 8 real components, so the parameter count is: (InputFeatures * OutputFeatures + OutputFeatures) * 8

SupportsGpuExecution

Gets a value indicating whether this layer supports GPU execution.

protected override bool SupportsGpuExecution { get; }

Property Value

bool

SupportsJitCompilation

Gets whether this layer supports JIT compilation. Returns true only if the activation function also supports JIT compilation.

public override bool SupportsJitCompilation { get; }

Property Value

bool

SupportsTraining

Gets whether this layer supports training.

public override bool SupportsTraining { get; }

Property Value

bool

Methods

Backward(Tensor<T>) 

public override Tensor<T> Backward(Tensor<T> outputGradient)

Parameters

outputGradient Tensor<T>

Gradient from the next layer.

Returns

Tensor<T>

Gradient to pass to the previous layer.

BackwardGpu(IGpuTensor<T>)

Performs the backward pass on GPU for the octonion linear layer. Computes gradients using proper octonion algebra with Jacobian of octonion multiplication.

public override IGpuTensor<T> BackwardGpu(IGpuTensor<T> outputGradient)

Parameters

outputGradient IGpuTensor<T>

The GPU tensor containing the gradient of the loss with respect to the output.

Returns

IGpuTensor<T>

The GPU tensor containing the gradient of the loss with respect to the input.

ExportComputationGraph(List<ComputationNode<T>>)

Exports the layer's forward pass as a JIT-compilable computation graph.

public override ComputationNode<T> ExportComputationGraph(List<ComputationNode<T>> inputNodes)

Parameters

inputNodes List<ComputationNode<T>>

List to populate with input computation nodes.

Returns

ComputationNode<T>

The output computation node.

Remarks

This method builds a computation graph for the octonion linear layer using the TensorOperations.OctonionMatMul operation. The weights and biases are converted to tensor format for use in the computation graph.

Forward(Tensor<T>)

Performs the forward pass through the layer.

public override Tensor<T> Forward(Tensor<T> input)

Parameters

input Tensor<T>

Input tensor with shape [inputFeatures * 8] or [batch, inputFeatures * 8].

Returns

Tensor<T>

Output tensor with shape [outputFeatures * 8] or [batch, outputFeatures * 8].

ForwardGpu(params IGpuTensor<T>[])

Performs the GPU-accelerated forward pass for octonion linear transformation.

public override IGpuTensor<T> ForwardGpu(params IGpuTensor<T>[] inputs)

Parameters

inputs IGpuTensor<T>[]

The GPU tensor inputs. First element is the input activation.

Returns

IGpuTensor<T>

A GPU tensor containing the octonion transformation output.

Remarks

The octonion layer performs matrix-vector multiplication in 8-dimensional octonion algebra. This method uses a specialized CUDA kernel to perform octonion operations entirely on GPU.

GetParameters()

Gets all trainable parameters as a single vector.

public override Vector<T> GetParameters()

Returns

Vector<T>

A vector containing all weights and biases.

ResetState()

Resets the internal state of the layer.

public override void ResetState()

SetParameters(Vector<T>)

Sets all trainable parameters from a single vector.

public override void SetParameters(Vector<T> parameters)

Parameters

parameters Vector<T>

A vector containing all parameters to set.

UpdateParameters(T)

using the calculated gradients.

public override void UpdateParameters(T learningRate)

Parameters

learningRate T

The learning rate to use for the update.

UpdateParametersGpu(IGpuOptimizerConfig)

Updates parameters using GPU-based optimizer.

public override void UpdateParametersGpu(IGpuOptimizerConfig config)

Parameters

config IGpuOptimizerConfig

GPU optimizer configuration specifying the optimizer type and hyperparameters.

Edit this page