Namespace AiDotNet.NeuralNetworks.Layers
Classes
- ActivationLayer<T>
A layer that applies an activation function to transform the input data.
Activation functions introduce non-linearity to neural networks. Non-linearity means the output isn't simply proportional to the input (like y = 2x). Instead, it can follow curves or more complex patterns. severely limiting what it can learn.
Common activation functions include: - ReLU: Returns 0 for negative inputs, or the input value for positive inputs - Sigmoid: Squashes values between 0 and 1, useful for probabilities - Tanh: Similar to sigmoid but outputs values between -1 and 1
- AdaptiveAveragePoolingLayer<T>
Implements adaptive average pooling that outputs a fixed spatial size regardless of input dimensions.
- AddLayer<T>
A layer that adds multiple input tensors element-wise and optionally applies an activation function.
- AnomalyDetectorLayer<T>
Represents a layer that detects anomalies by comparing predictions with actual inputs.
- AttentionLayer<T>
Represents an Attention Layer for focusing on relevant parts of input sequences.
- AveragePoolingLayer<T>
Implements an average pooling layer for neural networks, which reduces the spatial dimensions of the input by taking the average value in each pooling window.
- BasicBlock<T>
Implements the BasicBlock used in ResNet18 and ResNet34 architectures.
- BatchNormalizationLayer<T>
Implements batch normalization for neural networks, which normalizes the inputs across a mini-batch.
- BidirectionalLayer<T>
Represents a bidirectional layer that processes input sequences in both forward and backward directions.
- BottleneckBlock<T>
Implements the BottleneckBlock used in ResNet50, ResNet101, and ResNet152 architectures.
- CapsuleLayer<T>
Represents a capsule neural network layer that encapsulates groups of neurons to better preserve spatial information.
- ConcatenateLayer<T>
Represents a neural network layer that concatenates multiple inputs along a specified axis.
- ConditionalRandomFieldLayer<T>
Represents a Conditional Random Field (CRF) layer for sequence labeling tasks.
- ContinuumMemorySystemLayer<T>
Continuum Memory System (CMS) layer for neural networks. Implements a sequential chain of MLP blocks with different update frequencies. Based on Equations 30-31 from "Nested Learning" paper. yt = MLP^(fk)(MLP^(fk-1)(...MLP^(f1)(xt)))
- Conv3DLayer<T>
Represents a 3D convolutional layer for processing volumetric data like voxel grids.
- ConvLSTMLayer<T>
Implements a Convolutional Long Short-Term Memory (ConvLSTM) layer for processing sequential spatial data.
- ConvolutionalLayer<T>
Represents a convolutional layer in a neural network that applies filters to input data.
- CroppingLayer<T>
Represents a cropping layer that removes portions of input tensors from the edges.
- CrossAttentionLayer<T>
Implements cross-attention for conditioning diffusion models on text or other context.
- DecoderLayer<T>
Represents a Decoder Layer in a Transformer architecture.
- DeconvolutionalLayer<T>
Represents a deconvolutional layer (also known as transposed convolution) in a neural network.
- DeformableConvolutionalLayer<T>
Deformable Convolutional Layer that learns spatial sampling offsets.
- DenseBlock<T>
Implements a Dense Block from the DenseNet architecture.
- DenseLayer<T>
Represents a fully connected (dense) layer in a neural network.
- DepthwiseSeparableConvolutionalLayer<T>
Represents a depthwise separable convolutional layer that performs convolution as two separate operations.
- DiffusionConvLayer<T>
Implements diffusion convolution for mesh surface processing using the heat diffusion equation.
- DigitCapsuleLayer<T>
Represents a digit capsule layer that implements the dynamic routing algorithm between capsules.
- DilatedConvolutionalLayer<T>
Represents a dilated convolutional layer for neural networks that applies filters with gaps between filter elements.
- DirectionalGraphLayer<T>
Implements Directional Graph Networks for directed graph processing with separate in/out aggregations.
- DropoutLayer<T>
Implements a dropout layer for neural networks to prevent overfitting.
- EdgeConditionalConvolutionalLayer<T>
Implements Edge-Conditioned Convolution for incorporating edge features in graph convolutions.
- EmbeddingLayer<T>
Represents an embedding layer that converts discrete token indices into dense vector representations.
- ExpertLayer<T>
Represents an expert module in a Mixture-of-Experts architecture, containing a sequence of layers.
- FeedForwardLayer<T>
Represents a fully connected (dense) feed-forward layer in a neural network.
- FlattenLayer<T>
Represents a flatten layer that reshapes multi-dimensional input data into a 1D vector.
- FullyConnectedLayer<T>
Represents a fully connected layer in a neural network where every input neuron connects to every output neuron.
- GRULayer<T>
Represents a Gated Recurrent Unit (GRU) layer for processing sequential data.
- GatedLinearUnitLayer<T>
Represents a Gated Linear Unit (GLU) layer in a neural network that combines linear transformation with multiplicative gating.
- GaussianNoiseLayer<T>
A neural network layer that adds random Gaussian noise to inputs during training.
- GlobalPoolingLayer<T>
Represents a global pooling layer that reduces spatial dimensions to a single value per channel.
- GraphAttentionLayer<T>
Implements Graph Attention Network (GAT) layer for processing graph-structured data with attention mechanisms.
- GraphConvolutionalLayer<T>
Represents a Graph Convolutional Network (GCN) layer for processing graph-structured data.
- GraphIsomorphismLayer<T>
Implements Graph Isomorphism Network (GIN) layer for powerful graph representation learning.
- GraphSAGELayer<T>
Implements GraphSAGE (Graph Sample and Aggregate) layer for inductive learning on graphs.
- GraphTransformerLayer<T>
Implements Graph Transformer layer using self-attention mechanisms on graph-structured data.
- GroupNormalizationLayer<T>
Represents a Group Normalization layer that normalizes inputs across groups of channels.
- HeterogeneousGraphLayer<T>
Implements Heterogeneous Graph Neural Network layer for graphs with multiple node and edge types.
- HeterogeneousGraphMetadata
Represents metadata for heterogeneous graphs with multiple node and edge types.
- HighwayLayer<T>
Represents a Highway Neural Network layer that allows information to flow unchanged through the network.
- HyperbolicLinearLayer<T>
Represents a fully connected layer operating in hyperbolic (Poincare ball) space.
- InputLayer<T>
Represents an input layer that passes input data through unchanged to the next layer in the neural network.
- InstanceNormalizationLayer<T>
Represents an Instance Normalization layer that normalizes each channel independently across spatial dimensions.
- InvertedResidualBlock<T>
Implements an Inverted Residual Block (MBConv) used in MobileNetV2 and MobileNetV3.
- LSTMLayer<T>
Represents a Long Short-Term Memory (LSTM) layer for processing sequential data.
- LambdaLayer<T>
Represents a customizable layer that applies user-defined functions for both forward and backward passes.
- LayerBase<T>
Represents the base class for all neural network layers, providing common functionality and interfaces.
- LayerNormalizationLayer<T>
Represents a Layer Normalization layer that normalizes inputs across the feature dimension.
- LocallyConnectedLayer<T>
Represents a Locally Connected layer which applies different filters to different regions of the input, unlike a convolutional layer which shares filters.
- LogVarianceLayer<T>
Represents a layer that computes the logarithm of variance along a specified axis in the input tensor.
- MaskingLayer<T>
Represents a layer that masks specified values in the input tensor, typically used to ignore padding in sequential data.
- MaxPool3DLayer<T>
Represents a 3D max pooling layer for downsampling volumetric data.
- MaxPoolingLayer<T>
Implements a max pooling layer for neural networks, which reduces the spatial dimensions of the input by taking the maximum value in each pooling window.
- MeanLayer<T>
Represents a layer that computes the mean (average) of input values along a specified axis.
- MeasurementLayer<T>
Represents a layer that performs quantum measurement operations on complex-valued input tensors.
- MemoryReadLayer<T>
Represents a layer that reads from a memory tensor using an attention mechanism.
- MemoryWriteLayer<T>
Represents a layer that writes to a memory tensor using an attention mechanism.
- MeshEdgeConvLayer<T>
Implements edge convolution for mesh-based neural networks (MeshCNN style).
- MeshPoolLayer<T>
Implements mesh pooling via edge collapse for MeshCNN-style networks.
- MessagePassingLayer<T>
Implements a general Message Passing Neural Network (MPNN) layer.
- MixtureOfExpertsBuilder<T>
A builder class that helps create and configure Mixture-of-Experts layers with sensible defaults.
- MixtureOfExpertsLayer<T>
Implements a Mixture-of-Experts (MoE) layer that routes inputs through multiple expert networks.
- MultiHeadAttentionLayer<T>
Implements a multi-head attention layer for neural networks, a key component in transformer architectures.
- MultiplyLayer<T>
Represents a layer that performs element-wise multiplication of multiple input tensors.
- OctonionLinearLayer<T>
Represents a fully connected layer using octonion-valued weights and inputs.
- PaddingLayer<T>
Represents a layer that adds padding to the input tensor.
- PatchEmbeddingLayer<T>
Implements a patch embedding layer for Vision Transformer (ViT) architecture.
- PixelShuffleLayer<T>
Pixel shuffle (sub-pixel convolution) layer for efficient spatial upsampling.
- PoolingLayer<T>
Represents a layer that performs pooling operations on input tensors.
- PositionalEncodingLayer<T>
Represents a layer that adds positional encodings to input sequences.
- PrimaryCapsuleLayer<T>
Represents a primary capsule layer for capsule networks.
- PrincipalNeighbourhoodAggregationLayer<T>
Implements Principal Neighbourhood Aggregation (PNA) layer for powerful graph representation learning.
- QuantumLayer<T>
Represents a neural network layer that uses quantum computing principles for processing inputs.
- RBFLayer<T>
Represents a Radial Basis Function (RBF) layer for neural networks.
- RBMLayer<T>
Represents a Restricted Boltzmann Machine (RBM) layer for neural networks.
- RRDBLayer<T>
Residual in Residual Dense Block (RRDB) - the core building block of ESRGAN and Real-ESRGAN generators.
- RRDBNetGenerator<T>
RRDBNet Generator - the full generator architecture from ESRGAN and Real-ESRGAN.
- ReadoutLayer<T>
Represents a readout layer that performs the final mapping from features to output in a neural network.
- ReconstructionLayer<T>
Represents a reconstruction layer that uses multiple fully connected layers to transform inputs into outputs.
- RecurrentLayer<T>
Represents a recurrent neural network layer that processes sequential data by maintaining a hidden state.
- RepParameterizationLayer<T>
Represents a reparameterization layer used in variational autoencoders (VAEs) to enable backpropagation through random sampling.
- ReservoirLayer<T>
Represents a reservoir layer used in Echo State Networks (ESNs) for processing sequential data with fixed random weights.
- ReshapeLayer<T>
Represents a reshape layer that transforms the dimensions of input data without changing its content.
- ResidualDenseBlock<T>
Residual Dense Block (RDB) as used in ESRGAN and Real-ESRGAN generators.
- ResidualLayer<T>
Represents a residual layer that adds the identity mapping (input) to the output of an inner layer.
- SelfAttentionLayer<T>
Represents a self-attention layer that allows a sequence to attend to itself, capturing relationships between elements.
- SeparableConvolutionalLayer<T>
Represents a separable convolutional layer that decomposes standard convolution into depthwise and pointwise operations.
- SequenceLastLayer<T>
A layer that extracts the last timestep from a sequence.
- SparseLinearLayer<T>
Represents a fully connected layer with sparse weight matrix for efficient computation.
- SpatialPoolerLayer<T>
Represents a spatial pooler layer inspired by hierarchical temporal memory (HTM) principles.
- SpatialTransformerLayer<T>
Represents a spatial transformer layer that enables spatial manipulations of data via a learnable transformation.
- SpectralNormalizationLayer<T>
Represents a spectral normalization layer that normalizes the weights of a layer by their spectral norm.
- SpikingLayer<T>
Represents a layer of spiking neurons that model the biological dynamics of neural activity.
- SpiralConvLayer<T>
Implements spiral convolution for mesh vertex processing.
- SplitLayer<T>
Represents a layer that splits the input tensor along a specific dimension into multiple equal parts.
- SpyNetLayer<T>
SPyNet (Spatial Pyramid Network) layer for optical flow estimation.
- SqueezeAndExcitationLayer<T>
Represents a Squeeze-and-Excitation layer that recalibrates channel-wise feature responses adaptively.
- SubpixelConvolutionalLayer<T>
Represents a subpixel convolutional layer that performs convolution followed by pixel shuffling for upsampling.
- SwinPatchEmbeddingLayer<T>
Patch embedding layer for Swin Transformer that converts images to patch sequences.
- SwinPatchMergingLayer<T>
Patch merging layer for Swin Transformer that performs downsampling between stages.
- SwinTransformerBlockLayer<T>
Swin Transformer block layer with windowed multi-head self-attention.
- SynapticPlasticityLayer<T>
Represents a synaptic plasticity layer that models biological learning mechanisms through spike-timing-dependent plasticity.
- TemporalMemoryLayer<T>
Represents a temporal memory layer that models sequence learning through hierarchical temporal memory concepts.
- TimeDistributedLayer<T>
Represents a wrapper layer that applies an inner layer to each time step of a sequence independently.
- TimeEmbeddingLayer<T>
Represents a time embedding layer that encodes timesteps using sinusoidal embeddings for diffusion models.
- TransformerDecoderLayer<T>
Represents a transformer decoder layer that processes sequences using self-attention, cross-attention, and feed-forward networks.
- TransformerEncoderLayer<T>
Represents a transformer encoder layer that processes sequences using self-attention and feed-forward networks.
- TransitionLayer<T>
Implements a Transition Layer from the DenseNet architecture.
- UNetDiscriminator<T>
U-Net Discriminator as used in Real-ESRGAN for improved perceptual quality.
- Upsample3DLayer<T>
Represents a 3D upsampling layer that increases the spatial dimensions of volumetric data using nearest-neighbor interpolation.
- UpsamplingLayer<T>
Represents an upsampling layer that increases the spatial dimensions of input tensors using nearest-neighbor interpolation.
Enums
Delegates
- AggregationFunction<T>
Defines the aggregation function type for combining messages.
- MessageFunction<T>
Defines the message function type for message passing neural networks.
- UpdateFunction<T>
Defines the update function type for updating node features.