Table of Contents

Class DilatedConvolutionalLayer<T>

Namespace
AiDotNet.NeuralNetworks.Layers
Assembly
AiDotNet.dll

Represents a dilated convolutional layer for neural networks that applies filters with gaps between filter elements.

public class DilatedConvolutionalLayer<T> : LayerBase<T>, ILayer<T>, IJitCompilable<T>, IDiagnosticsProvider, IWeightLoadable<T>, IDisposable

Type Parameters

T

The numeric type used for calculations, typically float or double.

Inheritance
DilatedConvolutionalLayer<T>
Implements
Inherited Members

Remarks

A dilated convolutional layer extends traditional convolutional layers by introducing gaps (dilation) between the elements of the convolution kernel. This increases the receptive field without increasing the number of parameters or computational cost. Dilated convolutions are particularly useful in tasks requiring a wide context without sacrificing resolution, such as semantic segmentation or audio generation.

For Beginners: A dilated convolutional layer is like looking at an image with a special magnifying glass.

Regular convolutions look at pixels that are right next to each other, like this:

  • Looking at a 3×3 area of an image (9 adjacent pixels)

Dilated convolutions skip some pixels, creating gaps, like this:

  • With dilation=2, it looks at pixels with a gap of 1 pixel between them
  • The 3×3 filter now covers a 5×5 area (still using only 9 values)

Benefits:

  • Sees a larger area without needing more computing power
  • Captures wider patterns in the data
  • Helps detect features at different scales

For example, in image processing, dilated convolutions can help the network understand both fine details and broader context at the same time.

Constructors

DilatedConvolutionalLayer(int, int, int, int, int, int, int, int, IActivationFunction<T>?) 

public DilatedConvolutionalLayer(int inputDepth, int outputDepth, int kernelSize, int inputHeight, int inputWidth, int dilation, int stride = 1, int padding = 0, IActivationFunction<T>? activation = null)

Parameters

inputDepth int
outputDepth int
kernelSize int
inputHeight int
inputWidth int
dilation int
stride int
padding int
activation IActivationFunction<T>

DilatedConvolutionalLayer(int, int, int, int, int, int, int, int, IVectorActivationFunction<T>?)

Initializes a new instance of the DilatedConvolutionalLayer<T> class with vector activation.

public DilatedConvolutionalLayer(int inputDepth, int outputDepth, int kernelSize, int inputHeight, int inputWidth, int dilation, int stride = 1, int padding = 0, IVectorActivationFunction<T>? vectorActivation = null)

Parameters

inputDepth int

The number of channels in the input data.

outputDepth int

The number of filters (output channels).

kernelSize int

The size of the convolution kernel.

inputHeight int

The height of the input data.

inputWidth int

The width of the input data.

dilation int

The dilation factor that determines spacing between kernel elements.

stride int

The stride of the convolution operation. Defaults to 1.

padding int

The amount of zero-padding added to the input. Defaults to 0.

vectorActivation IVectorActivationFunction<T>

The vector activation function to apply. Defaults to ReLU.

Remarks

This constructor creates a new dilated convolutional layer with the specified parameters. Unlike the other constructor, this one accepts a vector activation function that operates on entire vectors rather than individual scalar values.

For Beginners: This is an alternative setup that uses a different kind of activation function.

This constructor is almost identical to the first one, but with one key difference:

  • Regular activation: processes each output value separately
  • Vector activation: processes groups of values together

Vector activation functions can capture relationships between different output values, which might be useful for some advanced applications.

For most basic uses, the regular constructor is sufficient.

Properties

SupportsGpuExecution

Gets a value indicating whether this layer supports GPU execution.

protected override bool SupportsGpuExecution { get; }

Property Value

bool

SupportsJitCompilation

Gets whether this layer supports JIT compilation.

public override bool SupportsJitCompilation { get; }

Property Value

bool

True if the layer can be JIT compiled, false otherwise.

Remarks

This property indicates whether the layer has implemented ExportComputationGraph() and can benefit from JIT compilation. All layers MUST implement this property.

For Beginners: JIT compilation can make inference 5-10x faster by converting the layer's operations into optimized native code.

Layers should return false if they:

  • Have not yet implemented a working ExportComputationGraph()
  • Use dynamic operations that change based on input data
  • Are too simple to benefit from JIT compilation

When false, the layer will use the standard Forward() method instead.

SupportsTraining

Gets a value indicating whether this layer supports training.

public override bool SupportsTraining { get; }

Property Value

bool

Always true because this layer has trainable parameters (kernels and biases).

Remarks

This property indicates that the dilated convolutional layer supports training through backpropagation. The layer has trainable parameters (kernels and biases) that are updated during the training process.

For Beginners: This property tells you that this layer can learn from data.

A value of true means:

  • The layer adjusts its filters and biases during training
  • It improves over time as it sees more examples
  • It participates in the learning process of the neural network

This is different from some layers (like pooling layers) that don't have trainable parameters and therefore don't "learn" in the same way.

Methods

Backward(Tensor<T>)

Performs the backward pass of the convolutional layer to compute gradients.

public override Tensor<T> Backward(Tensor<T> outputGradient)

Parameters

outputGradient Tensor<T>

The gradient of the loss with respect to the layer's output.

Returns

Tensor<T>

The gradient of the loss with respect to the layer's input.

Remarks

This method implements the backward pass (backpropagation) of the dilated convolutional layer. It computes the gradients of the loss with respect to the layer's weights, biases, and inputs. These gradients are used to update the parameters during training.

For Beginners: This is where the layer learns from its mistakes during training.

During the backward pass:

  1. The layer receives information about how its output contributed to errors
  2. It calculates three things:
    • How to adjust each filter value (kernel gradients)
    • How to adjust each bias value (bias gradients)
    • How the error flows back to the previous layer (input gradients)
  3. These gradients are used to update the filters and biases

The dilation is also taken into account when calculating these gradients, ensuring that the learning process understands the dilated nature of the convolution.

Exceptions

InvalidOperationException

Thrown when backward is called before forward.

BackwardGpu(IGpuTensor<T>)

Performs a GPU-resident backward pass for dilated convolution.

public override IGpuTensor<T> BackwardGpu(IGpuTensor<T> outputGradient)

Parameters

outputGradient IGpuTensor<T>

GPU-resident gradient from subsequent layer.

Returns

IGpuTensor<T>

GPU-resident gradient with respect to input.

ExportComputationGraph(List<ComputationNode<T>>)

Exports the layer's computation graph for JIT compilation.

public override ComputationNode<T> ExportComputationGraph(List<ComputationNode<T>> inputNodes)

Parameters

inputNodes List<ComputationNode<T>>

List to populate with input computation nodes.

Returns

ComputationNode<T>

The output computation node representing the layer's operation.

Remarks

This method constructs a computation graph representation of the layer's forward pass that can be JIT compiled for faster inference. All layers MUST implement this method to support JIT compilation.

For Beginners: JIT (Just-In-Time) compilation converts the layer's operations into optimized native code for 5-10x faster inference.

To support JIT compilation, a layer must:

  1. Implement this method to export its computation graph
  2. Set SupportsJitCompilation to true
  3. Use ComputationNode and TensorOperations to build the graph

All layers are required to implement this method, even if they set SupportsJitCompilation = false.

Forward(Tensor<T>)

Performs the forward pass of the convolutional layer.

public override Tensor<T> Forward(Tensor<T> input)

Parameters

input Tensor<T>

The input tensor to process.

Returns

Tensor<T>

The output tensor after convolution and activation.

Remarks

This method implements the forward pass of the dilated convolutional layer. It applies the convolution operation with the specified dilation, stride, and padding to the input tensor. The result is passed through the activation function to produce the final output.

For Beginners: This is where the layer processes input data to detect patterns.

During the forward pass:

  1. For each position in the output:
    • Look at the corresponding area in the input (accounting for dilation)
    • Multiply each input value by the corresponding filter value
    • Sum up all these multiplications
    • Add the bias value
  2. Apply the activation function to the result
  3. Save the input and output for later use in training

The dilation controls how spread out the filter points are when looking at the input, allowing the network to capture wider patterns.

ForwardGpu(params IGpuTensor<T>[])

Performs a GPU-resident forward pass using fused DilatedConv2D + Bias + Activation.

public override IGpuTensor<T> ForwardGpu(params IGpuTensor<T>[] inputs)

Parameters

inputs IGpuTensor<T>[]

GPU-resident input tensor.

Returns

IGpuTensor<T>

GPU-resident output tensor.

Remarks

For Beginners: This is the GPU-optimized version of the Forward method. All data stays on the GPU throughout the computation, avoiding expensive CPU-GPU transfers.

GetParameters()

Gets all trainable parameters of the layer as a single vector.

public override Vector<T> GetParameters()

Returns

Vector<T>

A vector containing all trainable parameters.

Remarks

This method retrieves all trainable parameters (weights and biases) of the layer as a single vector. This is useful for optimization algorithms that operate on all parameters at once, or for saving and loading model weights.

For Beginners: This method collects all the layer's learnable values into a single list.

The parameters include:

  • All the filter weights (the majority of the parameters)
  • All the bias values (one per output channel)

This combined list is useful for:

  • Saving the model to disk
  • Loading parameters from a previously trained model
  • Advanced optimization techniques that need all parameters together

Think of it like packing all the knowledge the layer has learned into a single container.

ResetState()

Resets the internal state of the layer.

public override void ResetState()

Remarks

This method resets the internal state of the layer by clearing all cached values from forward and backward passes. This is useful when starting to process a new batch of data.

For Beginners: This method clears the layer's memory to start fresh.

When resetting the state:

  • The saved input and output are cleared
  • The calculated gradients are cleared
  • The layer forgets previous calculations it performed

This is useful:

  • Between training batches to free up memory
  • When switching from training to evaluation mode
  • When starting to process completely new data

It's like wiping a whiteboard clean before starting a new calculation.

SetParameters(Vector<T>)

Sets the trainable parameters of the layer from a single vector.

public override void SetParameters(Vector<T> parameters)

Parameters

parameters Vector<T>

A vector containing all parameters to set.

Remarks

This method sets all trainable parameters (weights and biases) of the layer from a single vector. This is useful for loading saved model weights or for implementing optimization algorithms that operate on all parameters at once.

For Beginners: This method updates all the layer's learnable values from a provided list.

When setting parameters:

  • The input must be a vector with the exact right length
  • The values are distributed back to the filter weights and biases
  • This allows loading previously trained weights

Use cases include:

  • Restoring a saved model
  • Using pre-trained weights
  • Testing specific weight configurations

The method throws an error if the provided vector doesn't contain exactly the right number of values.

Exceptions

ArgumentException

Thrown when the parameters vector has incorrect length.

UpdateParameters(T)

Updates the layer's weights and biases using the calculated gradients and the specified learning rate.

public override void UpdateParameters(T learningRate)

Parameters

learningRate T

The learning rate to use for the parameter updates.

Remarks

This method updates the layer's weights and biases based on the gradients calculated during the backward pass. The learning rate determines the size of the parameter updates. Smaller learning rates lead to more stable but slower training, while larger learning rates can lead to faster but potentially unstable training.

For Beginners: This method actually changes the filters and biases to improve future predictions.

After figuring out how each value should change (in the backward pass):

  • Each filter value is adjusted in the direction that reduces errors
  • Each bias value is also adjusted to optimize performance
  • The learning rate controls how big these adjustments are

Think of it like adjusting a recipe after tasting:

  • Too salty? Reduce salt next time
  • Too bland? Add more seasoning
  • But make small adjustments, not drastic ones

After updating, the gradients are cleared to prepare for the next training batch.

Exceptions

InvalidOperationException

Thrown when update is called before backward.

Edit this page