Table of Contents

Class DenseBlock<T>

Namespace
AiDotNet.NeuralNetworks.Layers
Assembly
AiDotNet.dll

Implements a Dense Block from the DenseNet architecture.

public class DenseBlock<T> : LayerBase<T>, ILayer<T>, IJitCompilable<T>, IDiagnosticsProvider, IWeightLoadable<T>, IDisposable

Type Parameters

T

The numeric type used for calculations.

Inheritance
DenseBlock<T>
Implements
Inherited Members

Remarks

A Dense Block is the core building block of DenseNet. It contains multiple layers where each layer receives feature maps from ALL preceding layers (dense connectivity). This creates strong gradient flow and feature reuse throughout the network.

Architecture of a Dense Block with n layers: 

Input (k0 channels)
  ↓
Layer 1: BN → ReLU → Conv1x1 → BN → ReLU → Conv3x3 → Output1 (k channels)
  ↓ concat
[Input, Output1] (k0 + k channels)
  ↓
Layer 2: BN → ReLU → Conv1x1 → BN → ReLU → Conv3x3 → Output2 (k channels)
  ↓ concat
[Input, Output1, Output2] (k0 + 2k channels)
  ↓
... (continues for n layers)
  ↓
Final: [Input, Output1, ..., OutputN] (k0 + n*k channels)
Where k is the growth rate (number of channels added per layer).

For Beginners: Dense connectivity means each layer can directly access features from all previous layers, promoting feature reuse and reducing the need for redundant feature learning.

Key benefits:

  • Strong gradient flow (helps with training very deep networks)
  • Feature reuse (each layer can use features from all previous layers)
  • Fewer parameters (layers can be narrow since they share features)

Constructors

DenseBlock(int, int, int, int, int, double)

Initializes a new instance of the DenseBlock<T> class.

public DenseBlock(int inputChannels, int numLayers, int growthRate, int inputHeight, int inputWidth, double bnMomentum = 0.1)

Parameters

inputChannels int

The number of input channels.

numLayers int

The number of layers in the dense block.

growthRate int

The number of channels each layer adds (k in the paper).

inputHeight int

The input feature map height.

inputWidth int

The input feature map width.

bnMomentum double

Batch normalization momentum (default: 0.1).

Properties

GrowthRate

Gets the growth rate (channels added per layer).

public int GrowthRate { get; }

Property Value

int

NumLayers

Gets the number of layers in this dense block.

public int NumLayers { get; }

Property Value

int

OutputChannels

Gets the number of output channels (inputChannels + numLayers * growthRate).

public int OutputChannels { get; }

Property Value

int

SupportsGpuExecution

Gets a value indicating whether this layer has a GPU implementation.

protected override bool SupportsGpuExecution { get; }

Property Value

bool

SupportsJitCompilation

Gets a value indicating whether this layer supports JIT compilation.

public override bool SupportsJitCompilation { get; }

Property Value

bool

SupportsTraining

Gets a value indicating whether this layer supports training.

public override bool SupportsTraining { get; }

Property Value

bool

Methods

Backward(Tensor<T>)

Performs the backward pass of the Dense Block.

public override Tensor<T> Backward(Tensor<T> outputGradient)

Parameters

outputGradient Tensor<T>

The gradient of the loss with respect to the output.

Returns

Tensor<T>

The gradient of the loss with respect to the input.

BackwardGpu(IGpuTensor<T>)

Performs GPU-accelerated backward pass for the Dense Block.

public override IGpuTensor<T> BackwardGpu(IGpuTensor<T> outputGradient)

Parameters

outputGradient IGpuTensor<T>

GPU tensor containing gradient of loss with respect to output.

Returns

IGpuTensor<T>

GPU tensor containing gradient with respect to input.

Remarks

Processes layers in reverse order, splitting gradients along channel dimension and accumulating gradients through dense connections.

ExportComputationGraph(List<ComputationNode<T>>)

Exports the computation graph for JIT compilation.

public override ComputationNode<T> ExportComputationGraph(List<ComputationNode<T>> inputNodes)

Parameters

inputNodes List<ComputationNode<T>>

List to populate with input computation nodes.

Returns

ComputationNode<T>

The output computation node representing the DenseBlock.

Remarks

This method builds a computation graph representing the DenseBlock with dense connectivity: Each layer's output is concatenated with all previous features along the channel dimension.

Forward(Tensor<T>)

Performs the forward pass of the Dense Block.

public override Tensor<T> Forward(Tensor<T> input)

Parameters

input Tensor<T>

The input tensor [B, C, H, W].

Returns

Tensor<T>

The output tensor with all layer outputs concatenated.

ForwardGpu(params IGpuTensor<T>[])

Performs the forward pass on GPU, keeping data GPU-resident.

public override IGpuTensor<T> ForwardGpu(params IGpuTensor<T>[] inputs)

Parameters

inputs IGpuTensor<T>[]

The input tensors (expects single input).

Returns

IGpuTensor<T>

The output tensor on GPU.

GetParameters()

Gets all trainable parameters from the block.

public override Vector<T> GetParameters()

Returns

Vector<T>

ResetState()

Resets the internal state of the block.

public override void ResetState()

SetParameters(Vector<T>)

Sets all trainable parameters from the given parameter vector.

public override void SetParameters(Vector<T> parameters)

Parameters

parameters Vector<T>

The parameter vector containing all layer parameters.

UpdateParameters(T)

Updates the parameters of all sub-layers.

public override void UpdateParameters(T learningRate)

Parameters

learningRate T

The learning rate for parameter updates.

Edit this page