Table of Contents

Class InvertedResidualBlock<T>

Namespace
AiDotNet.NeuralNetworks.Layers
Assembly
AiDotNet.dll

Implements an Inverted Residual Block (MBConv) used in MobileNetV2 and MobileNetV3.

public class InvertedResidualBlock<T> : LayerBase<T>, ILayer<T>, IJitCompilable<T>, IDiagnosticsProvider, IWeightLoadable<T>, IDisposable, IChainableComputationGraph<T>

Type Parameters

T

The numeric type used for calculations, typically float or double.

Inheritance
InvertedResidualBlock<T>
Implements
Inherited Members

Remarks

The Inverted Residual Block is the core building block for MobileNet architectures. Unlike traditional residual blocks that narrow then widen, inverted residual blocks expand then narrow, hence the name "inverted."

Architecture: 

Input ────────────────────────────────────────────────────────────────┐
  │                                                                   │
  └─► [Expand 1x1 ─► BN ─► Act] ─► DW 3x3 ─► BN ─► Act ─► [SE] ─►    │
                                                             │        │
                                          Project 1x1 ─► BN ─┘─► (+) ─► Output
                                                                   ↑
                                                            (if skip connection)

For Beginners: The Inverted Residual Block is designed for efficient mobile inference.

Key innovations:

  • Expansion: First EXPANDS the channels (opposite of traditional bottlenecks)
  • Depthwise separable convolution: Filters each channel independently, then mixes
  • Linear bottleneck: The final projection has NO activation (preserves information)
  • Skip connection: Only when input and output dimensions match

This design reduces computational cost while maintaining model accuracy.

Constructors

InvertedResidualBlock(int, int, int, int, int, int, bool, int, IActivationFunction<T>?)

Initializes a new instance of the InvertedResidualBlock<T> class.

public InvertedResidualBlock(int inChannels, int outChannels, int inputHeight, int inputWidth, int expansionRatio = 6, int stride = 1, bool useSE = false, int seRatio = 4, IActivationFunction<T>? activationFunction = null)

Parameters

inChannels int

The number of input channels.

outChannels int

The number of output channels.

inputHeight int

The height of the input feature map.

inputWidth int

The width of the input feature map.

expansionRatio int

The expansion ratio for the hidden layer (default: 6).

stride int

The stride for the depthwise convolution (default: 1).

useSE bool

Whether to use Squeeze-and-Excitation (for MobileNetV3).

seRatio int

The reduction ratio for SE block (default: 4).

activationFunction IActivationFunction<T>

Properties

ExpansionRatio

Gets the expansion ratio.

public int ExpansionRatio { get; }

Property Value

int

InChannels

Gets the number of input channels.

public int InChannels { get; }

Property Value

int

OutChannels

Gets the number of output channels.

public int OutChannels { get; }

Property Value

int

Stride

Gets the stride used in the depthwise convolution.

public int Stride { get; }

Property Value

int

SupportsGpuExecution

Gets a value indicating whether this layer has a GPU implementation.

protected override bool SupportsGpuExecution { get; }

Property Value

bool

SupportsJitCompilation

Gets whether this block supports JIT compilation.

public override bool SupportsJitCompilation { get; }

Property Value

bool

SupportsTraining

Gets a value indicating whether this layer supports training.

public override bool SupportsTraining { get; }

Property Value

bool

Methods

Backward(Tensor<T>)

Performs the backward pass of the Inverted Residual Block.

public override Tensor<T> Backward(Tensor<T> outputGradient)

Parameters

outputGradient Tensor<T>

The gradient of the loss with respect to the output.

Returns

Tensor<T>

The gradient of the loss with respect to the input.

BuildComputationGraph(ComputationNode<T>, string)

Builds the computation graph for this layer using the provided input node.

public ComputationNode<T> BuildComputationGraph(ComputationNode<T> inputNode, string namePrefix)

Parameters

inputNode ComputationNode<T>

The input computation node from the parent layer.

namePrefix string

Prefix for naming internal nodes (for debugging/visualization).

Returns

ComputationNode<T>

The output computation node representing this layer's computation.

Remarks

Unlike ILayer<T>.ExportComputationGraph, this method does NOT create a new input variable. Instead, it uses the provided inputNode as its input, allowing the parent layer to chain multiple sub-layers together in a single computation graph.

The namePrefix parameter should be used to prefix all internal node names to avoid naming conflicts when multiple instances of the same layer type are used.

ExportComputationGraph(List<ComputationNode<T>>)

Exports the computation graph for JIT compilation.

public override ComputationNode<T> ExportComputationGraph(List<ComputationNode<T>> inputNodes)

Parameters

inputNodes List<ComputationNode<T>>

List to populate with input computation nodes.

Returns

ComputationNode<T>

The output computation node.

Forward(Tensor<T>)

Performs the forward pass of the Inverted Residual Block.

public override Tensor<T> Forward(Tensor<T> input)

Parameters

input Tensor<T>

The input tensor [C, H, W] or [B, C, H, W].

Returns

Tensor<T>

The output tensor after the inverted residual computation.

ForwardGpu(params IGpuTensor<T>[])

Performs the forward pass on GPU, keeping data GPU-resident.

public override IGpuTensor<T> ForwardGpu(params IGpuTensor<T>[] inputs)

Parameters

inputs IGpuTensor<T>[]

The input tensors (expects single input).

Returns

IGpuTensor<T>

The output tensor on GPU.

GetParameters()

Gets all trainable parameters from the block.

public override Vector<T> GetParameters()

Returns

Vector<T>

A vector containing all trainable parameters.

ResetState()

Resets the internal state of the block.

public override void ResetState()

SetParameters(Vector<T>)

Sets all trainable parameters from the given parameter vector.

public override void SetParameters(Vector<T> parameters)

Parameters

parameters Vector<T>

The parameter vector containing all layer parameters.

UpdateParameters(T)

Updates the parameters of all sub-layers.

public override void UpdateParameters(T learningRate)

Parameters

learningRate T

The learning rate for parameter updates.

Edit this page