Table of Contents

Class VAEResBlock<T>

Namespace
AiDotNet.Diffusion.VAE
Assembly
AiDotNet.dll

Residual block for VAE encoder/decoder with GroupNorm and skip connections.

public class VAEResBlock<T> : LayerBase<T>, ILayer<T>, IJitCompilable<T>, IDiagnosticsProvider, IWeightLoadable<T>, IDisposable

Type Parameters

T

The numeric type used for calculations.

Inheritance
VAEResBlock<T>
Implements
Inherited Members

Remarks

This implements a proper VAE residual block following the Stable Diffusion VAE architecture: - GroupNorm -> SiLU -> Conv -> GroupNorm -> SiLU -> Conv - Skip connection with optional 1x1 convolution when input/output channels differ

For Beginners: A residual block helps the network learn more effectively.

Think of it like taking notes during a lecture:

  • The main path (two convolutions) learns new features
  • The skip connection preserves the original information
  • Adding them together means you learn the "difference" or "improvement"

The GroupNorm helps stabilize training by normalizing activations within groups of channels, which works well even with small batch sizes commonly used in image generation tasks.

Structure:

    input ─────────────────────────────────┐
      │                                    │
      ├─→ GroupNorm → SiLU → Conv3x3 ─→ h  │ (skip connection)
      │                                    │
      │        ↓                           │
      │                                    │
      │   GroupNorm → SiLU → Conv3x3 ─→ h  │
      │                                    │
      │        ↓                           ↓
      │                                 [1x1 Conv if channels differ]
      │        ↓                           ↓
      └────────────────→ (+) ←─────────────┘
                         │
                      output

Constructors

VAEResBlock(int, int, int, int)

Initializes a new instance of the VAEResBlock class.

public VAEResBlock(int inChannels, int outChannels, int numGroups = 32, int spatialSize = 32)

Parameters

inChannels int

Number of input channels.

outChannels int

Number of output channels.

numGroups int

Number of groups for GroupNorm (default: 32).

spatialSize int

Spatial dimensions (height/width) for conv layer setup.

Remarks

For Beginners: Create a VAE residual block with the specified channel configuration.

Typical configurations:

  • numGroups=32 for 256+ channels
  • numGroups=16 for 128 channels
  • numGroups=8 for 64 channels

The numGroups should evenly divide the channel count for proper normalization.

Properties

InputChannels

Gets the number of input channels.

public int InputChannels { get; }

Property Value

int

NumGroups

Gets the number of groups for GroupNorm.

public int NumGroups { get; }

Property Value

int

OutputChannels

Gets the number of output channels.

public int OutputChannels { get; }

Property Value

int

SupportsJitCompilation

Gets whether this layer supports JIT compilation.

public override bool SupportsJitCompilation { get; }

Property Value

bool

True if the layer can be JIT compiled, false otherwise.

Remarks

This property indicates whether the layer has implemented ExportComputationGraph() and can benefit from JIT compilation. All layers MUST implement this property.

For Beginners: JIT compilation can make inference 5-10x faster by converting the layer's operations into optimized native code.

Layers should return false if they:

  • Have not yet implemented a working ExportComputationGraph()
  • Use dynamic operations that change based on input data
  • Are too simple to benefit from JIT compilation

When false, the layer will use the standard Forward() method instead.

SupportsTraining

Gets a value indicating whether this layer supports training.

public override bool SupportsTraining { get; }

Property Value

bool

true if the layer has trainable parameters and supports backpropagation; otherwise, false.

Remarks

This property indicates whether the layer can be trained through backpropagation. Layers with trainable parameters such as weights and biases typically return true, while layers that only perform fixed transformations (like pooling or activation layers) typically return false.

For Beginners: This property tells you if the layer can learn from data.

A value of true means:

  • The layer has parameters that can be adjusted during training
  • It will improve its performance as it sees more data
  • It participates in the learning process

A value of false means:

  • The layer doesn't have any adjustable parameters
  • It performs the same operation regardless of training
  • It doesn't need to learn (but may still be useful)

Methods

Backward(Tensor<T>)

Performs the backward pass through the residual block.

public override Tensor<T> Backward(Tensor<T> outputGradient)

Parameters

outputGradient Tensor<T>

Gradient of loss with respect to output.

Returns

Tensor<T>

Gradient of loss with respect to input.

Deserialize(BinaryReader)

Loads the block's state from a binary reader.

public override void Deserialize(BinaryReader reader)

Parameters

reader BinaryReader

ExportComputationGraph(List<ComputationNode<T>>)

Exports the layer's computation graph for JIT compilation.

public override ComputationNode<T> ExportComputationGraph(List<ComputationNode<T>> inputNodes)

Parameters

inputNodes List<ComputationNode<T>>

List to populate with input computation nodes.

Returns

ComputationNode<T>

The output computation node representing the layer's operation.

Remarks

This method constructs a computation graph representation of the layer's forward pass that can be JIT compiled for faster inference. All layers MUST implement this method to support JIT compilation.

For Beginners: JIT (Just-In-Time) compilation converts the layer's operations into optimized native code for 5-10x faster inference.

To support JIT compilation, a layer must:

  1. Implement this method to export its computation graph
  2. Set SupportsJitCompilation to true
  3. Use ComputationNode and TensorOperations to build the graph

All layers are required to implement this method, even if they set SupportsJitCompilation = false.

Forward(Tensor<T>)

Performs the forward pass through the residual block.

public override Tensor<T> Forward(Tensor<T> input)

Parameters

input Tensor<T>

Input tensor with shape [batch, channels, height, width].

Returns

Tensor<T>

Output tensor with shape [batch, outChannels, height, width].

GetParameters()

Gets all trainable parameters as a single vector.

public override Vector<T> GetParameters()

Returns

Vector<T>

ResetState()

Resets the internal state of the block.

public override void ResetState()

Serialize(BinaryWriter)

Saves the block's state to a binary writer.

public override void Serialize(BinaryWriter writer)

Parameters

writer BinaryWriter

SetParameters(Vector<T>)

Sets all trainable parameters from a single vector.

public override void SetParameters(Vector<T> parameters)

Parameters

parameters Vector<T>

UpdateParameters(T)

Updates all learnable parameters using gradient descent.

public override void UpdateParameters(T learningRate)

Parameters

learningRate T

The learning rate for the update.

Edit this page