Table of Contents

Class GraphConvolutionalLoRAAdapter<T>

Namespace
AiDotNet.LoRA.Adapters
Assembly
AiDotNet.dll

LoRA adapter for Graph Convolutional layers, enabling parameter-efficient fine-tuning of GNN models.

public class GraphConvolutionalLoRAAdapter<T> : LoRAAdapterBase<T>, IDisposable, ILoRAAdapter<T>, IGraphConvolutionLayer<T>, ILayer<T>, IJitCompilable<T>, IDiagnosticsProvider, IWeightLoadable<T>

Type Parameters

T

The numeric type used for calculations, typically float or double.

Inheritance
GraphConvolutionalLoRAAdapter<T>
Implements
Inherited Members

Remarks

This adapter enables LoRA (Low-Rank Adaptation) for graph neural network layers. It wraps a graph convolutional layer (GCN, GAT, GraphSAGE, GIN) and adds a low-rank adaptation that can be efficiently trained while keeping the base layer frozen.

For Beginners: LoRA for GNNs allows you to fine-tune large pre-trained graph neural networks with a fraction of the trainable parameters.

Why LoRA for GNNs?

  • Pre-trained GNN models can be huge (millions of parameters)
  • Fine-tuning all parameters requires lots of memory
  • LoRA learns small "correction" matrices instead
  • Result: 10-100x fewer trainable parameters

How it works:

  • Original GNN layer stays frozen (no updates)
  • LoRA adds two small matrices (A and B) that learn adaptations
  • Output = original_output + LoRA_correction
  • Only A and B are trained, saving memory and time

Example - Fine-tuning a GNN for drug discovery:

// Wrap existing GAT layer with LoRA
var gatLayer = new GraphAttentionLayer<double>(128, 64, numHeads: 8);
var loraGat = new GraphConvolutionalLoRAAdapter<double>(
    gatLayer, rank: 8, alpha: 16);

// Now train only the LoRA parameters
loraGat.UpdateParameters(learningRate);

// After training, merge LoRA into original layer
var mergedLayer = loraGat.MergeToOriginalLayer();

Supported base layers:

  • GraphConvolutionalLayer (GCN)
  • GraphAttentionLayer (GAT)
  • GraphSAGELayer
  • GraphIsomorphismLayer (GIN)
  • Any layer implementing IGraphConvolutionLayer

Constructors

GraphConvolutionalLoRAAdapter(ILayer<T>, int, double, bool)

Initializes a new GraphConvolutionalLoRAAdapter.

public GraphConvolutionalLoRAAdapter(ILayer<T> baseLayer, int rank = 8, double alpha = -1, bool freezeBaseLayer = true)

Parameters

baseLayer ILayer<T>

The graph layer to adapt (must implement IGraphConvolutionLayer).

rank int

The rank of the LoRA decomposition (default: 8).

alpha double

The LoRA scaling factor (default: same as rank).

freezeBaseLayer bool

Whether to freeze the base layer during training (default: true).

Remarks

For Beginners: Creating a LoRA adapter for a graph layer: 

// Create base GAT layer
var gat = new GraphAttentionLayer<double>(
    inputFeatures: 128,
    outputFeatures: 64,
    numHeads: 8);

// Wrap with LoRA for efficient fine-tuning
var loraGat = new GraphConvolutionalLoRAAdapter<double>(
    gat,
    rank: 8,      // Low rank for efficiency
    alpha: 16,    // Scaling factor
    freezeBaseLayer: true);  // Freeze original weights

// Parameter count comparison:
// Original GAT: ~50,000 parameters
// LoRA adapter: ~2,000 parameters (only 4%!)

Exceptions

ArgumentException

Thrown when baseLayer doesn't implement IGraphConvolutionLayer.

Properties

InputFeatures

Gets the number of input features for this graph layer.

public int InputFeatures { get; }

Property Value

int

OutputFeatures

Gets the number of output features for this graph layer.

public int OutputFeatures { get; }

Property Value

int

Methods

Forward(Tensor<T>)

Performs forward pass through both base graph layer and LoRA layer.

public override Tensor<T> Forward(Tensor<T> input)

Parameters

input Tensor<T>

Input node features tensor.

Returns

Tensor<T>

Sum of base layer output and LoRA adaptation.

Remarks

The forward pass computes: output = graph_layer(input, adjacency) + lora_layer(input)

Note that the LoRA layer operates on raw features without the graph structure, providing a feature-space adaptation that complements the graph-aware base layer.

For Beginners: The graph layer aggregates neighbor information using the adjacency matrix. The LoRA layer learns to adjust the output features directly. Together, they provide adapted graph representations.

Exceptions

InvalidOperationException

Thrown when adjacency matrix is not set.

GetAdjacencyMatrix()

Gets the current adjacency matrix.

public Tensor<T>? GetAdjacencyMatrix()

Returns

Tensor<T>

The adjacency matrix, or null if not set.

MergeToOriginalLayer()

Merges the LoRA adaptation into the base graph layer.

public override ILayer<T> MergeToOriginalLayer()

Returns

ILayer<T>

A new graph layer with LoRA weights merged into its parameters.

Remarks

This creates a standalone graph layer that incorporates the LoRA adaptation. The merged layer behaves identically to the adapter but without the LoRA overhead.

For Beginners: After training with LoRA, you can "bake in" the adaptation to create a single layer for deployment. This is faster for inference since it doesn't need to compute the LoRA correction separately.

ResetState()

Resets the internal state of both graph layer and LoRA layer.

public override void ResetState()

SetAdjacencyMatrix(Tensor<T>)

Sets the adjacency matrix for graph convolution operations.

public void SetAdjacencyMatrix(Tensor<T> adjacencyMatrix)

Parameters

adjacencyMatrix Tensor<T>

The adjacency matrix defining graph structure.

Remarks

This must be called before Forward() to define the graph structure. The adjacency matrix is passed to the underlying graph layer.

For Beginners: The adjacency matrix tells the layer which nodes are connected. This is essential for graph convolution operations that aggregate neighbor information.

Edit this page