Table of Contents

Class FourierNeuralOperator<T>

Namespace
AiDotNet.PhysicsInformed.NeuralOperators
Assembly
AiDotNet.dll

Implements the Fourier Neural Operator (FNO) for learning operators between function spaces.

public class FourierNeuralOperator<T> : NeuralNetworkBase<T>, INeuralNetworkModel<T>, INeuralNetwork<T>, IFullModel<T, Tensor<T>, Tensor<T>>, IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>, IModelSerializer, ICheckpointableModel, IParameterizable<T, Tensor<T>, Tensor<T>>, IFeatureAware, IFeatureImportance<T>, ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>, IGradientComputable<T, Tensor<T>, Tensor<T>>, IJitCompilable<T>, IInterpretableModel<T>, IInputGradientComputable<T>, IDisposable

Type Parameters

T

The numeric type used for calculations.

Inheritance
FourierNeuralOperator<T>
Implements
IFullModel<T, Tensor<T>, Tensor<T>>
IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>
IParameterizable<T, Tensor<T>, Tensor<T>>
ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>
IGradientComputable<T, Tensor<T>, Tensor<T>>
Inherited Members
Extension Methods

Remarks

For Beginners: A Neural Operator learns mappings between entire functions, not just inputs to outputs.

Traditional Neural Networks:

  • Learn: point → point mappings
  • Input: a vector (x, y, z)
  • Output: a vector (u, v, w)
  • Example: (temperature, pressure) → (velocity)

Neural Operators:

  • Learn: function → function mappings
  • Input: an entire function a(x)
  • Output: an entire function u(x)
  • Example: initial condition → solution after time T

Why This Matters: Many problems in physics involve operators:

  • PDE solution operator: (initial/boundary conditions) → (solution)
  • Green's function: (source) → (response)
  • Transfer function: (input signal) → (output signal)

Traditionally, you'd need to solve the PDE from scratch for each new set of conditions. With neural operators, you train once, then can instantly evaluate for new conditions!

Fourier Neural Operator (FNO): The key innovation is doing computations in Fourier space.

How FNO Works:

  1. Lift: Embed input function into higher-dimensional space
  2. Fourier Layers (repeated): a) Apply FFT to transform to frequency domain b) Linear transformation in frequency space (learn weights) c) Apply inverse FFT to return to physical space d) Add skip connection and activation
  3. Project: Map back to output function

Why Fourier Space?

  • Many PDEs have simple form in frequency domain
  • Derivatives → multiplication (∂/∂x in physical space = ik in Fourier space)
  • Captures global information efficiently
  • Natural for periodic problems
  • Computational efficiency (FFT is O(n log n))

Key Advantages:

  1. Resolution-invariant: Train at one resolution, evaluate at another
  2. Fast: Instant evaluation after training (vs. solving PDE each time)
  3. Mesh-free: No discretization needed
  4. Generalizes well: Works for different parameter values
  5. Captures long-range dependencies naturally

Applications:

  • Fluid dynamics (Navier-Stokes)
  • Climate modeling (weather prediction)
  • Material science (stress-strain)
  • Seismic imaging
  • Quantum chemistry (electron density)

Example Use Case: Problem: Solve 2D Navier-Stokes for different initial vorticity fields Traditional: Solve PDE numerically for each initial condition (slow) FNO: Train once on many examples, then instantly predict solution for new initial conditions

Historical Context: FNO was introduced by Li et al. in 2021 and has achieved remarkable success in learning solution operators for PDEs, often matching or exceeding traditional numerical methods in accuracy while being orders of magnitude faster.

Constructors

FourierNeuralOperator(NeuralNetworkArchitecture<T>, int, int, int[]?, int, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>>?)

Initializes a new instance of the Fourier Neural Operator.

public FourierNeuralOperator(NeuralNetworkArchitecture<T> architecture, int modes = 16, int width = 64, int[]? spatialDimensions = null, int numLayers = 4, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>>? optimizer = null)

Parameters

architecture NeuralNetworkArchitecture<T>

The network architecture.

modes int

Number of Fourier modes to retain (higher = more detail, but more computation).

width int

Channel width of the network (similar to hidden layer size).

spatialDimensions int[]

Dimensions of the input function domain (e.g., [64, 64] for 64x64 grid).

numLayers int

Number of Fourier layers.

optimizer IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>>

Remarks

For Beginners:

Parameters Explained:

modes: How many Fourier modes to keep

  • Low modes = low frequency information (smooth, large-scale features)
  • High modes = high frequency information (fine details, sharp features)
  • Typical: 12-32 modes
  • Trade-off: accuracy vs. computational cost

width: Number of channels in the network

  • Like hidden layer size in regular networks
  • More width = more capacity, but slower
  • Typical: 32-128

spatialDimensions: Size of the discretized function

  • For 1D: [N] (function sampled at N points)
  • For 2D: [Nx, Ny] (function on Nx × Ny grid)
  • For 3D: [Nx, Ny, Nz]
  • FNO can handle different resolutions at test time!

numLayers: Depth of the network

  • More layers = more expressive, but diminishing returns
  • Typical: 4-8 layers

Properties

ParameterCount

Gets the total parameter count for lift, Fourier, and projection layers.

public override int ParameterCount { get; }

Property Value

int

SupportsJitCompilation

Gets whether this model currently supports JIT compilation.

public override bool SupportsJitCompilation { get; }

Property Value

bool

True if the model can be JIT compiled, false otherwise.

Remarks

Some models may not support JIT compilation due to: - Dynamic graph structure (changes based on input) - Lack of computation graph representation - Use of operations not yet supported by the JIT compiler

For Beginners: This tells you whether this specific model can benefit from JIT compilation.

Models return false if they:

  • Use layer-based architecture without graph export (e.g., current neural networks)
  • Have control flow that changes based on input data
  • Use operations the JIT compiler doesn't understand yet

In these cases, the model will still work normally, just without JIT acceleration.

SupportsTraining

Indicates whether this network supports training (learning from data).

public override bool SupportsTraining { get; }

Property Value

bool

Remarks

For Beginners: Not all neural networks can learn. Some are designed only for making predictions with pre-set parameters. This property tells you if the network can learn from data.

Methods

Backpropagate(Tensor<T>)

Performs backpropagation to compute gradients for network parameters.

public override Tensor<T> Backpropagate(Tensor<T> outputGradients)

Parameters

outputGradients Tensor<T>

The gradients of the loss with respect to the network outputs.

Returns

Tensor<T>

The gradients of the loss with respect to the network inputs.

Remarks

For Beginners: Backpropagation is how neural networks learn. After making a prediction, the network calculates how wrong it was (the error). Then it works backward through the layers to figure out how each parameter contributed to that error. This method handles that backward flow of information.

The "gradients" are numbers that tell us how to adjust each parameter to reduce the error.

API Change Note: The signature changed from Vector<T> to Tensor<T> to support multi-dimensional gradients. This is a breaking change. If you need backward compatibility, consider adding an overload that accepts Vector<T> and converts it internally to Tensor<T>.

Exceptions

InvalidOperationException

Thrown when the network is not in training mode or doesn't support training.

CreateNewInstance()

Creates a new instance with the same configuration.

protected override IFullModel<T, Tensor<T>, Tensor<T>> CreateNewInstance()

Returns

IFullModel<T, Tensor<T>, Tensor<T>>

New FNO instance.

DeserializeNetworkSpecificData(BinaryReader)

Deserializes FNO-specific data.

protected override void DeserializeNetworkSpecificData(BinaryReader reader)

Parameters

reader BinaryReader

Binary reader.

Forward(Tensor<T>)

Forward pass through the FNO.

public Tensor<T> Forward(Tensor<T> input)

Parameters

input Tensor<T>

Input function (discretized on a grid).

Returns

Tensor<T>

Output function (solution).

Remarks

For Beginners: The forward pass consists of:

  1. Lift: input channels → width channels
  2. Apply Fourier layers (multiple times)
  3. Project: width channels → output channels

Each Fourier layer does:

  • FFT to frequency domain
  • Learned linear transformation
  • Inverse FFT back to physical space
  • Add skip connection
  • Apply activation

ForwardWithMemory(Tensor<T>)

Performs a forward pass through the network while storing intermediate values for backpropagation.

public override Tensor<T> ForwardWithMemory(Tensor<T> input)

Parameters

input Tensor<T>

The input data to the network.

Returns

Tensor<T>

The output of the network.

Remarks

For Beginners: This method passes data through the network from input to output, but also remembers all the intermediate values. This is necessary for the learning process, as the network needs to know these values when figuring out how to improve.

API Change Note: The signature changed from Vector<T> to Tensor<T> to support multi-dimensional inputs. This is a breaking change. For backward compatibility, consider adding an overload that accepts Vector<T> and converts it internally to Tensor<T>.

Exceptions

InvalidOperationException

Thrown when the network doesn't support training.

GetGradients()

Gets the gradients from all layers in the neural network.

public override Vector<T> GetGradients()

Returns

Vector<T>

A vector containing all gradients from all layers concatenated together.

Remarks

This method collects the gradients from every layer in the network and combines them into a single vector. This is useful for optimization algorithms that need access to all gradients at once.

For Beginners: During training, each layer calculates how its parameters should change (the gradients). This method gathers all those gradients from every layer and puts them into one long list.

Think of it like:

  • Each layer has notes about how to improve (gradients)
  • This method collects all those notes into one document
  • The optimizer can then use this document to update the entire network

This is essential for the learning process, as it tells the optimizer how to adjust all the network's parameters to improve performance.

GetModelMetadata()

Gets metadata about the FNO model.

public override ModelMetadata<T> GetModelMetadata()

Returns

ModelMetadata<T>

Model metadata.

GetParameters()

Gets the trainable parameters as a flattened vector.

public override Vector<T> GetParameters()

Returns

Vector<T>

InitializeLayers()

Initializes the layers of the neural network based on the architecture.

protected override void InitializeLayers()

Remarks

For Beginners: This method sets up all the layers in your neural network according to the architecture you've defined. It's like assembling the parts of your network before you can use it.

Predict(Tensor<T>)

Makes a prediction using the FNO.

public override Tensor<T> Predict(Tensor<T> input)

Parameters

input Tensor<T>

Input tensor.

Returns

Tensor<T>

Predicted output tensor.

SerializeNetworkSpecificData(BinaryWriter)

Serializes FNO-specific data.

protected override void SerializeNetworkSpecificData(BinaryWriter writer)

Parameters

writer BinaryWriter

Binary writer.

Train(Tensor<T>, Tensor<T>)

Performs a basic supervised training step using MSE loss.

public override void Train(Tensor<T> input, Tensor<T> expectedOutput)

Parameters

input Tensor<T>

Training input tensor.

expectedOutput Tensor<T>

Expected output tensor.

Train(Tensor<T>[], Tensor<T>[], int, double)

Trains the FNO on input-output function pairs.

public TrainingHistory<T> Train(Tensor<T>[] inputFunctions, Tensor<T>[] outputFunctions, int epochs = 100, double learningRate = 0.001)

Parameters

inputFunctions Tensor<T>[]

Training input functions.

outputFunctions Tensor<T>[]

Training output functions (solutions).

epochs int

Number of training epochs.

learningRate double

Learning rate.

Returns

TrainingHistory<T>

Training history.

Remarks

For Beginners: Training an FNO is like training a regular network, but:

  • Inputs are functions (represented as discretized grids)
  • Outputs are functions
  • Loss measures difference between predicted and true output functions

Example:

  • Input: Initial temperature distribution T(x, y, t=0)
  • Output: Temperature distribution at later time T(x, y, t=1)
  • Loss: ||FNO(T_initial) - T_final||²

After training, you can:

  • Give it a new initial condition
  • Instantly get the solution (no PDE solving!)
  • Even evaluate at different resolutions

UpdateParameters(Vector<T>)

Updates the trainable parameters from a flattened vector.

public override void UpdateParameters(Vector<T> parameters)

Parameters

parameters Vector<T>

Parameter vector.

Edit this page