Table of Contents

Interface IRadianceField<T>

Namespace
AiDotNet.NeuralRadianceFields.Interfaces
Assembly
AiDotNet.dll

Defines the core functionality for neural radiance field models.

public interface IRadianceField<T> : INeuralNetwork<T>, IFullModel<T, Tensor<T>, Tensor<T>>, IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>, IModelSerializer, ICheckpointableModel, IParameterizable<T, Tensor<T>, Tensor<T>>, IFeatureAware, IFeatureImportance<T>, ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>, IGradientComputable<T, Tensor<T>, Tensor<T>>, IJitCompilable<T>

Type Parameters

T

The numeric type used for calculations (e.g., float, double).

Inherited Members
Extension Methods

Remarks

For Beginners: A neural radiance field represents a 3D scene as a continuous function.

Traditional 3D representations:

  • Meshes: Surfaces made of triangles
  • Voxels: 3D grid of cubes (like 3D pixels)
  • Point clouds: Collection of 3D points

Neural Radiance Fields (NeRF):

  • Represents scene as a neural network
  • Input: 3D position (X, Y, Z) and viewing direction
  • Output: Color (RGB) and density (opacity/volume density)

Think of it like this:

  • The neural network "knows" what the scene looks like from any position
  • Ask "What's at position (x, y, z) when viewed from direction (θ, φ)?"
  • Network responds "Color is (r, g, b) and density is σ"

Why this is powerful:

  • Continuous representation (query any position, not limited to discrete grid)
  • View-dependent effects (reflections, specularities)
  • Compact storage (just network weights, not millions of voxels)
  • Novel view synthesis (render from any camera angle)

Applications:

  • Virtual reality and AR: Create photorealistic 3D scenes
  • Film and gaming: Capture real locations and render from any angle
  • Robotics: Build 3D maps of environments
  • Cultural heritage: Digitally preserve historical sites

Methods

QueryField(Tensor<T>, Tensor<T>)

Queries the radiance field at specific 3D positions and viewing directions.

(Tensor<T> rgb, Tensor<T> density) QueryField(Tensor<T> positions, Tensor<T> viewingDirections)

Parameters

positions Tensor<T>

Tensor of 3D positions [N, 3] where N is number of query points.

viewingDirections Tensor<T>

Tensor of viewing directions [N, 3] (unit vectors).

Returns

(Tensor<T> grad1, Tensor<T> grad2)

A tuple containing RGB colors [N, 3] and density values [N, 1].

Remarks

For Beginners: This is the core operation of a radiance field.

For each query point:

  • Position (x, y, z): Where in 3D space are we looking?
  • Direction (dx, dy, dz): Which direction are we looking from?

The network returns:

  • RGB (r, g, b): The color at that point from that direction
  • Density σ: How "solid" or "opaque" that point is

Example query:

  • Position: (2.5, 1.0, -3.0) - a point in space
  • Direction: (0.0, 0.0, -1.0) - looking straight down negative Z axis
  • Result: (Red: 0.8, Green: 0.3, Blue: 0.1, Density: 5.2) This means the point appears orange when viewed from that direction, and it's fairly opaque (high density)

Density interpretation:

  • Density = 0: Completely transparent (empty space)
  • Density > 0: Increasingly opaque (solid material)
  • Higher density = light is more likely to stop at this point

RenderImage(Vector<T>, Matrix<T>, int, int, T)

Renders an image from a specific camera position and orientation.

Tensor<T> RenderImage(Vector<T> cameraPosition, Matrix<T> cameraRotation, int imageWidth, int imageHeight, T focalLength)

Parameters

cameraPosition Vector<T>

3D position of the camera [3].

cameraRotation Matrix<T>

Rotation matrix of the camera [3, 3].

imageWidth int

Width of the output image in pixels.

imageHeight int

Height of the output image in pixels.

focalLength T

Camera focal length.

Returns

Tensor<T>

Rendered RGB image tensor [height, width, 3].

Remarks

For Beginners: This renders a 2D image from the 3D scene representation.

The rendering process:

  1. For each pixel in the output image:
    • Cast a ray from camera through that pixel
    • Sample points along the ray
    • Query radiance field at each sample point
    • Combine colors using volume rendering (accumulate with alpha blending)

Volume rendering equation:

  • For each ray, accumulate: Color = Σ(transmittance × color × alpha)
  • Transmittance: How much light passes through previous points
  • Alpha: How much light is absorbed at this point (based on density)

Example:

  • Camera at (0, 0, 5) looking at origin
  • Image 512×512 pixels
  • Cast 512×512 = 262,144 rays
  • Sample 64 points per ray = 16.8 million queries
  • Blend results to get final image

This is why NeRF rendering can be slow - many network queries! Optimizations like Instant-NGP speed this up significantly.

RenderRays(Tensor<T>, Tensor<T>, int, T, T)

Renders rays through the radiance field using volume rendering.

Tensor<T> RenderRays(Tensor<T> rayOrigins, Tensor<T> rayDirections, int numSamples, T nearBound, T farBound)

Parameters

rayOrigins Tensor<T>

Origins of rays to render [N, 3].

rayDirections Tensor<T>

Directions of rays (unit vectors) [N, 3].

numSamples int

Number of samples per ray.

nearBound T

Near clipping distance.

farBound T

Far clipping distance.

Returns

Tensor<T>

Rendered RGB colors for each ray [N, 3].

Remarks

For Beginners: Volume rendering is how we convert the radiance field into images.

For each ray:

  1. Sample points: Generate sample positions between near and far bounds
  2. Query field: Get RGB and density at each sample
  3. Compute alpha: Convert density to opacity for each segment
  4. Accumulate color: Blend colors front-to-back

The algorithm:

For each sample i along ray:
  alpha_i = 1 - exp(-density_i * distance_i)
  transmittance_i = exp(-sum of all previous densities)
  color_contribution_i = transmittance_i * alpha_i * color_i
  total_color += color_contribution_i

Example with 4 samples:

  • Sample 0: Empty space (density ≈ 0) → contributes little
  • Sample 1: Empty space (density ≈ 0) → contributes little
  • Sample 2: Surface (density high) → contributes most of the color
  • Sample 3: Behind surface → mostly blocked by sample 2

Parameters:

  • numSamples: More samples = better quality but slower (typical: 64-192)
  • nearBound/farBound: Define region to sample (e.g., 0.1 to 10.0 meters)
Edit this page