Table of Contents

Interface I3DDiffusionModel<T>

Namespace
AiDotNet.Interfaces
Assembly
AiDotNet.dll

Interface for 3D diffusion models that generate 3D content like point clouds, meshes, and scenes.

public interface I3DDiffusionModel<T> : IDiffusionModel<T>, IFullModel<T, Tensor<T>, Tensor<T>>, IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>, IModelSerializer, ICheckpointableModel, IParameterizable<T, Tensor<T>, Tensor<T>>, IFeatureAware, IFeatureImportance<T>, ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>, IGradientComputable<T, Tensor<T>, Tensor<T>>, IJitCompilable<T>

Type Parameters

T

The numeric type used for calculations.

Inherited Members
Extension Methods

Remarks

3D diffusion models extend diffusion to generate three-dimensional content, including point clouds, meshes, textured models, and full 3D scenes. They are used in computer graphics, game development, and virtual reality.

For Beginners: 3D diffusion models create 3D objects instead of flat images.

Types of 3D generation:

  • Point Clouds: Sets of 3D points that form a shape
  • Meshes: Surfaces made of triangles (like in games)
  • Textured Models: Meshes with colors and materials
  • Novel Views: New angles of an object from one image

How it works:

  1. Text-to-3D: Describe what you want → 3D model
  2. Image-to-3D: Single image → full 3D model
  3. Score Distillation: Use 2D diffusion to guide 3D optimization

Applications:

  • Game asset creation
  • Product design visualization
  • VR/AR content generation
  • Architectural modeling

This interface extends IDiffusionModel<T> with 3D-specific operations.

Properties

DefaultPointCount

Gets the default number of points in generated point clouds.

int DefaultPointCount { get; }

Property Value

int

SupportsMesh

Gets whether this model supports mesh generation.

bool SupportsMesh { get; }

Property Value

bool

SupportsNovelView

Gets whether this model supports novel view synthesis.

bool SupportsNovelView { get; }

Property Value

bool

SupportsPointCloud

Gets whether this model supports point cloud generation.

bool SupportsPointCloud { get; }

Property Value

bool

SupportsScoreDistillation

Gets whether this model supports score distillation sampling (SDS).

bool SupportsScoreDistillation { get; }

Property Value

bool

Remarks

SDS uses gradients from a 2D diffusion model to optimize a 3D representation. This is the technique behind DreamFusion and similar text-to-3D methods.

SupportsTexture

Gets whether this model supports texture generation.

bool SupportsTexture { get; }

Property Value

bool

Methods

ColorizePointCloud(Tensor<T>, string, int, int?)

Adds colors/normals to a point cloud.

Tensor<T> ColorizePointCloud(Tensor<T> pointCloud, string prompt, int numInferenceSteps = 50, int? seed = null)

Parameters

pointCloud Tensor<T>

Point cloud positions [batch, numPoints, 3].

prompt string

Text prompt for coloring.

numInferenceSteps int

Number of denoising steps.

seed int?

Optional random seed.

Returns

Tensor<T>

Point cloud with colors [batch, numPoints, 6] (XYZ + RGB).

ComputeScoreDistillationGradients(Tensor<T>, string, int, double)

Computes score distillation gradients for 3D optimization.

Tensor<T> ComputeScoreDistillationGradients(Tensor<T> renderedViews, string prompt, int timestep, double guidanceScale = 100)

Parameters

renderedViews Tensor<T>

2D renders from the current 3D representation.

prompt string

Text prompt guiding the optimization.

timestep int

Diffusion timestep for noise level.

guidanceScale double

Classifier-free guidance scale.

Returns

Tensor<T>

Gradients to apply to the 3D representation.

Remarks

For Beginners: This helps create 3D from text using 2D knowledge: 1. Render your 3D object from multiple angles 2. Ask a 2D diffusion model "does this look like [prompt]?" 3. Get gradients that tell you how to improve the 3D 4. Repeat until the 3D looks right from all angles

This is how DreamFusion works - it uses 2D diffusion to guide 3D creation.

GenerateMesh(string, string?, int, int, double, int?)

Generates a mesh from a text description.

Mesh3D<T> GenerateMesh(string prompt, string? negativePrompt = null, int resolution = 128, int numInferenceSteps = 64, double guidanceScale = 3, int? seed = null)

Parameters

prompt string

Text description of the desired 3D object.

negativePrompt string

What to avoid.

resolution int

Mesh resolution (higher = more detail).

numInferenceSteps int

Number of denoising steps.

guidanceScale double

Classifier-free guidance scale.

seed int?

Optional random seed.

Returns

Mesh3D<T>

Mesh data containing vertices and faces.

Remarks

For Beginners: This creates a 3D surface you can render: - Vertices: 3D points that define corners - Faces: Triangles connecting vertices to form surfaces - Can be exported to common 3D formats (OBJ, STL, etc.)

GeneratePointCloud(string, string?, int?, int, double, int?)

Generates a point cloud from a text description.

Tensor<T> GeneratePointCloud(string prompt, string? negativePrompt = null, int? numPoints = null, int numInferenceSteps = 64, double guidanceScale = 3, int? seed = null)

Parameters

prompt string

Text description of the desired 3D object.

negativePrompt string

What to avoid.

numPoints int?

Number of points in the cloud.

numInferenceSteps int

Number of denoising steps.

guidanceScale double

Classifier-free guidance scale.

seed int?

Optional random seed.

Returns

Tensor<T>

Point cloud tensor [batch, numPoints, 3] for XYZ coordinates.

Remarks

For Beginners: This creates a cloud of 3D points that form a shape: - prompt: "A chair" → 4096 points arranged in a chair shape - The points define the surface of the object - Can be converted to a mesh for rendering

ImageTo3D(Tensor<T>, int, int, double, int?)

Generates a 3D model from a single input image.

Mesh3D<T> ImageTo3D(Tensor<T> inputImage, int numViews = 4, int numInferenceSteps = 50, double guidanceScale = 3, int? seed = null)

Parameters

inputImage Tensor<T>

The input image showing the object.

numViews int

Number of views to generate for reconstruction.

numInferenceSteps int

Number of denoising steps.

guidanceScale double

Classifier-free guidance scale.

seed int?

Optional random seed.

Returns

Mesh3D<T>

Reconstructed mesh with optional texture.

Remarks

For Beginners: This turns a flat picture into a 3D model: - Input: Photo of a shoe from the front - Output: Full 3D model you can view from any angle - The model infers what the hidden parts look like

PointCloudToMesh(Tensor<T>, SurfaceReconstructionMethod)

Converts a point cloud to a mesh.

Mesh3D<T> PointCloudToMesh(Tensor<T> pointCloud, SurfaceReconstructionMethod method = SurfaceReconstructionMethod.Poisson)

Parameters

pointCloud Tensor<T>

Point cloud [batch, numPoints, 3].

method SurfaceReconstructionMethod

Surface reconstruction method.

Returns

Mesh3D<T>

Reconstructed mesh.

SynthesizeNovelViews(Tensor<T>, (double azimuth, double elevation)[], int, double, int?)

Synthesizes novel views of an object from a reference image.

Tensor<T>[] SynthesizeNovelViews(Tensor<T> inputImage, (double azimuth, double elevation)[] targetAngles, int numInferenceSteps = 50, double guidanceScale = 3, int? seed = null)

Parameters

inputImage Tensor<T>

The reference image.

targetAngles (double azimuth, double elevation)[]

Target viewing angles (azimuth, elevation) in radians.

numInferenceSteps int

Number of denoising steps.

guidanceScale double

Classifier-free guidance scale.

seed int?

Optional random seed.

Returns

Tensor<T>[]

Array of images from the requested viewpoints.

Remarks

For Beginners: This shows an object from different angles: - Input: Front view of a car - Target: 45°, 90°, 135° rotations - Output: Images showing the car from those angles - Useful for product visualization

Edit this page