Table of Contents

Class ConsistencyModel<T>

Namespace
AiDotNet.Diffusion.Models
Assembly
AiDotNet.dll

Consistency Model for single-step or few-step image generation.

public class ConsistencyModel<T> : LatentDiffusionModelBase<T>, ILatentDiffusionModel<T>, IDiffusionModel<T>, IFullModel<T, Tensor<T>, Tensor<T>>, IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>, IModelSerializer, ICheckpointableModel, IParameterizable<T, Tensor<T>, Tensor<T>>, IFeatureAware, IFeatureImportance<T>, ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>, IGradientComputable<T, Tensor<T>, Tensor<T>>, IJitCompilable<T>

Type Parameters

T

The numeric type used for calculations.

Inheritance
ConsistencyModel<T>
Implements
IFullModel<T, Tensor<T>, Tensor<T>>
IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>
IParameterizable<T, Tensor<T>, Tensor<T>>
ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>
IGradientComputable<T, Tensor<T>, Tensor<T>>
Inherited Members
Extension Methods

Examples

// Create a Consistency Model
var model = new ConsistencyModel<float>();

// Single-step generation (fastest)
var image1 = model.GenerateFromText(
    prompt: "A beautiful sunset over mountains",
    numInferenceSteps: 1);

// Two-step generation (better quality)
var image2 = model.GenerateFromText(
    prompt: "A beautiful sunset over mountains",
    numInferenceSteps: 2);

// Four-step generation (highest quality)
var image4 = model.GenerateFromText(
    prompt: "A beautiful sunset over mountains",
    numInferenceSteps: 4);

// Generate with progressive refinement
var refined = model.GenerateWithProgressiveRefinement(
    prompt: "Detailed landscape painting",
    maxSteps: 4,
    returnIntermediates: true);

Remarks

Consistency Models can generate high-quality images in a single step by learning to map any point on a probability flow ODE trajectory directly to the trajectory's origin (the clean data). This enables extremely fast generation compared to traditional diffusion models that require 20-50+ steps.

For Beginners: Traditional diffusion models are like walking down a path one step at a time - they need many steps to go from noise to a clear image. Consistency Models are like teleportation - they can jump directly from any point on the path to the destination (clean image).

Key advantages:

  • Single-step generation possible (fastest mode)
  • Progressive refinement: 1-step, 2-step, 4-step, etc.
  • Quality improves with more steps but plateaus quickly
  • Same or better quality as DDPM with 1000x fewer steps

How it works:

  1. The model learns that all points on a denoising path should map to the same clean image
  2. This "consistency" property allows direct prediction from any noise level
  3. The model can self-refine by treating its output as a new starting point

Use cases:

  • Real-time image generation
  • Interactive applications
  • Mobile/edge deployment
  • Batch processing at scale

Technical details: - Based on probability flow ODEs (deterministic diffusion) - Two training methods: distillation from pretrained diffusion, direct training - Uses boundary condition f(x, eps) = x (identity at minimal noise) - Supports multistep sampling for quality/speed tradeoff

Reference: Song et al., "Consistency Models", ICML 2023

Constructors

ConsistencyModel()

Initializes a new instance of ConsistencyModel with default parameters.

public ConsistencyModel()

Remarks

Creates a Consistency Model with standard parameters:

  • 18 training timesteps
  • Sigma range: 0.002 to 80
  • Rho: 7 (for sigma schedule)

ConsistencyModel(DiffusionModelOptions<T>?, INoiseScheduler<T>?, UNetNoisePredictor<T>?, StandardVAE<T>?, IConditioningModule<T>?, int, double, double, double, bool, int?)

Initializes a new instance of ConsistencyModel with custom parameters.

public ConsistencyModel(DiffusionModelOptions<T>? options = null, INoiseScheduler<T>? scheduler = null, UNetNoisePredictor<T>? noisePredictor = null, StandardVAE<T>? vae = null, IConditioningModule<T>? conditioner = null, int numTrainSteps = 18, double sigmaMin = 0.002, double sigmaMax = 80, double rho = 7, bool isDistilled = false, int? seed = null)

Parameters

options DiffusionModelOptions<T>

Configuration options for the diffusion model.

scheduler INoiseScheduler<T>

Optional custom scheduler.

noisePredictor UNetNoisePredictor<T>

Optional custom noise predictor.

vae StandardVAE<T>

Optional custom VAE.

conditioner IConditioningModule<T>

Optional conditioning module for text encoding.

numTrainSteps int

Number of training timesteps (default: 18).

sigmaMin double

Minimum sigma value (default: 0.002).

sigmaMax double

Maximum sigma value (default: 80.0).

rho double

Rho parameter for sigma schedule (default: 7.0).

isDistilled bool

Whether the model was trained via distillation.

seed int?

Optional random seed for reproducibility.

Properties

Conditioner

Gets the conditioning module (optional, for conditioned generation).

public override IConditioningModule<T>? Conditioner { get; }

Property Value

IConditioningModule<T>

IsDistilled

Gets whether this model was trained via distillation.

public bool IsDistilled { get; }

Property Value

bool

LatentChannels

Gets the number of latent channels.

public override int LatentChannels { get; }

Property Value

int

Remarks

Typically 4 for Stable Diffusion models.

NoisePredictor

Gets the noise predictor model (U-Net, DiT, etc.).

public override INoisePredictor<T> NoisePredictor { get; }

Property Value

INoisePredictor<T>

ParameterCount

Gets the number of parameters in the model.

public override int ParameterCount { get; }

Property Value

int

Remarks

This property returns the total count of trainable parameters in the model. It's useful for understanding model complexity and memory requirements.

SigmaMax

Gets the maximum sigma value used by this model.

public double SigmaMax { get; }

Property Value

double

SigmaMin

Gets the minimum sigma value used by this model.

public double SigmaMin { get; }

Property Value

double

VAE

Gets the VAE model used for encoding and decoding.

public override IVAEModel<T> VAE { get; }

Property Value

IVAEModel<T>

Methods

Clone()

Creates a deep copy of the model.

public override IDiffusionModel<T> Clone()

Returns

IDiffusionModel<T>

A new instance with the same parameters.

ConsistencyFunction(Tensor<T>, T, Tensor<T>?)

Applies the consistency function to map from any noise level to clean data.

public virtual Tensor<T> ConsistencyFunction(Tensor<T> x, T sigma, Tensor<T>? conditioning)

Parameters

x Tensor<T>

The noisy sample.

sigma T

The current noise level (sigma).

conditioning Tensor<T>

Optional conditioning tensor.

Returns

Tensor<T>

The predicted clean sample.

Remarks

The consistency function f(x, sigma) should satisfy:

  • f(x, sigma_min) = x (boundary condition at minimal noise)
  • f(x, sigma) = f(x', sigma') for all x, x' on same ODE trajectory

DeepCopy()

Creates a deep copy of this object.

public override IFullModel<T, Tensor<T>, Tensor<T>> DeepCopy()

Returns

IFullModel<T, Tensor<T>, Tensor<T>>

GenerateFromText(string, string?, int, int, int, double?, int?)

Generates an image using consistency sampling.

public override Tensor<T> GenerateFromText(string prompt, string? negativePrompt = null, int width = 512, int height = 512, int numInferenceSteps = 2, double? guidanceScale = null, int? seed = null)

Parameters

prompt string

The text prompt describing the desired image.

negativePrompt string

Optional negative prompt.

width int

Image width (should be divisible by 8).

height int

Image height (should be divisible by 8).

numInferenceSteps int

Number of sampling steps (1, 2, 4, etc.).

guidanceScale double?

Classifier-free guidance scale.

seed int?

Optional random seed.

Returns

Tensor<T>

The generated image tensor.

Remarks

For consistency models:

  • 1 step: Fastest, single forward pass
  • 2 steps: Good balance of speed and quality
  • 4 steps: Near-optimal quality
  • 8+ steps: Diminishing returns

GenerateWithProgressiveRefinement(string, string?, int, int, int, bool, double?, int?)

Generates images with progressive refinement, optionally returning intermediates.

public virtual List<Tensor<T>> GenerateWithProgressiveRefinement(string prompt, string? negativePrompt = null, int width = 512, int height = 512, int maxSteps = 4, bool returnIntermediates = false, double? guidanceScale = null, int? seed = null)

Parameters

prompt string

The text prompt describing the desired image.

negativePrompt string

Optional negative prompt.

width int

Image width.

height int

Image height.

maxSteps int

Maximum number of refinement steps.

returnIntermediates bool

Whether to return intermediate images.

guidanceScale double?

Classifier-free guidance scale.

seed int?

Optional random seed.

Returns

List<Tensor<T>>

List of generated images at each step (if returnIntermediates) or just the final image.

GetModelMetadata()

Retrieves metadata and performance metrics about the trained model.

public override ModelMetadata<T> GetModelMetadata()

Returns

ModelMetadata<T>

An object containing metadata and performance metrics about the trained model.

Remarks

This method provides information about the model's structure, parameters, and performance metrics.

For Beginners: Model metadata is like a report card for your machine learning model.

Just as a report card shows how well a student is performing in different subjects, model metadata shows how well your model is performing and provides details about its structure.

This information typically includes:

  • Accuracy measures: How well does the model's predictions match actual values?
  • Error metrics: How far off are the model's predictions on average?
  • Model parameters: What patterns did the model learn from the data?
  • Training information: How long did training take? How many iterations were needed?

For example, in a house price prediction model, metadata might include:

  • Average prediction error (e.g., off by $15,000 on average)
  • How strongly each feature (bedrooms, location) influences the prediction
  • How well the model fits the training data

This information helps you understand your model's strengths and weaknesses, and decide if it's ready to use or needs more training.

GetParameters()

Gets the parameters that can be optimized.

public override Vector<T> GetParameters()

Returns

Vector<T>

SetParameters(Vector<T>)

Sets the model parameters.

public override void SetParameters(Vector<T> parameters)

Parameters

parameters Vector<T>

The parameter vector to set.

Remarks

This method allows direct modification of the model's internal parameters. This is useful for optimization algorithms that need to update parameters iteratively. If the length of parameters does not match ParameterCount, an ArgumentException should be thrown.

Exceptions

ArgumentException

Thrown when the length of parameters does not match ParameterCount.

Edit this page