Class ClipModelLoader

Namespace

AiDotNet.NeuralNetworks

Assembly

AiDotNet.dll

Loads CLIP models from HuggingFace Hub or local directories.

public static class ClipModelLoader

Inheritance

object

ClipModelLoader

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

This loader handles downloading and caching CLIP model files from HuggingFace, including ONNX model files for image and text encoders, tokenizer files, and model configuration.

For Beginners: Instead of manually downloading files, this loader automatically fetches CLIP models from HuggingFace Hub:

// Load a pretrained CLIP model
var clip = await ClipModelLoader.FromPretrainedAsync<float>(
    "openai/clip-vit-base-patch32");

// Use the model
var textEmb = clip.EncodeText("a photo of a cat");
var imageEmb = clip.EncodeImage(imageTensor);
var similarity = clip.CalculateSimilarity(textEmb, imageEmb);

The model files are cached locally so subsequent loads are fast.

Fields

KnownModels

Known CLIP model configurations on HuggingFace Hub.

public static readonly IReadOnlyDictionary<string, ClipModelConfig> KnownModels

Field Value

IReadOnlyDictionary<string, ClipModelConfig>

Methods

ClearCache(string, string?)

Clears the cached model files.

public static void ClearCache(string modelId, string? cacheDir = null)

Parameters

modelId string

The HuggingFace model ID.

cacheDir string

Optional custom cache directory.

FromDirectory<T>(string, ClipModelConfig?)

Loads a CLIP model from a local directory.

public static ClipNeuralNetwork<T> FromDirectory<T>(string modelPath, ClipModelConfig? config = null)

Parameters

modelPath string

Path to the directory containing model files.

config ClipModelConfig

Optional model configuration. If null, attempts to auto-detect.

Returns

ClipNeuralNetwork<T>

A configured CLIP neural network.

Type Parameters

T

The numeric type for the model.

FromPretrainedAsync<T>(string, string?, IProgress<double>?, CancellationToken)

Loads a CLIP model from HuggingFace Hub asynchronously.

public static Task<ClipNeuralNetwork<T>> FromPretrainedAsync<T>(string modelId, string? cacheDir = null, IProgress<double>? progress = null, CancellationToken cancellationToken = default)

Parameters

modelId string

The HuggingFace model ID (e.g., "openai/clip-vit-base-patch32").

cacheDir string

Optional custom cache directory.

progress IProgress<double>

Optional progress reporter (0.0 to 1.0).

cancellationToken CancellationToken

Cancellation token.

Returns

Task<ClipNeuralNetwork<T>>

A configured CLIP neural network.

Type Parameters

T

The numeric type for the model.

Exceptions

ArgumentException

If model ID is invalid.

InvalidOperationException

If model files cannot be loaded.

FromPretrained<T>(string, string?)

Loads a CLIP model from HuggingFace Hub synchronously.

public static ClipNeuralNetwork<T> FromPretrained<T>(string modelId, string? cacheDir = null)

Parameters

modelId string

cacheDir string

Returns

ClipNeuralNetwork<T>

Type Parameters

T

Remarks

Warning: This method uses sync-over-async internally and may cause deadlocks in UI applications or ASP.NET contexts with synchronization contexts. Prefer using FromPretrainedAsync<T>(string, string?, IProgress<double>?, CancellationToken) when possible.

IsModelCached(string, string?)

Checks if a model is downloaded and cached locally.

public static bool IsModelCached(string modelId, string? cacheDir = null)

Parameters

modelId string

The HuggingFace model ID.

cacheDir string

Optional custom cache directory.

Returns

bool

True if all required files are cached.