Class OnnxModelOptions

Namespace

AiDotNet.Onnx

Assembly

AiDotNet.dll

Configuration options for loading and running ONNX models.

public class OnnxModelOptions

Inheritance

object

OnnxModelOptions

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Remarks

This class provides comprehensive configuration for ONNX model inference, including execution provider selection, memory optimization, and threading.

For Beginners: Use defaults for most cases:

var options = new OnnxModelOptions();
var model = new OnnxModel<float>("model.onnx", options);

For GPU acceleration:

var options = new OnnxModelOptions { ExecutionProvider = OnnxExecutionProvider.Cuda };

Properties

AutoWarmUp

Gets or sets whether to automatically warm up the model after loading. Warming up runs a single inference to initialize lazy resources.

public bool AutoWarmUp { get; set; }

Property Value

bool

CudaMemoryLimit

Gets or sets the CUDA memory limit in bytes (0 = no limit). Only applies to CUDA execution provider.

public long CudaMemoryLimit { get; set; }

Property Value

long

CudaUseArena

Gets or sets whether to use CUDA memory arena for better performance.

public bool CudaUseArena { get; set; }

Property Value

bool

CustomOptions

Gets or sets custom session options as key-value pairs.

public Dictionary<string, string> CustomOptions { get; set; }

Property Value

Dictionary<string, string>

EnableMemoryArena

Gets or sets whether to enable memory arena. This pre-allocates memory to reduce allocation overhead.

public bool EnableMemoryArena { get; set; }

Property Value

bool

EnableMemoryPattern

Gets or sets whether to enable memory pattern optimization. This can reduce memory allocations during inference.

public bool EnableMemoryPattern { get; set; }

Property Value

bool

EnableProfiling

Gets or sets whether to enable profiling for performance analysis.

public bool EnableProfiling { get; set; }

Property Value

bool

ExecutionProvider

Gets or sets the preferred execution provider. Default is Auto, which selects the best available provider.

public OnnxExecutionProvider ExecutionProvider { get; set; }

Property Value

OnnxExecutionProvider

FallbackProviders

Gets or sets fallback execution providers if the primary fails. Default fallback order: CUDA → DirectML → CPU

public List<OnnxExecutionProvider> FallbackProviders { get; set; }

Property Value

List<OnnxExecutionProvider>

GpuDeviceId

Gets or sets the GPU device ID for CUDA/TensorRT/DirectML providers. Default is 0 (first GPU).

public int GpuDeviceId { get; set; }

Property Value

int

InterOpNumThreads

Gets or sets the number of threads for parallel operations. 0 means use the default (typically number of CPU cores).

public int InterOpNumThreads { get; set; }

Property Value

int

IntraOpNumThreads

Gets or sets the number of threads for CPU execution. 0 means use the default (typically number of CPU cores).

public int IntraOpNumThreads { get; set; }

Property Value

int

LogLevel

Gets or sets the log severity level for ONNX Runtime.

public OnnxLogLevel LogLevel { get; set; }

Property Value

OnnxLogLevel

OptimizationLevel

Gets or sets the graph optimization level. Higher levels may increase load time but improve inference speed.

public GraphOptimizationLevel OptimizationLevel { get; set; }

Property Value

GraphOptimizationLevel

ProfileOutputPath

Gets or sets the path for saving profiling output. Only used if EnableProfiling is true.

public string? ProfileOutputPath { get; set; }

Property Value

string

Methods

ForCpu(int?)

Creates default options for CPU execution.

public static OnnxModelOptions ForCpu(int? threads = null)

Parameters

threads int?

Returns

OnnxModelOptions

ForCuda(int)

Creates default options for CUDA GPU execution.

public static OnnxModelOptions ForCuda(int deviceId = 0)

Parameters

deviceId int

Returns

OnnxModelOptions

ForDirectML(int)

Creates default options for DirectML GPU execution (Windows).

public static OnnxModelOptions ForDirectML(int deviceId = 0)

Parameters

deviceId int

Returns

OnnxModelOptions

ForTensorRT(int)

Creates default options for TensorRT execution (NVIDIA optimized).

public static OnnxModelOptions ForTensorRT(int deviceId = 0)

Parameters

deviceId int

Returns

OnnxModelOptions