Class ProxylessNAS<T>

Namespace

AiDotNet.AutoML.NAS

Assembly

AiDotNet.dll

ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware. Uses path binarization and latency-aware loss to search directly on the target device without requiring a proxy task or separate hardware lookup tables.

Reference: "ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware" (ICLR 2019)

public class ProxylessNAS<T> : NasAutoMLModelBase<T>, IAutoMLModel<T, Tensor<T>, Tensor<T>>, IFullModel<T, Tensor<T>, Tensor<T>>, IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>, IModelSerializer, ICheckpointableModel, IParameterizable<T, Tensor<T>, Tensor<T>>, IFeatureAware, IFeatureImportance<T>, ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>, IGradientComputable<T, Tensor<T>, Tensor<T>>, IJitCompilable<T>

Type Parameters

T

The numeric type for calculations

Inheritance

object

AutoMLModelBase<T, Tensor<T>, Tensor<T>>

NasAutoMLModelBase<T>

ProxylessNAS<T>

Implements

IAutoMLModel<T, Tensor<T>, Tensor<T>>

IFullModel<T, Tensor<T>, Tensor<T>>

IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>

IModelSerializer

ICheckpointableModel

IParameterizable<T, Tensor<T>, Tensor<T>>

IFeatureAware

IFeatureImportance<T>

ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>

IGradientComputable<T, Tensor<T>, Tensor<T>>

IJitCompilable<T>

Inherited Members

NasAutoMLModelBase<T>.BestArchitecture

NasAutoMLModelBase<T>.SearchAsync(Tensor<T>, Tensor<T>, Tensor<T>, Tensor<T>, TimeSpan, CancellationToken)

NasAutoMLModelBase<T>.SuggestNextTrialAsync()

NasAutoMLModelBase<T>.CreateModelAsync(ModelType, Dictionary<string, object>)

NasAutoMLModelBase<T>.GetDefaultSearchSpace(ModelType)

NasAutoMLModelBase<T>.ApplyArchitectureToModel(SuperNet<T>, Architecture<T>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>._trialHistory

AutoMLModelBase<T, Tensor<T>, Tensor<T>>._searchSpace

AutoMLModelBase<T, Tensor<T>, Tensor<T>>._candidateModels

AutoMLModelBase<T, Tensor<T>, Tensor<T>>._constraints

AutoMLModelBase<T, Tensor<T>, Tensor<T>>._lock

AutoMLModelBase<T, Tensor<T>, Tensor<T>>._optimizationMetric

AutoMLModelBase<T, Tensor<T>, Tensor<T>>._maximize

AutoMLModelBase<T, Tensor<T>, Tensor<T>>._optimizationMetricExplicitlySet

AutoMLModelBase<T, Tensor<T>, Tensor<T>>._earlyStoppingPatience

AutoMLModelBase<T, Tensor<T>, Tensor<T>>._earlyStoppingMinDelta

AutoMLModelBase<T, Tensor<T>, Tensor<T>>._trialsSinceImprovement

AutoMLModelBase<T, Tensor<T>, Tensor<T>>._modelEvaluator

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.OptimizationMetric

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.MaximizeOptimizationMetric

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.Type

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.Status

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.BestModel

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.BestScore

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.TimeLimit

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.TrialLimit

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.SearchAsync(Tensor<T>, Tensor<T>, Tensor<T>, Tensor<T>, TimeSpan, CancellationToken)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.SetSearchSpace(Dictionary<string, ParameterRange>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.SetCandidateModels(List<ModelType>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.SetOptimizationMetric(MetricType, bool)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.GetTrialHistory()

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.GetFeatureImportanceAsync()

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.SuggestNextTrialAsync()

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.ReportTrialResultAsync(Dictionary<string, object>, double, TimeSpan)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.EnableEarlyStopping(int, double)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.SetConstraints(List<SearchConstraint>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.Train(double[][], double[])

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.Predict(double[][])

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.GetModelMetadata()

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.ShouldStop()

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.ValidateConstraints(Dictionary<string, object>, IFullModel<T, Tensor<T>, Tensor<T>>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.CreateModelAsync(ModelType, Dictionary<string, object>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.EvaluateModelAsync(IFullModel<T, Tensor<T>, Tensor<T>>, Tensor<T>, Tensor<T>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.GetDefaultSearchSpace(ModelType)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.Train(Tensor<T>, Tensor<T>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.Predict(Tensor<T>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.SaveModel(string)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.LoadModel(string)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.Serialize()

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.Deserialize(byte[])

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.GetParameters()

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.SetParameters(Vector<T>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.ParameterCount

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.WithParameters(Vector<T>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.FeatureNames

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.GetFeatureImportance()

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.GetActiveFeatureIndices()

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.IsFeatureUsed(int)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.SetActiveFeatureIndices(IEnumerable<int>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.Clone()

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.DeepCopy()

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.CreateInstanceForCopy()

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.SetModelEvaluator(IModelEvaluator<T, Tensor<T>, Tensor<T>>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.ExtractMetricFromEvaluation(ModelEvaluationData<T, Tensor<T>, Tensor<T>>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.ReportTrialFailureAsync(Dictionary<string, object>, Exception, TimeSpan)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.ConfigureSearchSpace(Dictionary<string, ParameterRange>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.SetTimeLimit(TimeSpan)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.SetTrialLimit(int)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.EnableNAS(bool)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.SearchBestModel(Tensor<T>, Tensor<T>, Tensor<T>, Tensor<T>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.Search(Tensor<T>, Tensor<T>, Tensor<T>, Tensor<T>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.GetResults()

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.Run(Tensor<T>, Tensor<T>, Tensor<T>, Tensor<T>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.SetModelsToTry(List<ModelType>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.DefaultLossFunction

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.ComputeGradients(Tensor<T>, Tensor<T>, ILossFunction<T>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.ApplyGradients(Vector<T>, T)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.SupportsJitCompilation

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.ExportComputationGraph(List<ComputationNode<T>>)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.SaveState(Stream)

AutoMLModelBase<T, Tensor<T>, Tensor<T>>.LoadState(Stream)

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Extension Methods

DistributedExtensions.AsDistributedForHighBandwidth<T, TInput, TOutput>(IFullModel<T, TInput, TOutput>, ICommunicationBackend<T>)

DistributedExtensions.AsDistributedForLowBandwidth<T, TInput, TOutput>(IFullModel<T, TInput, TOutput>, ICommunicationBackend<T>)

DistributedExtensions.AsDistributed<T, TInput, TOutput>(IFullModel<T, TInput, TOutput>, ICommunicationBackend<T>)

DistributedExtensions.AsDistributed<T, TInput, TOutput>(IFullModel<T, TInput, TOutput>, IShardingConfiguration<T>)

Constructors

ProxylessNAS(SearchSpaceBase<T>, int, HardwarePlatform, double, bool)

public ProxylessNAS(SearchSpaceBase<T> searchSpace, int numNodes = 4, HardwarePlatform targetPlatform = HardwarePlatform.Mobile, double latencyWeight = 0.1, bool useBinarization = true)

Parameters

searchSpace SearchSpaceBase<T>

numNodes int

targetPlatform HardwarePlatform

latencyWeight double

useBinarization bool

Properties

NasNumNodes

Gets the number of nodes to search over.

protected override int NasNumNodes { get; }

Property Value

int

NasSearchSpace

Gets the NAS search space.

protected override SearchSpaceBase<T> NasSearchSpace { get; }

Property Value

SearchSpaceBase<T>

NumOps

Gets the numeric operations provider for T.

protected override INumericOperations<T> NumOps { get; }

Property Value

INumericOperations<T>

Methods

BinarizePaths(Matrix<T>)

Applies path binarization for memory-efficient single-path sampling. Only one operation is active at a time during forward pass.

public Matrix<T> BinarizePaths(Matrix<T> alpha)

Parameters

alpha Matrix<T>

Returns

Matrix<T>

ComputeExpectedLatency(int, int)

Computes the expected latency cost of the architecture

public T ComputeExpectedLatency(int inputChannels, int spatialSize)

Parameters

inputChannels int

spatialSize int

Returns

T

ComputeTotalLoss(T, int, int)

Computes the total loss including task loss and latency regularization

public T ComputeTotalLoss(T taskLoss, int inputChannels, int spatialSize)

Parameters

taskLoss T

inputChannels int

spatialSize int

Returns

T

CreateInstanceForCopy()

Factory method for creating a new instance for deep copy. Derived classes must implement this to return a new instance of themselves. This ensures each copy has its own collections and lock object.

protected override AutoMLModelBase<T, Tensor<T>, Tensor<T>> CreateInstanceForCopy()

Returns

AutoMLModelBase<T, Tensor<T>, Tensor<T>>

A fresh instance of the derived class with default parameters

Remarks

When implementing this method, derived classes should create a fresh instance with default parameters, and should not attempt to preserve runtime or initialization state from the original instance. The deep copy logic will transfer relevant state (trial history, search space, etc.) after construction.

DeriveArchitecture()

Derives the final discrete architecture by selecting operations with highest weights

public Architecture<T> DeriveArchitecture()

Returns

Architecture<T>

EstimateArchitectureCost(int, int)

Estimates the final architecture's hardware cost

public HardwareCost<T> EstimateArchitectureCost(int inputChannels, int spatialSize)

Parameters

inputChannels int

spatialSize int

Returns

HardwareCost<T>

GetArchitectureGradients()

Gets architecture gradients

public List<Matrix<T>> GetArchitectureGradients()

Returns

List<Matrix<T>>

GetArchitectureParameters()

Gets architecture parameters for optimization

public List<Matrix<T>> GetArchitectureParameters()

Returns

List<Matrix<T>>

SearchArchitecture(Tensor<T>, Tensor<T>, Tensor<T>, Tensor<T>, TimeSpan, CancellationToken)

Performs algorithm-specific architecture search.

protected override Architecture<T> SearchArchitecture(Tensor<T> inputs, Tensor<T> targets, Tensor<T> validationInputs, Tensor<T> validationTargets, TimeSpan timeLimit, CancellationToken cancellationToken)

Parameters

inputs Tensor<T>

targets Tensor<T>

validationInputs Tensor<T>

validationTargets Tensor<T>

timeLimit TimeSpan

cancellationToken CancellationToken

Returns

Architecture<T>

SetBinarizationTemperature(double)

Sets the binarization temperature

public void SetBinarizationTemperature(double temperature)

Parameters

temperature double