Class GDAS<T>
Gradient-based Differentiable Architecture Search with Gumbel-Softmax sampling. GDAS uses Gumbel-Softmax to make the architecture search fully differentiable while maintaining discrete selection during forward pass.
Reference: "Searching for A Robust Neural Architecture in Four GPU Hours" (CVPR 2019)
public class GDAS<T> : NasAutoMLModelBase<T>, IAutoMLModel<T, Tensor<T>, Tensor<T>>, IFullModel<T, Tensor<T>, Tensor<T>>, IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>, IModelSerializer, ICheckpointableModel, IParameterizable<T, Tensor<T>, Tensor<T>>, IFeatureAware, IFeatureImportance<T>, ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>, IGradientComputable<T, Tensor<T>, Tensor<T>>, IJitCompilable<T>Type Parameters
TThe numeric type for calculations
- Inheritance
-
GDAS<T>
- Implements
- Inherited Members
-
AutoMLModelBase<T, Tensor<T>, Tensor<T>>.SetModelEvaluator(IModelEvaluator<T, Tensor<T>, Tensor<T>>)
- Extension Methods
Constructors
GDAS(SearchSpaceBase<T>, int, double, double)
public GDAS(SearchSpaceBase<T> searchSpace, int numNodes = 4, double initialTemperature = 5, double finalTemperature = 0.1)Parameters
searchSpaceSearchSpaceBase<T>numNodesintinitialTemperaturedoublefinalTemperaturedouble
Properties
NasNumNodes
Gets the number of nodes to search over.
protected override int NasNumNodes { get; }Property Value
NasSearchSpace
Gets the NAS search space.
protected override SearchSpaceBase<T> NasSearchSpace { get; }Property Value
NumOps
Gets the numeric operations provider for T.
protected override INumericOperations<T> NumOps { get; }Property Value
- INumericOperations<T>
Methods
AnnealTemperature(int, int)
Anneals the Gumbel-Softmax temperature during training
public void AnnealTemperature(int currentEpoch, int maxEpochs)Parameters
CreateInstanceForCopy()
Factory method for creating a new instance for deep copy. Derived classes must implement this to return a new instance of themselves. This ensures each copy has its own collections and lock object.
protected override AutoMLModelBase<T, Tensor<T>, Tensor<T>> CreateInstanceForCopy()Returns
- AutoMLModelBase<T, Tensor<T>, Tensor<T>>
A fresh instance of the derived class with default parameters
Remarks
When implementing this method, derived classes should create a fresh instance with default parameters, and should not attempt to preserve runtime or initialization state from the original instance. The deep copy logic will transfer relevant state (trial history, search space, etc.) after construction.
DeriveArchitecture()
Derives the discrete architecture by selecting the operation with highest weight
public Architecture<T> DeriveArchitecture()Returns
- Architecture<T>
GetArchitectureGradients()
Gets architecture gradients
public List<Matrix<T>> GetArchitectureGradients()Returns
- List<Matrix<T>>
GetArchitectureParameters()
Gets architecture parameters for optimization
public List<Matrix<T>> GetArchitectureParameters()Returns
- List<Matrix<T>>
GetTemperature()
Gets current temperature
public T GetTemperature()Returns
- T
GumbelSoftmax(Matrix<T>, bool)
Applies Gumbel-Softmax sampling to architecture parameters. This makes the discrete sampling operation differentiable.
public Matrix<T> GumbelSoftmax(Matrix<T> alpha, bool hard = false)Parameters
alphaMatrix<T>hardbool
Returns
- Matrix<T>
SearchArchitecture(Tensor<T>, Tensor<T>, Tensor<T>, Tensor<T>, TimeSpan, CancellationToken)
Performs algorithm-specific architecture search.
protected override Architecture<T> SearchArchitecture(Tensor<T> inputs, Tensor<T> targets, Tensor<T> validationInputs, Tensor<T> validationTargets, TimeSpan timeLimit, CancellationToken cancellationToken)Parameters
inputsTensor<T>targetsTensor<T>validationInputsTensor<T>validationTargetsTensor<T>timeLimitTimeSpancancellationTokenCancellationToken
Returns
- Architecture<T>