Class TransformerSearchSpace<T>

Namespace

AiDotNet.AutoML.SearchSpace

Assembly

AiDotNet.dll

Defines the Transformer-based search space for neural architecture search. Includes attention mechanisms, feed-forward networks, and various architectural choices.

public class TransformerSearchSpace<T> : SearchSpaceBase<T>

Type Parameters

T

The numeric type for calculations

Inheritance

object

SearchSpaceBase<T>

TransformerSearchSpace<T>

Inherited Members

SearchSpaceBase<T>.Operations

SearchSpaceBase<T>.MaxNodes

SearchSpaceBase<T>.InputChannels

SearchSpaceBase<T>.OutputChannels

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Constructors

TransformerSearchSpace()

public TransformerSearchSpace()

Properties

AttentionHeads

Number of attention heads to search over

public List<int> AttentionHeads { get; set; }

Property Value

List<int>

DropoutRates

Dropout rates to search over

public List<double> DropoutRates { get; set; }

Property Value

List<double>

FeedForwardMultipliers

Feed-forward expansion ratios

public List<int> FeedForwardMultipliers { get; set; }

Property Value

List<int>

HiddenDimensions

Hidden dimensions to consider

public List<int> HiddenDimensions { get; set; }

Property Value

List<int>

UsePreNorm

Whether to use Pre-LayerNorm (true) or Post-LayerNorm (false)

public bool UsePreNorm { get; set; }

Property Value

bool