Class FusedGELUOp

Namespace

AiDotNet.JitCompiler.IR.Operations

Assembly

AiDotNet.dll

Fused GELU activation operation.

public class FusedGELUOp : IROp

Inheritance

object

IROp

FusedGELUOp

Inherited Members

IROp.OutputIds

IROp.OutputId

IROp.InputIds

IROp.OutputType

IROp.OutputShape

IROp.OutputShapes

IROp.OpType

IROp.ToString()

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

Remarks

For Beginners: Gaussian Error Linear Unit.

GELU(x) = x * Phi(x), where Phi is the standard Gaussian CDF. Approximation: 0.5 * x * (1 + tanh(sqrt(2/pi) * (x + 0.044715 * x^3)))

Very popular in transformers (BERT, GPT, etc.)

Properties

Approximate

Whether to use the approximate version.

public bool Approximate { get; set; }

Property Value

bool

Methods

Validate()

Validates inputs (single input).

public override bool Validate()

Returns

bool