Table of Contents

Class SileroVad<T>

Namespace
AiDotNet.Audio.VoiceActivity
Assembly
AiDotNet.dll

Silero Voice Activity Detection model - high accuracy neural network VAD.

public class SileroVad<T> : AudioNeuralNetworkBase<T>, INeuralNetworkModel<T>, INeuralNetwork<T>, IFullModel<T, Tensor<T>, Tensor<T>>, IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>, IModelSerializer, ICheckpointableModel, IParameterizable<T, Tensor<T>, Tensor<T>>, IFeatureAware, IFeatureImportance<T>, ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>, IGradientComputable<T, Tensor<T>, Tensor<T>>, IJitCompilable<T>, IInterpretableModel<T>, IInputGradientComputable<T>, IDisposable, IVoiceActivityDetector<T>

Type Parameters

T

The numeric type used for calculations.

Inheritance
SileroVad<T>
Implements
IFullModel<T, Tensor<T>, Tensor<T>>
IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>
IParameterizable<T, Tensor<T>, Tensor<T>>
ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>
IGradientComputable<T, Tensor<T>, Tensor<T>>
Inherited Members
Extension Methods

Remarks

Silero VAD is a state-of-the-art voice activity detector that uses a lightweight neural network architecture to achieve high accuracy with low latency. It can:

  • Detect speech with very high accuracy (better than energy-based methods)
  • Handle noisy environments well
  • Run in real-time on CPU
  • Work across multiple languages

For Beginners: Silero VAD tells you when someone is speaking vs silence. Unlike simple energy-based VAD, it uses a neural network that has learned what speech "looks like" from millions of examples.

Why use neural network VAD?

  • Much more accurate than energy/threshold-based methods
  • Handles background noise better (music, crowd noise, etc.)
  • Detects speech even when quiet
  • Doesn't false-trigger on non-speech sounds

Two ways to use this class:

  1. ONNX Mode: Load pretrained Silero model for fast inference
  2. Native Mode: Train your own VAD model from scratch

ONNX Mode Example (recommended):

var vad = new SileroVad<float>(
    architecture,
    modelPath: "path/to/silero_vad.onnx");
var (isSpeech, probability) = vad.ProcessChunk(audioFrame);
if (isSpeech)
    Console.WriteLine($"Speech detected! Confidence: {probability:P0}");

Training Mode Example:

var vad = new SileroVad<float>(architecture);
for (int epoch = 0; epoch < 100; epoch++)
{
    foreach (var (audio, labels) in trainingData)
    {
        vad.Train(audio, labels);
    }
}

Constructors

SileroVad(NeuralNetworkArchitecture<T>, int, int, double, int, int, int, int, int)

Creates a Silero VAD in native training mode for training from scratch.

public SileroVad(NeuralNetworkArchitecture<T> architecture, int sampleRate = 16000, int frameSize = 512, double threshold = 0.5, int minSpeechDurationMs = 250, int minSilenceDurationMs = 100, int convFilters = 64, int lstmHiddenDim = 64, int numLstmLayers = 2)

Parameters

architecture NeuralNetworkArchitecture<T>

The neural network architecture.

sampleRate int

Expected sample rate (default: 16000 Hz).

frameSize int

Frame size in samples (default: 512).

threshold double

Detection threshold 0-1 (default: 0.5).

minSpeechDurationMs int

Minimum speech duration in ms (default: 250).

minSilenceDurationMs int

Minimum silence duration in ms (default: 100).

convFilters int

Number of convolutional filters (default: 64).

lstmHiddenDim int

LSTM hidden dimension (default: 64).

numLstmLayers int

Number of LSTM layers (default: 2).

SileroVad(NeuralNetworkArchitecture<T>, string, int, int, double, int, int)

Creates a Silero VAD in ONNX inference mode with a pretrained model.

public SileroVad(NeuralNetworkArchitecture<T> architecture, string modelPath, int sampleRate = 16000, int frameSize = 512, double threshold = 0.5, int minSpeechDurationMs = 250, int minSilenceDurationMs = 100)

Parameters

architecture NeuralNetworkArchitecture<T>

The neural network architecture.

modelPath string

Path to the Silero VAD ONNX model.

sampleRate int

Expected sample rate (default: 16000 Hz).

frameSize int

Frame size in samples (default: 512).

threshold double

Detection threshold 0-1 (default: 0.5).

minSpeechDurationMs int

Minimum speech duration in ms (default: 250).

minSilenceDurationMs int

Minimum silence duration in ms (default: 100).

Properties

FrameSize

Gets the frame size in samples used for detection.

public int FrameSize { get; }

Property Value

int

MinSilenceDurationMs

Gets or sets the minimum silence duration in milliseconds.

public int MinSilenceDurationMs { get; set; }

Property Value

int

Remarks

Silence gaps shorter than this don't split speech segments.

MinSpeechDurationMs

Gets or sets the minimum speech duration in milliseconds.

public int MinSpeechDurationMs { get; set; }

Property Value

int

Remarks

Speech segments shorter than this are ignored (reduces false triggers).

Threshold

Gets or sets the detection threshold (0.0 to 1.0).

public double Threshold { get; set; }

Property Value

double

Remarks

Higher threshold = fewer false positives but may miss quiet speech. Lower threshold = catches more speech but may trigger on noise. Default is typically 0.5.

Methods

CreateNewInstance()

Creates a new instance of the same type as this neural network.

protected override IFullModel<T, Tensor<T>, Tensor<T>> CreateNewInstance()

Returns

IFullModel<T, Tensor<T>, Tensor<T>>

A new instance of the same neural network type.

Remarks

For Beginners: This creates a blank version of the same type of neural network.

It's used internally by methods like DeepCopy and Clone to create the right type of network before copying the data into it.

DeserializeNetworkSpecificData(BinaryReader)

Deserializes network-specific data that was not covered by the general deserialization process.

protected override void DeserializeNetworkSpecificData(BinaryReader reader)

Parameters

reader BinaryReader

The BinaryReader to read the data from.

Remarks

This method is called at the end of the general deserialization process to allow derived classes to read any additional data specific to their implementation.

For Beginners: Continuing the suitcase analogy, this is like unpacking that special compartment. After the main deserialization method has unpacked the common items (layers, parameters), this method allows each specific type of neural network to unpack its own unique items that were stored during serialization.

DetectSpeech(Tensor<T>)

Detects whether speech is present in an audio frame.

public bool DetectSpeech(Tensor<T> audioFrame)

Parameters

audioFrame Tensor<T>

Audio frame with shape [samples] or [channels, samples].

Returns

bool

True if speech is detected, false otherwise.

DetectSpeechSegments(Tensor<T>)

Detects speech segments in a longer audio recording.

public IReadOnlyList<(int StartSample, int EndSample)> DetectSpeechSegments(Tensor<T> audio)

Parameters

audio Tensor<T>

Full audio recording.

Returns

IReadOnlyList<(int StartSample, int EndSample)>

List of (startSample, endSample) tuples for each speech segment.

Remarks

For Beginners: This finds all the parts where someone is talking.

Example result for a 10-second recording: [(0.5s, 2.3s), (4.1s, 6.8s), (8.0s, 9.5s)] Meaning: Speech from 0.5-2.3s, silence, speech from 4.1-6.8s, etc.

Dispose(bool)

Disposes of resources used by this model.

protected override void Dispose(bool disposing)

Parameters

disposing bool

True if disposing managed resources.

Forward(Tensor<T>)

Performs a forward pass through the native neural network layers.

protected override Tensor<T> Forward(Tensor<T> input)

Parameters

input Tensor<T>

Preprocessed input tensor.

Returns

Tensor<T>

Model output tensor.

GetFrameProbabilities(Tensor<T>)

Gets frame-by-frame speech probabilities for the entire audio.

public T[] GetFrameProbabilities(Tensor<T> audio)

Parameters

audio Tensor<T>

Full audio recording.

Returns

T[]

Array of speech probabilities, one per frame.

GetModelMetadata()

Gets the metadata for this neural network model.

public override ModelMetadata<T> GetModelMetadata()

Returns

ModelMetadata<T>

A ModelMetaData object containing information about the model.

GetSpeechProbability(Tensor<T>)

Gets the speech probability for an audio frame.

public T GetSpeechProbability(Tensor<T> audioFrame)

Parameters

audioFrame Tensor<T>

Audio frame to analyze.

Returns

T

Probability of speech (0.0 = definitely not speech, 1.0 = definitely speech).

InitializeLayers()

Initializes the layers of the neural network based on the architecture.

protected override void InitializeLayers()

Remarks

For Beginners: This method sets up all the layers in your neural network according to the architecture you've defined. It's like assembling the parts of your network before you can use it.

PostprocessOutput(Tensor<T>)

Postprocesses model output into the final result format.

protected override Tensor<T> PostprocessOutput(Tensor<T> modelOutput)

Parameters

modelOutput Tensor<T>

Raw output from the model.

Returns

Tensor<T>

Postprocessed output in the expected format.

Predict(Tensor<T>)

Makes a prediction using the neural network.

public override Tensor<T> Predict(Tensor<T> input)

Parameters

input Tensor<T>

The input data to process.

Returns

Tensor<T>

The network's prediction.

Remarks

For Beginners: This is the main method you'll use to get results from your trained neural network. You provide some input data (like an image or text), and the network processes it through all its layers to produce an output (like a classification or prediction).

PreprocessAudio(Tensor<T>)

Preprocesses raw audio for model input.

protected override Tensor<T> PreprocessAudio(Tensor<T> rawAudio)

Parameters

rawAudio Tensor<T>

Raw audio waveform tensor [samples] or [batch, samples].

Returns

Tensor<T>

Preprocessed audio features suitable for model input.

Remarks

For Beginners: Raw audio is just a series of numbers representing sound pressure. Neural networks often work better with transformed representations like mel spectrograms. This method converts raw audio into the format the model expects.

ProcessChunk(Tensor<T>)

Processes audio in streaming mode, maintaining state between calls.

public (bool IsSpeech, T Probability) ProcessChunk(Tensor<T> audioChunk)

Parameters

audioChunk Tensor<T>

A chunk of audio for real-time processing.

Returns

(bool SameLanguage, T Confidence)

Speech detection result with probability.

SerializeNetworkSpecificData(BinaryWriter)

Serializes network-specific data that is not covered by the general serialization process.

protected override void SerializeNetworkSpecificData(BinaryWriter writer)

Parameters

writer BinaryWriter

The BinaryWriter to write the data to.

Remarks

This method is called at the end of the general serialization process to allow derived classes to write any additional data specific to their implementation.

For Beginners: Think of this as packing a special compartment in your suitcase. While the main serialization method packs the common items (layers, parameters), this method allows each specific type of neural network to pack its own unique items that other networks might not have.

Train(Tensor<T>, Tensor<T>)

Trains the neural network on a single input-output pair.

public override void Train(Tensor<T> input, Tensor<T> expectedOutput)

Parameters

input Tensor<T>

The input data.

expectedOutput Tensor<T>

The expected output for the given input.

Remarks

This method performs one training step on the neural network using the provided input and expected output. It updates the network's parameters to reduce the error between the network's prediction and the expected output.

For Beginners: This is how your neural network learns. You provide: - An input (what the network should process) - The expected output (what the correct answer should be)

The network then:

  1. Makes a prediction based on the input
  2. Compares its prediction to the expected output
  3. Calculates how wrong it was (the loss)
  4. Adjusts its internal values to do better next time

After training, you can get the loss value using the GetLastLoss() method to see how well the network is learning.

UpdateParameters(Vector<T>)

Updates the network's parameters with new values.

public override void UpdateParameters(Vector<T> parameters)

Parameters

parameters Vector<T>

The new parameter values to set.

Remarks

For Beginners: During training, a neural network's internal values (parameters) get adjusted to improve its performance. This method allows you to update all those values at once by providing a complete set of new parameters.

This is typically used by optimization algorithms that calculate better parameter values based on training data.

Edit this page