Table of Contents

Class SceneClassifier<T>

Namespace
AiDotNet.Audio.Classification
Assembly
AiDotNet.dll

Acoustic scene classification model for identifying recording environments.

public class SceneClassifier<T> : AudioClassifierBase<T>, INeuralNetworkModel<T>, INeuralNetwork<T>, IInterpretableModel<T>, IInputGradientComputable<T>, IDisposable, ISceneClassifier<T>, IFullModel<T, Tensor<T>, Tensor<T>>, IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>, IModelSerializer, ICheckpointableModel, IParameterizable<T, Tensor<T>, Tensor<T>>, IFeatureAware, IFeatureImportance<T>, ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>, IGradientComputable<T, Tensor<T>, Tensor<T>>, IJitCompilable<T>

Type Parameters

T

The numeric type used for calculations.

Inheritance
SceneClassifier<T>
Implements
IFullModel<T, Tensor<T>, Tensor<T>>
IModel<Tensor<T>, Tensor<T>, ModelMetadata<T>>
IParameterizable<T, Tensor<T>, Tensor<T>>
ICloneable<IFullModel<T, Tensor<T>, Tensor<T>>>
IGradientComputable<T, Tensor<T>, Tensor<T>>
Inherited Members
Extension Methods

Remarks

Classifies audio recordings by their acoustic environment or scene context. Based on DCASE (Detection and Classification of Acoustic Scenes and Events) challenge.

For Beginners: Scene classification answers "Where was this recorded?":

  • Indoor: office, home, shopping mall, restaurant, library
  • Outdoor: street, park, beach, forest, construction site
  • Transportation: bus, train, metro, airport, car

Usage with ONNX model:

var classifier = new SceneClassifier<float>("model.onnx");
var result = classifier.Classify(audioTensor);
Console.WriteLine($"Scene: {result.PredictedScene} ({result.Confidence})");

Usage for training:

var architecture = new NeuralNetworkArchitecture<float>(inputFeatures: 60, outputSize: 30);
var classifier = new SceneClassifier<float>(architecture);
classifier.Train(features, labels);
var result = classifier.Classify(newAudio);

Constructors

SceneClassifier(SceneClassifierOptions?)

Creates a SceneClassifier with default options for basic classification.

public SceneClassifier(SceneClassifierOptions? options = null)

Parameters

options SceneClassifierOptions

Optional configuration options.

SceneClassifier(NeuralNetworkArchitecture<T>, SceneClassifierOptions?, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>>?)

Creates a SceneClassifier for native training mode.

public SceneClassifier(NeuralNetworkArchitecture<T> architecture, SceneClassifierOptions? options = null, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>>? optimizer = null)

Parameters

architecture NeuralNetworkArchitecture<T>

Neural network architecture.

options SceneClassifierOptions

Optional configuration options.

optimizer IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>>

Optional custom optimizer.

SceneClassifier(string, SceneClassifierOptions?)

Creates a SceneClassifier for ONNX inference mode.

public SceneClassifier(string modelPath, SceneClassifierOptions? options = null)

Parameters

modelPath string

Path to the ONNX model file.

options SceneClassifierOptions

Optional configuration options.

Fields

StandardScenes

Standard acoustic scene labels (DCASE-style).

public static readonly string[] StandardScenes

Field Value

string[]

Properties

MinimumDurationSeconds

Gets the minimum audio duration required for reliable classification.

public double MinimumDurationSeconds { get; }

Property Value

double

Scenes

Gets the scenes (alias for SupportedScenes for legacy API compatibility).

public IReadOnlyList<string> Scenes { get; }

Property Value

IReadOnlyList<string>

SupportedScenes

Gets the list of scenes this model can classify.

public IReadOnlyList<string> SupportedScenes { get; }

Property Value

IReadOnlyList<string>

Methods

Classify(Tensor<T>)

Classifies the acoustic scene of an audio recording.

public SceneClassificationResult<T> Classify(Tensor<T> audio)

Parameters

audio Tensor<T>

Returns

SceneClassificationResult<T>

ClassifyAsync(Tensor<T>, CancellationToken)

Classifies the acoustic scene asynchronously.

public Task<SceneClassificationResult<T>> ClassifyAsync(Tensor<T> audio, CancellationToken cancellationToken = default)

Parameters

audio Tensor<T>
cancellationToken CancellationToken

Returns

Task<SceneClassificationResult<T>>

ClassifyCategory(Tensor<T>)

Classifies audio and returns category with confidence (legacy API compatibility).

public (string Category, T Confidence) ClassifyCategory(Tensor<T> audio)

Parameters

audio Tensor<T>

Audio waveform tensor.

Returns

(string Label, T Confidence)

Tuple of (category, confidence).

CreateAsync(SceneClassifierOptions?, IProgress<double>?, CancellationToken)

Creates a SceneClassifier asynchronously with model download.

public static Task<SceneClassifier<T>> CreateAsync(SceneClassifierOptions? options = null, IProgress<double>? progress = null, CancellationToken cancellationToken = default)

Parameters

options SceneClassifierOptions
progress IProgress<double>
cancellationToken CancellationToken

Returns

Task<SceneClassifier<T>>

CreateNewInstance()

Creates a new instance of this network type.

protected override IFullModel<T, Tensor<T>, Tensor<T>> CreateNewInstance()

Returns

IFullModel<T, Tensor<T>, Tensor<T>>

DeserializeNetworkSpecificData(BinaryReader)

Deserializes network-specific data.

protected override void DeserializeNetworkSpecificData(BinaryReader reader)

Parameters

reader BinaryReader

Dispose(bool)

Disposes of managed resources.

protected override void Dispose(bool disposing)

Parameters

disposing bool

ExtractAcousticFeatures(Tensor<T>)

Extracts acoustic features used for scene classification.

public Tensor<T> ExtractAcousticFeatures(Tensor<T> audio)

Parameters

audio Tensor<T>

Returns

Tensor<T>

GetModelMetadata()

Gets model metadata for serialization.

public override ModelMetadata<T> GetModelMetadata()

Returns

ModelMetadata<T>

GetSceneProbabilities(Tensor<T>)

Gets scene probabilities for all supported scenes.

public IReadOnlyDictionary<string, T> GetSceneProbabilities(Tensor<T> audio)

Parameters

audio Tensor<T>

Returns

IReadOnlyDictionary<string, T>

GetTopScenes(Tensor<T>, int)

Gets top-K scene predictions.

public IReadOnlyList<ScenePrediction<T>> GetTopScenes(Tensor<T> audio, int k = 5)

Parameters

audio Tensor<T>
k int

Returns

IReadOnlyList<ScenePrediction<T>>

InitializeLayers()

Initializes the neural network layers.

protected override void InitializeLayers()

PostprocessOutput(Tensor<T>)

Postprocesses model output into final predictions.

protected override Tensor<T> PostprocessOutput(Tensor<T> modelOutput)

Parameters

modelOutput Tensor<T>

Returns

Tensor<T>

Predict(Tensor<T>)

Predicts scene probabilities from audio features.

public override Tensor<T> Predict(Tensor<T> input)

Parameters

input Tensor<T>

Returns

Tensor<T>

PreprocessAudio(Tensor<T>)

Preprocesses raw audio into model input format.

protected override Tensor<T> PreprocessAudio(Tensor<T> rawAudio)

Parameters

rawAudio Tensor<T>

Returns

Tensor<T>

SerializeNetworkSpecificData(BinaryWriter)

Serializes network-specific data.

protected override void SerializeNetworkSpecificData(BinaryWriter writer)

Parameters

writer BinaryWriter

TrackSceneChanges(Tensor<T>, double)

Tracks scene changes over time in longer audio.

public SceneTrackingResult<T> TrackSceneChanges(Tensor<T> audio, double segmentDuration = 10)

Parameters

audio Tensor<T>
segmentDuration double

Returns

SceneTrackingResult<T>

Train(Tensor<T>, Tensor<T>)

Trains the model on labeled audio samples.

public override void Train(Tensor<T> input, Tensor<T> expected)

Parameters

input Tensor<T>
expected Tensor<T>

UpdateParameters(Vector<T>)

Updates parameters from a flattened parameter vector.

public override void UpdateParameters(Vector<T> parameters)

Parameters

parameters Vector<T>
Edit this page