Namespace AiDotNet.KnowledgeDistillation.Strategies
Classes
- AccuracyBasedAdaptiveStrategy<T>
Adaptive distillation strategy that adjusts temperature based on student accuracy.
- AdaptiveDistillationStrategyBase<T>
Abstract base class for adaptive distillation strategies with performance tracking.
- AttentionDistillationStrategy<T>
Implements attention-based knowledge distillation for transformer models. Transfers knowledge through attention patterns rather than just final outputs.
- ConfidenceBasedAdaptiveStrategy<T>
Adaptive distillation strategy that adjusts temperature based on student confidence.
- ContrastiveDistillationStrategy<T>
Implements Contrastive Representation Distillation (CRD) which transfers knowledge through contrastive learning of sample relationships rather than just matching outputs.
- CurriculumDistillationStrategyBase<T>
Abstract base class for curriculum distillation strategies with progressive difficulty adjustment.
- EasyToHardCurriculumStrategy<T>
Curriculum distillation strategy that progresses from easy to hard samples.
- EntropyBasedAdaptiveStrategy<T>
Adaptive distillation strategy that adjusts temperature based on prediction entropy.
- FlowBasedDistillationStrategy<T>
Flow-based distillation that matches the information flow between layers.
- HardToEasyCurriculumStrategy<T>
Curriculum distillation strategy that progresses from hard to easy samples.
- HybridDistillationStrategy<T>
Hybrid distillation strategy that combines multiple distillation strategies with configurable weights.
- NeuronSelectivityDistillationStrategy<T>
Neuron selectivity distillation that transfers the activation patterns and selectivity of individual neurons.
- ProbabilisticDistillationStrategy<T>
Probabilistic distillation that transfers distributional knowledge by matching statistical properties.
- SimilarityPreservingStrategy<T>
Similarity-preserving distillation that preserves pairwise similarity structure.
- VariationalDistillationStrategy<T>
Variational distillation based on variational inference principles and information theory.
Interfaces
- IAdaptiveDistillationStrategy<T>
Interface for adaptive distillation strategies that adjust temperature based on student performance.
- ICurriculumDistillationStrategy<T>
Interface for curriculum distillation strategies that progressively adjust training difficulty.
Enums
- AttentionMatchingMode
Defines how to match attention patterns between teacher and student.
- ContrastiveMode
Defines the contrastive learning mode.
- RelationalDistanceMetric
Distance metrics for relational knowledge distillation.