Table of Contents

Namespace AiDotNet.Tokenization.Models

Classes

EncodingOptions

Options for encoding text into tokens.

SpecialTokens

Represents special tokens used by tokenizers.

TokenizationResult

Represents the result of tokenizing text, including token IDs, tokens, and attention masks.