Table of Contents

Class ContextWindowManager

Namespace
AiDotNet.PromptEngineering
Assembly
AiDotNet.dll

Manages context window limits for LLM prompts, providing token estimation and text truncation utilities.

public class ContextWindowManager
Inheritance
ContextWindowManager
Inherited Members

Remarks

This class helps manage the token limits of large language models by providing utilities to estimate token counts, check if text fits within the context window, and truncate or split text that exceeds the limit.

For Beginners: Large Language Models have a maximum number of tokens they can process at once (the "context window"). This class helps you:

Example:

// Create a manager with 4096 token limit
var manager = new ContextWindowManager(4096);

// Check if your prompt fits
var prompt = "Your long prompt here...";
if (!manager.FitsInWindow(prompt))
{
    // Truncate to fit
    prompt = manager.TruncateToFit(prompt);
}

// Or split long text into chunks
var chunks = manager.SplitIntoChunks(longDocument);

A token is roughly 4 characters or 0.75 words in English, but varies by language and model.

Constructors

ContextWindowManager(int)

Initializes a new instance of the ContextWindowManager with the specified maximum tokens.

public ContextWindowManager(int maxTokens)

Parameters

maxTokens int

The maximum number of tokens allowed in the context window.

Remarks

Uses a default token estimator that approximates tokens as text.Length / 4, which is a reasonable approximation for English text.

For Beginners: Just specify the token limit for your model: 

// For GPT-4 with 8K context
var manager = new ContextWindowManager(8192);

// For Claude with 100K context
var manager = new ContextWindowManager(100000);

ContextWindowManager(int, Func<string, int>)

Initializes a new instance of the ContextWindowManager with a custom token estimator.

public ContextWindowManager(int maxTokens, Func<string, int> tokenEstimator)

Parameters

maxTokens int

The maximum number of tokens allowed in the context window.

tokenEstimator Func<string, int>

A function that estimates the number of tokens in a given text.

Remarks

Use this constructor when you need precise token counting, such as when using a specific tokenizer like tiktoken for OpenAI models.

For Beginners: Use a custom estimator for more accurate counting: 

// Simple character-based estimator
var manager = new ContextWindowManager(4096, text => text.Length / 4);

// Word-based estimator
var manager = new ContextWindowManager(4096, text =>
    text.Split(' ', StringSplitOptions.RemoveEmptyEntries).Length * 4 / 3);

Properties

MaxTokens

Gets the maximum number of tokens allowed in the context window.

public int MaxTokens { get; }

Property Value

int

Remarks

For Beginners: This is the total capacity of the model's context window. Common values are 4096, 8192, 16384, 32768, or 128000 depending on the model.

Methods

EstimateTokens(string)

Estimates the number of tokens in the given text.

public int EstimateTokens(string text)

Parameters

text string

The text to estimate tokens for.

Returns

int

The estimated number of tokens.

Remarks

The accuracy depends on the token estimator provided. The default estimator uses a simple character-based approximation.

For Beginners: Use this to check how many tokens your text uses: 

var tokens = manager.EstimateTokens("Hello, world!");
Console.WriteLine($"This text uses approximately {tokens} tokens");

FitsInWindow(string, int)

Checks if the given text fits within the context window.

public bool FitsInWindow(string text, int reservedTokens = 0)

Parameters

text string

The text to check.

reservedTokens int

Number of tokens to reserve for other content (e.g., system prompt, response).

Returns

bool

True if the text fits within the available window space; otherwise, false.

Remarks

The reserved tokens parameter allows you to account for tokens that will be used by system prompts, few-shot examples, or expected response length.

For Beginners: Check if your prompt will fit: 

// Basic check
if (manager.FitsInWindow(myPrompt))
{
    // Safe to send to the model
}

// Reserve space for the response
if (manager.FitsInWindow(myPrompt, reservedTokens: 500))
{
    // Prompt fits with 500 tokens reserved for response
}

RemainingTokens(string, int)

Calculates the remaining tokens available after accounting for the given text.

public int RemainingTokens(string text, int reservedTokens = 0)

Parameters

text string

The text currently using the window.

reservedTokens int

Number of tokens to reserve for other content.

Returns

int

The number of remaining tokens available, or 0 if exceeded.

Remarks

This is useful for determining how much additional content can be added to a prompt.

For Beginners: Find out how much space is left: 

var remaining = manager.RemainingTokens(currentPrompt);
Console.WriteLine($"You can add approximately {remaining} more tokens");

SplitIntoChunks(string, int)

Splits the text into chunks that each fit within the context window.

public List<string> SplitIntoChunks(string text, int overlapTokens = 0)

Parameters

text string

The text to split.

overlapTokens int

Number of tokens to overlap between chunks for context continuity.

Returns

List<string>

A list of text chunks that each fit within the window.

Remarks

This is useful for processing long documents that exceed the context window. The overlap parameter allows maintaining context between chunks.

For Beginners: Process long documents in pieces: 

var document = LoadLongDocument();
var chunks = manager.SplitIntoChunks(document);

foreach (var chunk in chunks)
{
    var response = await model.ProcessAsync(chunk);
    // Handle each chunk's response
}

// With overlap for better context continuity
var overlappingChunks = manager.SplitIntoChunks(document, overlapTokens: 100);

TruncateToFit(string, int)

Truncates the text to fit within the context window.

public string TruncateToFit(string text, int reservedTokens = 0)

Parameters

text string

The text to truncate.

reservedTokens int

Number of tokens to reserve for other content.

Returns

string

The truncated text that fits within the window, or the original if it already fits.

Remarks

Uses binary search to find the optimal truncation point, ensuring the result fits within the available token budget.

For Beginners: Automatically shorten text to fit: 

var longDocument = LoadDocument(); // Very long text
var fittingText = manager.TruncateToFit(longDocument);
// fittingText is guaranteed to fit in the context window
Edit this page