Table of Contents

Class TokenCountAnalyzer

Namespace
AiDotNet.PromptEngineering.Analysis
Assembly
AiDotNet.dll

Analyzer that provides accurate token counting and cost estimation for prompts.

public class TokenCountAnalyzer : PromptAnalyzerBase, IPromptAnalyzer
Inheritance
TokenCountAnalyzer
Implements
Inherited Members

Remarks

This analyzer focuses on token counting and cost estimation, supporting various tokenization methods and model-specific pricing.

For Beginners: Counts how many tokens your prompt uses and estimates costs.

Example:

var analyzer = new TokenCountAnalyzer("gpt-4");
var metrics = analyzer.Analyze("Explain quantum computing in simple terms");

Console.WriteLine($"Tokens: {metrics.TokenCount}");
Console.WriteLine($"Estimated cost: ${metrics.EstimatedCost}");

Supports different models with their pricing:

  • GPT-4: $0.03/1K tokens
  • GPT-3.5-Turbo: $0.001/1K tokens
  • Claude: $0.008/1K tokens

Constructors

TokenCountAnalyzer(string, Func<string, int>?)

Initializes a new instance of the TokenCountAnalyzer class.

public TokenCountAnalyzer(string modelName = "gpt-4", Func<string, int>? tokenCounter = null)

Parameters

modelName string

The target model for token counting.

tokenCounter Func<string, int>

Optional custom token counter function.

Methods

ForClaude()

Creates an analyzer pre-configured for Claude.

public static TokenCountAnalyzer ForClaude()

Returns

TokenCountAnalyzer

ForGemini()

Creates an analyzer pre-configured for Gemini.

public static TokenCountAnalyzer ForGemini()

Returns

TokenCountAnalyzer

ForGpt35Turbo()

Creates an analyzer pre-configured for GPT-3.5-Turbo.

public static TokenCountAnalyzer ForGpt35Turbo()

Returns

TokenCountAnalyzer

ForGpt4()

Creates an analyzer pre-configured for GPT-4.

public static TokenCountAnalyzer ForGpt4()

Returns

TokenCountAnalyzer

GetModelPrice(string)

Gets the price per 1000 tokens for a given model.

public static decimal GetModelPrice(string modelName)

Parameters

modelName string

The model name.

Returns

decimal

The price per 1000 tokens in USD.

GetSupportedModels()

Gets the list of supported models.

public static IReadOnlyList<string> GetSupportedModels()

Returns

IReadOnlyList<string>

A list of model names with known pricing.

Edit this page