owl with large round glasses pecking away at a keyboard while writing codes

How to Count Tokens with LangChain

by

Table of Contents

Introduction

When working with AI language models, tracking token usage is crucial for managing costs and ensuring optimal performance. While LangChain provides methods for token counting, it’s worth examining whether this approach is the most efficient solution for your needs.

The LangChain Approach

LangChain offers token counting through its callback system. Here’s how you can implement token counting in LangChain:

import asyncio
from langchain_community.callbacks import get_openai_callback
from langchain_openai import OpenAI

# Initialize the LLM
llm = OpenAI(temperature=0)

# Basic token counting
with get_openai_callback() as cb:
    llm.invoke("What is the square root of 4?")
    total_tokens = cb.total_tokens

# Multiple calls
with get_openai_callback() as cb:
    llm.invoke("What is the square root of 4?")
    llm.invoke("What is the square root of 4?")
    # Total tokens will be doubled

# Async implementation
async def count_tokens_async():
    with get_openai_callback() as cb:
        await asyncio.gather(
            *[llm.agenerate(["What is the square root of 4?"]) for _ in range(3)]
        )
    return cb.total_tokens

A Better Alternative

While LangChain’s token counting works, it’s worth noting that the framework often introduces unnecessary complexity through multiple layers of abstraction. Many developers, including our team at Stammer.ai, have found that implementing direct API calls with simple wrappers is more maintainable and easier to debug.

For those looking for a straightforward solution to token counting, TokenCounter.co offers a user-friendly interface that:

  • Provides instant token counts
  • Estimates costs across multiple AI models
  • Requires no code implementation
  • Supports various AI models (with more being added regularly)

Important Considerations

When deciding how to handle token counting in your application, consider these factors:

  1. Complexity vs. Necessity: LangChain’s abstractions might be overkill for simple applications. Most AI providers (except Anthropic’s Claude) share similar API patterns that can be handled with minimal code.
  2. Maintenance Overhead: Direct API implementations are often easier to maintain and debug compared to framework-dependent solutions.
  3. Flexibility: Using simpler implementations makes it easier to switch between different AI providers or update your code as APIs evolve.

Conclusion

While LangChain provides a functional way to count tokens, it’s worth considering whether its complexity aligns with your project’s needs. For many developers, either implementing a simple direct solution or using a dedicated tool like TokenCounter.co might be more appropriate.

Need support for additional AI models in TokenCounter.co? Let us know, and we’ll prioritize adding them to our platform. Our goal is to make token counting as simple and accessible as possible for developers working with AI models.