LLM Token Counter #1649
Description
What feature or new tool do you think should be added to DevToys?
A tool that given a piece of text estimates the number of tokens
Why do you think this is needed?
Lots of developers now either code using AI, or are writing AI agents, sometimes it can be good to know how many tokens a prompt contains, or to estimate what requests look like.
Solution/Idea
A large text box optimized for large pieces of text (up to 750k words) that invokes https://github.com/openai/tiktoken to return an estimate token count for OpenAI models, support for Anthropic and Google is harder and not official so could be added later.
Comments
No response