File tree Expand file tree Collapse file tree 3 files changed +5
-2
lines changed
src/openai_messages_token_helper Expand file tree Collapse file tree 3 files changed +5
-2
lines changed Original file line number Diff line number Diff line change 2
2
3
3
All notable changes to this project will be documented in this file.
4
4
5
+ ## [ 0.1.5] - June 4, 2024
6
+
7
+ - Remove spurious ` print ` call when counting tokens for function calling.
8
+
5
9
## [ 0.1.4] - May 14, 2024
6
10
7
11
- Add support and tests for gpt-4o, which has a different tokenizer.
Original file line number Diff line number Diff line change 1
1
[project ]
2
2
name = " openai-messages-token-helper"
3
3
description = " A helper library for estimating tokens used by messages sent through OpenAI Chat Completions API."
4
- version = " 0.1.4 "
4
+ version = " 0.1.5 "
5
5
authors = [{name = " Pamela Fox" }]
6
6
requires-python = " >=3.9"
7
7
readme = " README.md"
Original file line number Diff line number Diff line change @@ -145,7 +145,6 @@ def count_tokens_for_system_and_tools(
145
145
tokens += count_tokens_for_message (model , system_message , default_to_cl100k )
146
146
if tools :
147
147
encoding = tiktoken .encoding_for_model (model )
148
- print (format_function_definitions (tools ))
149
148
tokens += len (encoding .encode (format_function_definitions (tools )))
150
149
tokens += 9 # Additional tokens for function definition of tools
151
150
# If there's a system message and tools are present, subtract four tokens
You can’t perform that action at this time.
0 commit comments