Skip to content

Commit 7d532db

Browse files
Update genai/count_tokens/count_tokens_compute_with_txt.py
Co-authored-by: Sampath Kumar <[email protected]>
1 parent ae0786e commit 7d532db

File tree

1 file changed

+5
-1
lines changed

1 file changed

+5
-1
lines changed

genai/count_tokens/count_tokens_compute_with_txt.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,11 @@ def compute_tokens() -> int:
2626
print(response)
2727

2828
# Example output:
29-
# tokens_info=[TokensInfo(role='user', token_ids=[1841, 235303, 235256, 573, 32514, 2204, 575, 573, 4645, 5255, 235336], tokens=[b'What', b"'", b's', b' the', b' longest', b' word', b' in', b' the', b' English', b' language', b'?'])]
29+
# tokens_info=[TokensInfo(
30+
# role='user',
31+
# token_ids=[1841, 235303, 235256, 573, 32514, 2204, 575, 573, 4645, 5255, 235336],
32+
# tokens=[b'What', b"'", b's', b' the', b' longest', b' word', b' in', b' the', b' English', b' language', b'?']
33+
# )]
3034

3135
# [END googlegenaisdk_count_tokens_compute_with_txt]
3236
return response.tokens_info

0 commit comments

Comments
 (0)