Skip to content

Report LLM token usage #991

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
clavedeluna merged 2 commits into from
Feb 6, 2025
Merged

Report LLM token usage #991

clavedeluna merged 2 commits into from
Feb 6, 2025

Conversation

@clavedeluna
Copy link
Contributor

@clavedeluna clavedeluna commented Feb 6, 2025

Any codemod which may use an LLM and spends prompt and completion tokens will now be reported, per each codemod and at the end of codemodder run.

Looks something like this (redacted)

Codemod REDACTED
	completion_tokens = 2309
	prompt_tokens = 13742
...
running codemod SOME OTHER CODEMOD
...
...
[report]
scanned: 8 files
failed: 0 files (0 unique)
changed: 8 files (4 unique)
report file: here.txt
All token usage
	completion_tokens = 5176
	prompt_tokens = 40960

@pixeebot pixeebot bot mentioned this pull request Feb 6, 2025
Closed
andrecsilva
andrecsilva approved these changes Feb 6, 2025
View reviewed changes
src/codemodder/llm.py
if not AzureOpenAI:
logger.info("Azure OpenAI API client not available")
return None
logger.info("Azure OpenAI API client not available")
Copy link
Contributor

@andrecsilva andrecsilva Feb 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing this is a mistake, this log is unreacheable and will never execute.

Copy link
Contributor Author

@clavedeluna clavedeluna Feb 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wooops yes very much

@clavedeluna clavedeluna enabled auto-merge February 6, 2025 18:03
@sonarqubecloud
Copy link

sonarqubecloud bot commented Feb 6, 2025

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@clavedeluna clavedeluna added this pull request to the merge queue Feb 6, 2025
Merged via the queue into main with commit b2baa38 Feb 6, 2025
15 checks passed
@clavedeluna clavedeluna deleted the report-token-usage branch February 6, 2025 18:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@drdavella drdavella drdavella approved these changes

@andrecsilva andrecsilva andrecsilva approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@clavedeluna @drdavella @andrecsilva