Skip to content

Add token usage logging and dashboard for inference plugin#260956

Open
sphilipse wants to merge 2 commits intoelastic:mainfrom
sphilipse:inference-token-usage-logger
Open

Add token usage logging and dashboard for inference plugin#260956
sphilipse wants to merge 2 commits intoelastic:mainfrom
sphilipse:inference-token-usage-logger

Conversation

@sphilipse
Copy link
Copy Markdown
Member

  • Add TokenUsageLogger service that logs token usage to Elasticsearch
  • Add handleTokenUsageLogging operator for chat completion pipelines
  • Extend Model type with creator, name, and platform fields
  • Extend InferenceEndpoint with metadata (display name, creator)
  • Install token usage dashboard via saved objects on plugin start
  • Thread tokenUsageLogger through inference client creation chain

Summary

Summarize your PR. If it involves visual changes include a screenshot or gif.

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

  • Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
  • Documentation was added for features that require explanation or tutorials
  • Unit or functional tests were updated or added to match the most common scenarios
  • If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
  • This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
  • Flaky Test Runner was used on any tests changed
  • The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
  • Review the backport guidelines and apply applicable backport:* labels.

Identify risks

Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.

- Add TokenUsageLogger service that logs token usage to Elasticsearch
- Add handleTokenUsageLogging operator for chat completion pipelines
- Extend Model type with creator, name, and platform fields
- Extend InferenceEndpoint with metadata (display name, creator)
- Install token usage dashboard via saved objects on plugin start
- Thread tokenUsageLogger through inference client creation chain

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@sphilipse sphilipse marked this pull request as ready for review April 2, 2026 15:46
@sphilipse sphilipse requested a review from a team as a code owner April 2, 2026 15:46
@sphilipse sphilipse added backport:skip This PR does not require backporting 9.4.0 release_note:feature Makes this part of the condensed release notes labels Apr 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

9.4.0 backport:skip This PR does not require backporting release_note:feature Makes this part of the condensed release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants