Skip to content

feat: add aspects charts#193

Open
Henrrypg wants to merge 1 commit intoopenedx:mainfrom
eduNEXT:hpg/aspects-charts
Open

feat: add aspects charts#193
Henrrypg wants to merge 1 commit intoopenedx:mainfrom
eduNEXT:hpg/aspects-charts

Conversation

@Henrrypg
Copy link
Copy Markdown
Contributor

This pull request standardizes how LLM (Large Language Model) usage data (such as token counts) is captured, serialized, and emitted in workflow events and API responses throughout the codebase. It replaces the previous tokens_used field with a more comprehensive usage dictionary/object, ensures this data is JSON-serializable for xAPI tracking, and updates orchestrators and tests accordingly.

LLM Usage Data Handling and Serialization:

  • Replaces the tokens_used field with a usage dictionary/object in all LLM processor responses and orchestrators, ensuring richer and more flexible usage tracking. [1] [2] [3] [4] [5] [6]
  • Adds a _convert_usage_to_json_serializable method to BaseOrchestrator to handle serialization of usage data, including support for Pydantic models, so that all emitted event payloads are JSON-serializable.

Workflow Event Emission and xAPI Integration:

  • Modifies all orchestrators to emit workflow events with the new usage field, and updates the event emission logic to include usage data only when present. [1] [2] [3] [4] [5]
  • Adds a new xAPI extension constant for usage data and updates the xAPI transformer to include usage information in the event extensions if available. [1] [2]

Testing Updates:

  • Expands and adds tests to verify correct event emission, including context handling, usage serialization (including Pydantic objects), and ensures that the usage field is only included when appropriate.
  • Updates or removes references to the deprecated tokens_used and model_used metadata fields in orchestrator and LLM processor tests. [1] [2] [3] [4]

Charts

  1. Total Tokens By Course
image
  1. Event Count by Workflow
image
  1. Daily Token Consumption Trend
image
  1. Unique Users by Course
image
  1. Event Count by Verb
image

@openedx-webhooks
Copy link
Copy Markdown

Thanks for the pull request, @Henrrypg!

This repository is currently maintained by @felipemontoya.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

  • If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
    • This process (including the steps you'll need to take) is documented here.
  • If it doesn't, simply proceed with the next step.
🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

  • Dependencies

    This PR must be merged before / after / at the same time as ...

  • Blockers

    This PR is waiting for OEP-1234 to be accepted.

  • Timeline information

    This PR must be merged by XX date because ...

  • Partner information

    This is for a course on edx.org.

  • Supporting documentation
  • Relevant Open edX discussion forum threads
🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details
Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

  • The size and impact of the changes that it introduces
  • The need for product review
  • Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

@openedx-webhooks openedx-webhooks added open-source-contribution PR author is not from Axim or 2U core contributor PR author is a Core Contributor (who may or may not have write access to this repo). labels Mar 27, 2026
@github-project-automation github-project-automation bot moved this to Needs Triage in Contributions Mar 27, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 27, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.33%. Comparing base (35df7a4) to head (9b3d2ac).
⚠️ Report is 23 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #193      +/-   ##
==========================================
- Coverage   94.34%   94.33%   -0.01%     
==========================================
  Files          60       60              
  Lines        6858     6846      -12     
  Branches      370      370              
==========================================
- Hits         6470     6458      -12     
  Misses        296      296              
  Partials       92       92              
Flag Coverage Δ
unittests 94.33% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Henrrypg Henrrypg force-pushed the hpg/aspects-charts branch from 5b56ced to 9b3d2ac Compare March 27, 2026 21:19
@Henrrypg Henrrypg requested a review from felipemontoya March 27, 2026 22:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core contributor PR author is a Core Contributor (who may or may not have write access to this repo). open-source-contribution PR author is not from Axim or 2U

Projects

Status: Needs Triage

Development

Successfully merging this pull request may close these issues.

2 participants