|
| 1 | +# Track language model usage and costs |
| 2 | + |
| 3 | +## Summary |
| 4 | + |
| 5 | +Sample web app that demonstrates how to use Dev Proxy to monitor and track LLM usage and costs. |
| 6 | + |
| 7 | + |
| 8 | + |
| 9 | +The sample showcases: |
| 10 | + |
| 11 | +- **LLM Cost Tracking**: Monitor token usage and costs for OpenAI API calls |
| 12 | +- **Functional API Integration**: Serve test data using CrudApiPlugin |
| 13 | +- **Latency Simulation**: Add realistic delays to API responses |
| 14 | +- **VS Code Integration**: Use Dev Proxy Toolkit for local development and testing |
| 15 | +- **End-to-End Testing**: Playwright tests that verify the complete AI pipeline |
| 16 | +- **CI/CD Integration**: GitHub Actions workflow with automated testing |
| 17 | + |
| 18 | +## Compatibility |
| 19 | + |
| 20 | + |
| 21 | + |
| 22 | +## Contributors |
| 23 | + |
| 24 | +* [Garry Trinder](https://github.com/garrytrinder) |
| 25 | + |
| 26 | +## Version history |
| 27 | + |
| 28 | +Version|Date|Comments |
| 29 | +-------|----|-------- |
| 30 | +1.0|July 28, 2025|Initial release |
| 31 | + |
| 32 | +## Minimal path to awesome |
| 33 | + |
| 34 | +1. Clone this repository (or [download this solution as a .ZIP file](https://pnp.github.io/download-partial/?url=https://github.com/pnp/proxy-samples/tree/main/samples/llm-usage) then unzip it) |
| 35 | +1. Open the repository in Visual Studio Code |
| 36 | +1. Install the [Dev Proxy Toolkit](https://marketplace.visualstudio.com/items?itemName=garrytrinder.dev-proxy-toolkit) extension |
| 37 | + |
| 38 | +### Run locally (with local AI) |
| 39 | + |
| 40 | +> [!NOTE] |
| 41 | +> |
| 42 | +> For Dev Proxy to simulate AI responses, you will need to run a local model on your machine which is accessible via an OpenAI API compatible endpoint, .e.g. Ollama. By default, this sample uses the Llama 3.2 model, change the `languageModel.model` property in the `.devproxy/simulate-ai.json` file to use a different model. Ensure that the model is running before starting Dev Proxy. |
| 43 | +
|
| 44 | +1. Start debug session in Visual Studio Code by pressing <kbd>F5</kbd> |
| 45 | +1. Wait for the process to complete |
| 46 | +1. Open the markdown file that is created in the root of the project to view the LLM usage report for the run. |
| 47 | + |
| 48 | +### Run locally (with cloud AI) |
| 49 | + |
| 50 | +1. Generate a fine-grained personal access token with `models:read` permission granted. |
| 51 | +1. Update the `apiKey` variable value in `js/env.js` with your token. |
| 52 | +1. Open **Run and Debug** panel and select **🧪 Run tests, cloud AI & local API** debug configuration |
| 53 | +1. Start the debug session by pressing <kbd>F5</kbd> |
| 54 | +1. Wait for the process to complete |
| 55 | +1. Open the markdown file that is created in the root of the project to view the LLM usage report for the run. |
| 56 | + |
| 57 | +### Run in GitHub Actions |
| 58 | + |
| 59 | +> [!NOTE] |
| 60 | +> |
| 61 | +> Enable GitHub Actions in your repository settings before running the workflow. |
| 62 | +
|
| 63 | +1. Push the **main** branch to your GitHub repository |
| 64 | +1. Open a browser and navigate to your repository |
| 65 | +1. Open the **Actions** tab in your repository |
| 66 | +1. Trigger the **Test Dev Proxy LLM Usage with Playwright** workflow manually |
| 67 | +1. Wait for the workflow to complete |
| 68 | +1. View the usage report in the job summary |
| 69 | + |
| 70 | +## Help |
| 71 | + |
| 72 | +We do not support samples, but this community is always willing to help, and we want to improve these samples. We use GitHub to track issues, which makes it easy for community members to volunteer their time and help resolve issues. |
| 73 | + |
| 74 | +You can try looking at [issues related to this sample](https://github.com/pnp/proxy-samples/issues?q=label%3A%22sample%3A%20llm-usage%22) to see if anybody else is having the same issues. |
| 75 | + |
| 76 | +If you encounter any issues using this sample, [create a new issue](https://github.com/pnp/proxy-samples/issues/new). |
| 77 | + |
| 78 | +Finally, if you have an idea for improvement, [make a suggestion](https://github.com/pnp/proxy-samples/issues/new). |
| 79 | + |
| 80 | +## Disclaimer |
| 81 | + |
| 82 | +**THIS CODE IS PROVIDED *AS IS* WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING ANY IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR NON-INFRINGEMENT.** |
| 83 | + |
| 84 | + |
0 commit comments