Merge pull request #138 from github-copilot-resources/feature/new-copilot-metrics

Feature/new copilot metrics API - replaces #125

Author: karpikpl

.github/workflows/playwright.yml

@@ -56,7 +56,7 @@ jobs:
           -e SESSION_SECRET=dummy \
           api:test "@org"
     - uses: actions/upload-artifact@v4
-      if: ${{ !cancelled() }}
+      if: always()
       with:
         name: playwright-report-docker-org
         path: test-results-docker-org/
@@ -89,7 +89,7 @@ jobs:
           -e SESSION_SECRET=dummy \
           api:test "@ent"
     - uses: actions/upload-artifact@v4
-      if: ${{ !cancelled() }}
+      if: always()
       with:
         name: playwright-report-docker-ent
         path: test-results-docker-ent/

README.md

@@ -11,49 +11,51 @@ This application displays a set of charts with various metrics related to GitHub
 
 https://github.com/github-copilot-resources/copilot-metrics-viewer/assets/3329307/bc7e2a16-cc73-43c4-887a-b50809c08533
 
-
 ## Charts
 
 ## Key Metrics
+>[!NOTE]
+> Metrics details are described in detail in [GitHub API response schema](https://docs.github.com/en/rest/copilot/copilot-metrics?apiVersion=2022-11-28#get-copilot-metrics-for-an-organization)
+
 Here are the key metrics visualized in these charts:
-1. **Acceptance Rate:** This metric represents the ratio of accepted lines to the total lines suggested by GitHub Copilot. This rate is an indicator of the relevance and usefulness of Copilot's suggestions.
 <p align="center">
-  <img width="800" alt="image" src="https://github.com/martedesco/copilot-metrics-viewer/assets/3329307/875a5f5f-5d8a-44bd-a4e9-0f663f2b2628">
+  <img width="800" alt="image" src="./images/KeyMetrics.png">
 </p>
 
-2. **Total Suggestions** This chart illustrates the total number of code suggestions made by GitHub Copilot. It offers a view of the tool's activity and its engagement with users over time.
+1. **Acceptance Rate:** This metric represents the ratio of accepted lines and suggestions to the total suggested by GitHub Copilot. This rate is an indicator of the relevance and usefulness of Copilot's suggestions. However, as with any metric, it should be used with caution as developers use Copilot in many different ways (research, confirm, verify, etc., not always "inject").
+<p align="center">
+  <img width="800" alt="image" src="./images/Acceptance_rate_bycount.png">
+</p>
 
-3. **Total Acceptances:** This visualization focuses on the total number of suggestions accepted by users.
+2. **Total Suggestions:** This chart illustrates the total number of code suggestions made by GitHub Copilot. It offers a view of the tool's activity and its engagement with users over time.
 
+3. **Total Acceptances:** This visualization focuses on the total number of suggestions accepted by users.
 <p align="center">
-  <img width="800" alt="image" src="https://github.com/martedesco/copilot-metrics-viewer/assets/3329307/b84220ae-fbdc-4503-b50b-4689362bf364">
+  <img width="800" alt="image" src="./images/Total_suggestions_count.png">
 </p>
 
 4. **Total Lines Suggested:** Showcases the total number of lines of code suggested by GitHub Copilot. This gives an idea of the volume of code generation and assistance provided.
 
-5. **Total Lines Accepted:** As the name says, the total lines of code accepted by users (full acceptances) offering insights into how much of the suggested code is actually being utilized incorporated to the codebase.
-
+5. **Total Lines Accepted:** As the name suggests, the total lines of code accepted by users (full acceptances) offering insights into how much of the suggested code is actually being utilized and incorporated into the codebase.
 <p align="center">
-  <img width="800" alt="image" src="https://github.com/martedesco/copilot-metrics-viewer/assets/3329307/788c9b33-8e63-43a5-9ab9-98d8938dd9d9">
+  <img width="800" alt="image" src="./images/Total Lines.png">
 </p>
 
 6. **Total Active Users:** Represents the number of active users engaging with GitHub Copilot. This helps in understanding the user base growth and adoption rate.
-
 <p align="center">
-  <img width="800" alt="image" src="https://github.com/martedesco/copilot-metrics-viewer/assets/3329307/bd92918f-3a11-492b-8490-877aaa768ca3">
+  <img width="800" alt="image" src="./images/Total_Active_users.png">
 </p>
 
 ## Languages Breakdown Analysis
 
-Pie charts with the top 5 languages by accepted prompts and acceptance rate are displayed at the top.
+Pie charts with the top 5 languages by accepted prompts and acceptance rate (by count/by lines) are displayed at the top.
 <p align="center">
-  <img width="800" alt="image" src="https://github.com/github-copilot-resources/copilot-metrics-viewer/assets/3329307/8ab0488a-89e6-486d-aa61-df3d178cd57c">
+  <img width="800" alt="image" src="./images/Language_breakdown.png">
 </p>
 
 The language breakdown analysis tab also displays a table showing the Accepted Prompts, Accepted Lines of Code, and Acceptance Rate (%) for each language over the past 28 days. The entries are sorted by the number of _accepted lines of code descending_.
-
 <p align="center">
-  <img width="800" alt="image" src="https://github.com/github-copilot-resources/copilot-metrics-viewer/assets/3329307/38a4ff57-4974-4f60-a154-91db17b03678">
+  <img width="800" alt="image" src="./images/Language_breakdown_list.png">
 </p>
 
 ## Copilot Chat Metrics
@@ -66,25 +68,24 @@ The language breakdown analysis tab also displays a table showing the Accepted P
 
 2. **Cumulative Number of Acceptances:** This metric shows the total number of lines of code suggested by Copilot that have been accepted by users over the past 28 days.
 
-3. **Total Turns | Total Acceptances Count:** This is a chart that displays the total number of turns and acceptances
+3. **Total Turns | Total Acceptances Count:** This is a chart that displays the total number of turns and acceptances.
 
-4. **Total Active Copilot Chat Users:** a bar chart that illustrates the total number of users who have actively interacted with Copilot over the past 28 days.
+4. **Total Active Copilot Chat Users:** A bar chart that illustrates the total number of users who have actively interacted with Copilot over the past 28 days.
 
 ## Seat Analysis
 <p align="center">
   <img width="800" alt="image" src="https://github.com/github-copilot-resources/copilot-metrics-viewer/assets/54096296/51747194-df30-4bfb-8849-54a0510fffcb">
 </p>
 
-1. **Total Assigned:** This metric represents the total number of Copilot seats assigned within current organization/enterprise.
+1. **Total Assigned:** This metric represents the total number of Copilot seats assigned within the current organization/enterprise.
 
 2. **Assigned But Never Used:** This metric shows seats that were assigned but never used within the current organization/enterprise. The assigned timestamp is also displayed in the chart.
 
-3. **No Activity in the Last 7 days:** never used seats or seats used, but with no activity in the past 7 days.
+3. **No Activity in the Last 7 Days:** Never used seats or seats used, but with no activity in the past 7 days.
 
-4. **No Activity in the last 7 days (including never used seats):** a table to display seats that have had no activity in the past 7 days, ordered by the date of last activity. Seats that were used earlier are displayed at the top.
+4. **No Activity in the Last 7 Days (including never used seats):** A table to display seats that have had no activity in the past 7 days, ordered by the date of last activity. Seats that were used earlier are displayed at the top.
 
-
-## Setup instructions
+## Setup Instructions
 
 In the `.env` file, you can configure several environment variables that control the behavior of the application.
 
@@ -100,77 +101,81 @@ For example, if you want to target the API calls to an organization, you would s
 ````
 VUE_APP_SCOPE=organization
 
-VUE_APP_GITHUB_ORG= <YOUR-ORGANIZATION>
+VUE_APP_GITHUB_ORG=<YOUR-ORGANIZATION>
 
 VUE_APP_GITHUB_ENT=
 ````
+
 #### VUE_APP_GITHUB_TEAM
 
 The `VUE_APP_GITHUB_TEAM` environment variable filters metrics for a specific GitHub team within an Enterprise or Organization account.
 ‼️ Important ‼️ When this variable is set, all displayed metrics will pertain exclusively to the specified team. To view metrics for the entire Organization or Enterprise, remove this environment variable.
 
 ````
 VUE_APP_GITHUB_TEAM=
-
 ````
 
 #### VUE_APP_MOCKED_DATA
 
 To access Copilot metrics from the last 28 days via the API and display actual data, set the following boolean environment variable to `false`:
 
-```
-  VUE_APP_MOCKED_DATA=false
-```
+````
+VUE_APP_MOCKED_DATA=false
+````
 
 #### VUE_APP_GITHUB_TOKEN
+
 Specifies the GitHub Personal Access Token utilized for API requests. Generate this token with the following scopes: _copilot_, _manage_billing:copilot_, _manage_billing:enterprise_, _read:enterprise_, _read:org_.
 
-```
-  VUE_APP_GITHUB_TOKEN=
-```
+````
+VUE_APP_GITHUB_TOKEN=
+````
 
-## Install dependencies
-```
+## Install Dependencies
+
+```bash
 npm install
 ```
 
-### Compiles and runs the application
-```
+### Compiles and Runs the Application
+
+```bash
 npm run serve
 ```
 
-### Docker build
-```
+### Docker Build
+
+```bash
 docker build -t copilot-metrics-viewer .
 ```
 
-### Docker run
-```
+### Docker Run
+
+```bash
 docker run -p 8080:80 --env-file ./.env copilot-metrics-viewer
 ```
+
 The application will be accessible at http://localhost:8080
 
 ## Running with API Proxy
 
-Project can run with an API proxy which hides GitHub tokens and is secure enough to be deployed.
-Api Proxy project is in `\api` directory. Vue app makes the calls to `/api/github` which then are proxied to `https://api.github.com` with appropriate bearer token.
-
-Proxy can authenticate user using GitHub App. In order to do that, following environment variables are required:
+The project can run with an API proxy which hides GitHub tokens and is secure enough to be deployed.
+The API Proxy project is in the `\api` directory. The Vue app makes the calls to `/api/github` which are then proxied to `https://api.github.com` with the appropriate bearer token.
 
-* `GITHUB_CLIENT_ID` - client Id of the GitHub App registered and installed in the enterprise/org with permissions listed above.
-* `GITHUB_CLIENT_SECRET` - client secret of the GitHub App
-* `SESSION_SECRET` - random string for securing session state
+The proxy can authenticate the user using a GitHub App. In order to do that, the following environment variables are required:
 
-If you want use a custom path for your `.env` file you can set the environment variable `DOTENV_CONFIG_PATH`.
+* `GITHUB_CLIENT_ID` - client ID of the GitHub App registered and installed in the enterprise/org with permissions listed above.
+* `GITHUB_CLIENT_SECRET` - client secret of the GitHub App.
+* `SESSION_SECRET` - random string for securing session state.
 
-https://github.com/user-attachments/assets/e5596067-da9c-409d-9b9f-0a688cc1f2c4
+If you want to use a custom path for your `.env` file, you can set the environment variable `DOTENV_CONFIG_PATH`.
 
-It's also possible to run with **PAT Token**, see examples below for required variables.
+It's also possible to run with a **PAT Token**, see examples below for required variables.
 
-For local development register `http://localhost:3000/callback` as GH App callback Uri.
-For deployed version use the Uri of your app.
+For local development, register `http://localhost:3000/callback` as the GitHub App callback URI.
+For the deployed version, use the URI of your app.
 
-To build and run the app with API proxy:
+To build and run the app with the API proxy:
 
 ```bash
 docker build -t copilot-metrics-viewer-with-proxy -f api.Dockerfile .
@@ -182,7 +187,7 @@ To run:
 docker run -it --rm -p 8080:3000 --env-file ./.env copilot-metrics-viewer-with-proxy
 ```
 
-Or with custom path for your `.env` file:
+Or with a custom path for your `.env` file:
 
 ```bash
 docker run -it --rm -p 8080:3000 \
@@ -191,7 +196,7 @@ docker run -it --rm -p 8080:3000 \
 copilot-metrics-viewer-with-proxy
 ```
 
-Proxy can also run with token hardcoded on the backend (which hides it from frontend calls), here's a sample:
+The proxy can also run with the token hardcoded on the backend (which hides it from frontend calls), here's a sample:
 
 ```bash
 docker run -it --rm -p 3000:3000 \
@@ -225,6 +230,6 @@ This project is licensed under the terms of the MIT open source license. Please
 
 ## Support
 
-This project is independently developed and maintained, and is not an official GitHub product. It thrives through the dedicated efforts of ([@martedesco](https://github.com/martedesco)), ([@karpikpl](https://github.com/karpikpl)) our wonderful contributors. A heartfelt thanks to all our contributors! ✨
+This project is independently developed and maintained, and is not an official GitHub product. It thrives through the dedicated efforts of ([@martedesco](https://github.com/martedesco)), ([@karpikpl](https://github.com/karpikpl)) and our wonderful contributors. A heartfelt thanks to all our contributors! ✨
 
 I aim to provide support through [GitHub Issues](https://github.com/github-copilot-resources/copilot-metrics-viewer/issues). While I strive to stay responsive, I can't guarantee immediate responses. For critical issues, please include "CRITICAL" in the title for quicker attention. 🙏🏼

api/server.mjs

@@ -17,6 +17,10 @@ const MemoryStore = MemoryStoreFactory(session);
 const limiter = RateLimit({
   windowMs: 15 * 60 * 1000, // 15 minutes
   max: 100, // max 100 requests per windowMs
+  skip: (req) => {
+    // Skip rate limiting for localhost
+    return isLocalhost(req);
+  }
 });
 
 if (DOTENV_CONFIG_PATH) {
@@ -27,8 +31,13 @@ if (DOTENV_CONFIG_PATH) {
 }
 
 const app = express();
+console.log('ENVIRONMENT: ', app.get('env'));
+
+// Disable rate limiter and secure cookies for localhost
+const isLocalhost = (req) => {
+  return req.hostname === 'localhost' || req.hostname === '127.0.0.1';
+};
 
-// apply rate limiter to all requests
 app.use(limiter);
 
 app.use(session({
@@ -38,7 +47,8 @@ app.use(session({
   store: new MemoryStore({
     checkPeriod: 86400000 // prune expired entries every 24h
   }),
-  cookie: { secure: process.env.IS_PROD ? true : false, maxAge: 86400000 }
+  // may need to use secure: false if using http during local development
+  cookie: { secure: app.get('env') === 'production', maxAge: 86400000 }
 }));
 
 // Middleware to add Authorization header
@@ -70,16 +80,24 @@ const mockResponses = (proxyServer, options) => {
     // Do not send to GitHub when mocked
     switch (req.path) {
       case "/orgs/octodemo/copilot/usage":
-        res.json(JSON.parse(readFileSync(path.join(__dirname, '../mock-data/organization_response_sample.json'), 'utf8')));
+      case "/orgs/octodemo/team/the-a-team/copilot/usage":
+        res.json(JSON.parse(readFileSync(path.join(__dirname, '../mock-data/organization_usage_response_sample.json'), 'utf8')));
+        break;
+      case "/orgs/octodemo/copilot/metrics":
+      case "/orgs/octodemo/team/the-a-team/copilot/metrics":
+        res.json(JSON.parse(readFileSync(path.join(__dirname, '../mock-data/organization_metrics_response_sample.json'), 'utf8')));
         break;
       case "/orgs/octodemo/copilot/billing/seats":
-        res.json(JSON.parse(readFileSync(path.join(__dirname, '../mock-data/organization_response_sample_seats.json'), 'utf8')));
+        res.json(JSON.parse(readFileSync(path.join(__dirname, '../mock-data/organization_seats_response_sample.json'), 'utf8')));
         break;
       case "/enterprises/octodemo/copilot/usage":
-        res.json(JSON.parse(readFileSync(path.join(__dirname, '../mock-data/enterprise_response_sample.json'), 'utf8')));
+        res.json(JSON.parse(readFileSync(path.join(__dirname, '../mock-data/enterprise_usage_response_sample.json'), 'utf8')));
+        break;
+      case "/enterprises/octodemo/copilot/metrics":
+        res.json(JSON.parse(readFileSync(path.join(__dirname, '../mock-data/enterprise_metrics_response_sample.json'), 'utf8')));
         break;
       case "/enterprises/octodemo/copilot/billing/seats":
-        res.json(JSON.parse(readFileSync(path.join(__dirname, '../mock-data/enterprise_response_sample_seats.json'), 'utf8')));
+        res.json(JSON.parse(readFileSync(path.join(__dirname, '../mock-data/enterprise_seats_response_sample.json'), 'utf8')));
         break;
       default:
         res.status(418).send('🫖Request Not Mocked');