Skip to content

Commit 2811e51

Browse files
jasonmoodiejasonmoodie
authored
Merge pull request #5 from github-copilot-resources/release/v2.0
Upgrade API to Metrics Version and Add KPI Dashboard
2 parents a2c4668 + 4c773f4 commit 2811e51

18 files changed

+11459
-11119
lines changed

‎README.md

Lines changed: 125 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -1,80 +1,146 @@
11
# GitHub Copilot Metrics Viewer for Power BI
22

3-
With the release of the [GitHub Copilot Metrics API](https://github.blog/changelog/2024-04-23-github-copilot-metrics-api-now-available-in-public-beta/) many teams are looking to leverage this data to help monitor usage against their KPIs. For some, the Copilot Metrics Viewer ([github-copilot-resources/copilot-metrics-viewer](https://github.com/github-copilot-resources/copilot-metrics-viewer)) might be a great option.
3+
With the release of the latest [GitHub Copilot Metrics API](https://github.com/orgs/community/discussions/141071) many teams are looking to leverage this data to help monitor usage against their KPIs. For some, the Copilot Metrics Viewer ([github-copilot-resources/copilot-metrics-viewer](https://github.com/github-copilot-resources/copilot-metrics-viewer)) might be a great option.
44

55
However, many organizations that we work with already have established Power BI teams. If your organization is **already using Power BI, please read on!**
66

7-
Located in the `./samples` directory you'll find sample JSON and PBIX files used to create the dashboard below.
8-
9-
![Image of a Power BI dashboard with GitHuub Copilot Metrics API data displayed.](https://github.com/jasonmoodie/pbi-4-ghcopilot/blob/main/assets/Sample_PBI.png)
7+
## Table of Contents
8+
- [New Metrics API Features](#new-features)
9+
- [Metrics Dashboard Setup](#metrics-dashboard-setup)
10+
- [Connect to Metrics API](#connect-to-metrics-api)
11+
- [Connect to local JSON data source](#connect-to-local-json-data-source)
12+
- [KPI - Savings Dashboard](#kpi---savings-dashboard)
13+
- [Publishing](#publishing)
14+
- [Maintainers](#maintainers)
15+
- [Support](#support)
16+
17+
## New Metrics API Features
18+
- New metrics for Pull Request summaries and Copilot Chat on GitHub.com.
19+
- Clarity on code completions and Copilot Chat usage in IDEs.
20+
- User engagement summaries updated daily.
21+
- Custom model slicing support, coming soon!
22+
- Data can be aggregated by enterprise, organization or team.
1023

11-
## Setup
12-
13-
### Test: Modify the local JSON data source
14-
> Note: This example provided a proof of concept for loading metrics data and requires an exported JSON file. If you have access to the REST API you can configure the **Source** accordingly.
15-
16-
1. Download and open the sample `GitHub Copilot - Telemetry Sample (DM).pbix` file.
17-
2. The file contains three data sources on the right hand side.
24+
Located in the `./samples` directory you'll find sample JSON and PBIX files used to create the dashboard below.
25+
> Note: The legacy GitHub Copilot Usage API is still available and can be used with the `GitHub Copilot - Telemetry Sample (Usage).pbix` file. For instructions, see [Usage_API.md](USAGE_API.md).
1826
19-
| Name | Description |
20-
| :-------------------- | :----------------------------------------------------- |
21-
| GH Copilot - Details | Detailed breakdown of metrics data by language and editor. |
22-
| GH Copilot - Summary | Daily summary of metrics data. |
23-
| Last Refresh | Used to display the data time of data refresh on the top-right corner of the dashboard. |
27+
![Image of the Power BI dashboard with sample GitHub Copilot Metrics API data displayed.](./assets/Sample_Metrics_PBI.png)
2428

25-
3. Open the **Power Query Editor** by right clicking the `GH Copilot - Details` and selecting **Edit query**.
26-
4. Modify the **Source** step by clicking the settings icon, selecting your JSON file and clicking **OK**.
27-
![Image of a data source selector in Power Query Editor.](https://github.com/jasonmoodie/pbi-4-ghcopilot/blob/main/assets/Modify_JSON_source.png)
28-
6. Repeat steps 3 and 4 for the `GH Copilot - Summary` **Source**.
29-
7. Click **Close and Apply** in the top-left of the **Power Query Editor**.
30-
8. On the **Report View** page click **Refresh** to load the new data into your dashboard.
31-
9. **Happy Customizing!**
29+
## Metrics Dashboard Setup
3230

3331
### Connect to Metrics API
34-
> Notes: The REST API provides metrics for the previous 28 days and is refreshed daily with data from the previous day. This is currently in beta, so please ensure you are using the latest version of the [REST API](https://docs.github.com/en/rest/copilot/copilot-usage).
32+
> Notes: The REST API provides metrics for the previous 28 days and is refreshed daily with data from the previous day. Please ensure you are using the latest version of the [REST API](https://docs.github.com/en/enterprise-cloud@latest/rest/copilot/copilot-metrics).
3533
3634
In order to connect we'll need to generate a token and link to your metrics data:
37-
1. Download and open the sample `GitHub Copilot - Telemetry Sample (DM).pbix` file.
38-
2. Determine if you'll be using the `Enterprise` or `Organization` URL.
35+
1. Download and open the sample `GitHub Copilot - Telemetry Sample (Metrics with KPI).pbix` file.
36+
2. Determine if you'll be using the `Enterprise`, `Organization` or `Team` URL.
3937
3. Follow the instructions below to generate a token with permissions to access the API:
40-
[REST API endpoints for GitHub Copilot usage metrics - GitHub Docs](https://docs.github.com/en/rest/copilot/copilot-usage)
38+
[REST API endpoints for Copilot metrics - GitHub Enterprise Cloud Docs](https://docs.github.com/en/enterprise-cloud@latest/rest/copilot/copilot-metrics)
4139
>**IMPORTANT: Do not share this token and ensure you follow you organizations security policies.**
42-
4. Open the **Power Query Editor** by right clicking the `GH Copilot - Details` and selecting **Edit query**.
43-
5. Select **Advanced editor**.
44-
![Image of Power Query Advanced Editor.](https://github.com/jasonmoodie/pbi-4-ghcopilot/blob/main/assets/Advanced_editor.png)
45-
6. Replace the first 2 lines with following, ensure to replace <YOUR-TOKEN> and <ORG> with the values from step 1 and 2.
40+
4. The file contains the following data sources, descriptions are inlcuded below.
41+
42+
| Name | Description |
43+
| :--------------------------------- | :------------------------------------------------------------ |
44+
| config | Configuration used to display date of refresh and KPI dashboard |
45+
| source | Base source used to connect to API or local JSON files. |
46+
| GH Copilot - dotcom chat | Detailed metrics of code completions using chat on the web. |
47+
| GH Copilot - ide chat | Detailed metrics of code completions using chat within the IDE. |
48+
| GH Copilot - ide code completions editors | Detailed metrics of code completions within the IDE. |
49+
| GH Copilot - ide code completions languages | Detailed metrics of engaged users by language within the IDE. |
50+
| GH Copilot - pull requests | Detailed metrics of pull requests. |
51+
| GH Copilot - summary | Daily summary of active and engaged users. |
52+
5. Open the **Power Query Editor** by clicking **Transform data** in the top-menu and selecting **Tranform data**.
53+
6. Click on **source** in the left-menu.
54+
7. Select **Advanced editor**.
55+
![Image of Power Query Advanced Editor.](./assets/Advanced_editor.png)
56+
8. Replace the first 2 lines with following, ensure to replace `<YOUR-TOKEN>` along with the relevant `<ENTERPRISE>`, `<ORG>` and `<TEAM_SLUG>` with the values from step 2.
4657

47-
**Organization**
48-
```powerquery
49-
let
50-
// Replace <YOUR-TOKEN> and <ORG> with your actual token and org name.
51-
url = "https://api.github.com/orgs/<ORG>/copilot/usage",
52-
headers = [
53-
#"Accept" = "application/vnd.github+json",
54-
#"Authorization" = "Bearer <YOUR-TOKEN>",
55-
#"X-GitHub-Api-Version" = "2022-11-28"
56-
],
57-
Source = Json.Document(Web.Contents(url, [Headers=headers])),
58-
```
59-
**Enterprise**
60-
```powerquery
61-
let
62-
// Replace <YOUR-TOKEN> and <ENTERPRISE> with your actual token and enterprise name.
63-
url = "https://api.github.com/enterprises/<ENTERPRISE>/copilot/usage",
64-
headers = [
65-
#"Accept" = "application/vnd.github+json",
66-
#"Authorization" = "Bearer <YOUR-TOKEN>",
67-
#"X-GitHub-Api-Version" = "2022-11-28"
68-
],
69-
Source = Json.Document(Web.Contents(url, [Headers=headers])),
70-
```
71-
7. Your Power Query will look something like this:
72-
![Image of Power Query Advanced Editor.](https://github.com/jasonmoodie/pbi-4-ghcopilot/blob/main/assets/Advanced_editor_query.png)
73-
8. Click **OK** to close the editor and select `Anonymous` authentication if prompted.
74-
9. Repeat steps 4 and 8 for the `GH Copilot - Summary` **Source**.
58+
**Enterprise**
59+
```powerquery
60+
let
61+
// Replace <YOUR-TOKEN> and <ENTERPRISE> with your actual token and enterprise name.
62+
url = "https://api.github.com/enterprises/<ENTERPRISE>/copilot/metrics",
63+
headers = [
64+
#"Accept" = "application/vnd.github+json",
65+
#"Authorization" = "Bearer <YOUR-TOKEN>",
66+
#"X-GitHub-Api-Version" = "2022-11-28"
67+
],
68+
Source = Json.Document(Web.Contents(url, [Headers=headers])),
69+
```
70+
71+
**Enterprise Team**
72+
```powerquery
73+
let
74+
// Replace <YOUR-TOKEN>, <ENTERPRISE> and <TEAM_SLUG> with your actual token and enterprise name.
75+
url = "https://api.github.com/enterprises/<ENTERPRISE>/team/<TEAM_SLUG>/copilot/metrics",
76+
headers = [
77+
#"Accept" = "application/vnd.github+json",
78+
#"Authorization" = "Bearer <YOUR-TOKEN>",
79+
#"X-GitHub-Api-Version" = "2022-11-28"
80+
],
81+
Source = Json.Document(Web.Contents(url, [Headers=headers])),
82+
```
83+
84+
**Organization**
85+
```powerquery
86+
let
87+
// Replace <YOUR-TOKEN> and <ORG> with your actual token and org name.
88+
url = "https://api.github.com/orgs/<ORG>/copilot/metrics",
89+
headers = [
90+
#"Accept" = "application/vnd.github+json",
91+
#"Authorization" = "Bearer <YOUR-TOKEN>",
92+
#"X-GitHub-Api-Version" = "2022-11-28"
93+
],
94+
Source = Json.Document(Web.Contents(url, [Headers=headers])),
95+
```
96+
97+
**Team**
98+
```powerquery
99+
let
100+
// Replace <YOUR-TOKEN>, <ORG> and <TEAM_SLUG> with your actual token and enterprise name.
101+
url = "https://api.github.com/orgs/<ORG>/team/<TEAM_SLUG>/copilot/metrics",
102+
headers = [
103+
#"Accept" = "application/vnd.github+json",
104+
#"Authorization" = "Bearer <YOUR-TOKEN>",
105+
#"X-GitHub-Api-Version" = "2022-11-28"
106+
],
107+
Source = Json.Document(Web.Contents(url, [Headers=headers])),
108+
```
109+
8. Your Power Query will look something like this:
110+
![Image of Power Query Advanced Editor.](./assets/Advanced_editor_metrics_query.png)
111+
9. Click **OK** to close the editor and select `Anonymous` authentication if prompted.
75112
10. Click **Close and Apply** in the top-left of the **Power Query Editor**.
76113
11. On the **Report View** page click **Refresh** to load the new data into your dashboard.
77114
115+
### Connect to local JSON data source
116+
> Note: This example provided a proof of concept for loading metrics data and requires an exported JSON file. If you have access to the REST API you can configure the **Source** accordingly.
117+
118+
1. Download and open the sample `GitHub Copilot - Telemetry Sample (Metrics with KPIs).pbix` file.
119+
2. Open the **Power Query Editor** by clicking **Transform data** in the top-menu and selecting **Tranform data**.
120+
3. Click on **source** query in the left-menu
121+
4. In the right-menu under **APPLIED STEPS** click the gear (settings) icon, selecting your JSON file and clicking **OK**.
122+
![Image of a data source selector in Power Query Editor.](./assets/Modify_JSON_source.png)
123+
5. Click **Close and Apply** in the top-left of the **Power Query Editor**.
124+
6. On the **Report View** page click **Refresh** to load the new data into your dashboard.
125+
7. **Happy Customizing!**
126+
127+
## KPI - Savings Dashboard
128+
A new KPI tab has been added to the dashboard to help you estimate savings. The KPI tab is configured to display the potential time and case savings. You can configure the KPI tab to display these details by modifying the following fields in the `config` data sourcefrom the **Tabel view**:
129+
130+
| Name | Description |
131+
| :------------------------- | :----------------------------------------------- |
132+
| total_devs | Total number of developers at your organization. |
133+
| avg_hourly_salary | Average hourly salary of developers. |
134+
| annual_work_weeks | Total number of work weeks in a year. |
135+
| average_weekly_hour_savings| Average number of hours developers saved per week. The default is 3.5 hours and assumed a 10% time saving, but this can be updated based on customer survey data or other measurements. |
136+
137+
These values can be modified in the `config` data source below:
138+
![Image of the KPI config table in the Power BI.](./assets/KPI_config.png)
139+
140+
Once configured, the KPI dashboard will display this potential savings against current usage pulled from the Metrics API:
141+
![Image of the KPI tab in the Power BI.](./assets/Sample_KPI_PBI.png)
142+
143+
78144
## Publishing
79145
If you need help deploying or publishing this script, please see: [Publish README](/publish/README.md)
80146

‎USAGE_API.md

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# GitHub Copilot Metrics Viewer for Power BI
2+
3+
> **Note:** The legacy GitHub Copilot Usage API is deprecated. Please refer to the new [GitHub Copilot Metrics API](https://github.com/orgs/community/discussions/141071).
4+
5+
Located in the `./samples` directory you'll find sample JSON and PBIX files used to create the dashboard below.
6+
7+
![Image of a Power BI dashboard with GitHuub Copilot Metrics API data displayed.](./assets/Sample_PBI.png)
8+
9+
## Setup
10+
### Connect to Metrics API
11+
> Notes: The REST API provides metrics for the previous 28 days and is refreshed daily with data from the previous day. This is currently in beta, so please ensure you are using the latest version of the [REST API](https://docs.github.com/en/rest/copilot/copilot-usage).
12+
13+
In order to connect we'll need to generate a token and link to your metrics data:
14+
1. Download and open the sample `GitHub Copilot - Telemetry Sample (DM).pbix` file.
15+
2. Determine if you'll be using the `Enterprise` or `Organization` URL.
16+
3. Follow the instructions below to generate a token with permissions to access the API:
17+
[REST API endpoints for GitHub Copilot usage metrics - GitHub Docs](https://docs.github.com/en/rest/copilot/copilot-usage)
18+
>**IMPORTANT: Do not share this token and ensure you follow you organizations security policies.**
19+
4. Open the **Power Query Editor** by right clicking the `GH Copilot - Details` and selecting **Edit query**.
20+
5. Select **Advanced editor**.
21+
![Image of Power Query Advanced Editor.](./assets/Advanced_editor.png)
22+
6. Replace the first 2 lines with following, ensure to replace <YOUR-TOKEN> and <ORG> with the values from step 1 and 2.
23+
24+
**Organization**
25+
```powerquery
26+
let
27+
// Replace <YOUR-TOKEN> and <ORG> with your actual token and org name.
28+
url = "https://api.github.com/orgs/<ORG>/copilot/usage",
29+
headers = [
30+
#"Accept" = "application/vnd.github+json",
31+
#"Authorization" = "Bearer <YOUR-TOKEN>",
32+
#"X-GitHub-Api-Version" = "2022-11-28"
33+
],
34+
Source = Json.Document(Web.Contents(url, [Headers=headers])),
35+
```
36+
**Enterprise**
37+
```powerquery
38+
let
39+
// Replace <YOUR-TOKEN> and <ENTERPRISE> with your actual token and enterprise name.
40+
url = "https://api.github.com/enterprises/<ENTERPRISE>/copilot/usage",
41+
headers = [
42+
#"Accept" = "application/vnd.github+json",
43+
#"Authorization" = "Bearer <YOUR-TOKEN>",
44+
#"X-GitHub-Api-Version" = "2022-11-28"
45+
],
46+
Source = Json.Document(Web.Contents(url, [Headers=headers])),
47+
```
48+
7. Your Power Query will look something like this:
49+
![Image of Power Query Advanced Editor.](./assets/Advanced_editor_query.png)
50+
8. Click **OK** to close the editor and select `Anonymous` authentication if prompted.
51+
9. Repeat steps 4 and 8 for the `GH Copilot - Summary` **Source**.
52+
10. Click **Close and Apply** in the top-left of the **Power Query Editor**.
53+
11. On the **Report View** page click **Refresh** to load the new data into your dashboard.
54+
55+
### Modify to use a local JSON file
56+
> Note: This example provided a proof of concept for loading metrics data and requires an exported JSON file. If you have access to the REST API you can configure the **Source** accordingly.
57+
58+
1. Download and open the sample `GitHub Copilot - Telemetry Sample (Usage).pbix` file.
59+
2. The file contains three data sources on the right hand side.
60+
61+
| Name | Description |
62+
| :-------------------- | :----------------------------------------------------- |
63+
| GH Copilot - Details | Detailed breakdown of metrics data by language and editor. |
64+
| GH Copilot - Summary | Daily summary of metrics data. |
65+
| Last Refresh | Used to display the data time of data refresh on the top-right corner of the dashboard. |
66+
67+
3. Open the **Power Query Editor** by right clicking the `GH Copilot - Details` and selecting **Edit query**.
68+
4. Modify the **Source** step by clicking the settings icon, selecting your JSON file and clicking **OK**.
69+
![Image of a data source selector in Power Query Editor.](./assets/Modify_JSON_source.png)
70+
6. Repeat steps 3 and 4 for the `GH Copilot - Summary` **Source**.
71+
7. Click **Close and Apply** in the top-left of the **Power Query Editor**.
72+
8. On the **Report View** page click **Refresh** to load the new data into your dashboard.
73+
9. **Happy Customizing!**
74+
75+
## Publishing
76+
If you need help deploying or publishing this script, please see: [Publish README](/publish/README.md)
77+
78+
## Maintainers
79+
80+
@jasonmoodie, @Eldrick19
81+
82+
## Support
83+
84+
These are just files for you to download and use as you see fit. If you have questions about how to use them, please reach out to the maintainers, but we cannot guarantee a response with SLAs.

‎assets/Advanced_editor_metrics_query.png

35.9 KB
Loading

‎assets/KPI_config.png

11.8 KB
Loading

‎assets/Sample_KPI_PBI.png

92.8 KB
Loading

‎assets/Sample_Metrics_PBI.png

158 KB
Loading

‎queries/enterprise_metrics.pq

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
let
2+
// Replace <YOUR-TOKEN> and <ENTERPRISE> with your actual token and enterprise name.
3+
url = "https://api.github.com/enterprises/<ENTERPRISE>/copilot/metrics",
4+
headers = [
5+
#"Accept" = "application/vnd.github+json",
6+
#"Authorization" = "Bearer <YOUR-TOKEN>",
7+
#"X-GitHub-Api-Version" = "2022-11-28"
8+
],
9+
Source = Json.Document(Web.Contents(url, [Headers=headers])),

‎queries/enterprise_team_metrics.pq

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
let
2+
// Replace <YOUR-TOKEN>, <ENTERPRISE> and <TEAM_SLUG> with your actual token and enterprise name.
3+
url = "https://api.github.com/enterprises/<ENTERPRISE>/team/<TEAM_SLUG>/copilot/metrics",
4+
headers = [
5+
#"Accept" = "application/vnd.github+json",
6+
#"Authorization" = "Bearer <YOUR-TOKEN>",
7+
#"X-GitHub-Api-Version" = "2022-11-28"
8+
],
9+
Source = Json.Document(Web.Contents(url, [Headers=headers])),

‎queries/enterprise.pq renamed to ‎queries/enterprise_usage.pq

File renamed without changes.

‎queries/organization_metrics.pq

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
let
2+
// Replace <YOUR-TOKEN> and <ORG> with your actual token and org name.
3+
url = "https://api.github.com/orgs/<ORG>/copilot/metrics",
4+
headers = [
5+
#"Accept" = "application/vnd.github+json",
6+
#"Authorization" = "Bearer <YOUR-TOKEN>",
7+
#"X-GitHub-Api-Version" = "2022-11-28"
8+
],
9+
Source = Json.Document(Web.Contents(url, [Headers=headers])),

0 commit comments

Comments
 (0)