Telemetry support for SPFx Toolkit. Closes #570 by Saurabh7019 · Pull Request #643 · pnp/vscode-viva

Saurabh7019 · 2025-11-17T15:12:17Z

🎯 Aim

This PR adds telemetry support for SPFx Toolkit to help us understand how the extension is used and improve its features.

📷 Result

App Insights

CSV report

✅ What was done

[] Proper way of handling the AI connection string
[] AI capabilities such as chat participants and LM tools
[] Usage of vscode.open actions
Created a centralized telemetry service
Extension lifecycle tracking - Activation, Deactivation
Authentication actions tracking - Login/Logout
Usage of actions from the Actions view
Usage of actions from the Tasks view
App management actions including adding app catalog, managing tenant-wide extensions, and managing SPFx apps
Support to disable telemetry through VS Code settings
Updated documentation

✅ How to test

Set up application insights and get your AI connection string
Search for key 'aiConnectionString' in package.json and replace the value
Run the PR locally and click on below actions:

🔗 Related issue

Closes: #570

Adam-it · 2025-11-17T19:56:44Z

uuuu looks interesting.... I will try to give it a check/review next week unless someone is quicker than me🤩

Adam-it · 2025-12-17T22:11:48Z

@Saurabh7019 I just did a minor and in this process the dev gets rebased/aligned with main.
Please perform the rebase on your side before we proceed

Copilot

Pull request overview

This PR adds comprehensive telemetry support to the SPFx Toolkit extension to track usage patterns and improve feature prioritization. The implementation uses the official @vscode/extension-telemetry package and respects VS Code's built-in telemetry settings.

Key changes:

Implemented a centralized TelemetryService singleton with a withTelemetry wrapper for command handlers
Added telemetry tracking for extension lifecycle (activation/deactivation), authentication, and all user-facing actions across Actions, Tasks, and Environment views
Updated documentation to explain what data is collected and how users can opt out

Reviewed changes

Copilot reviewed 13 out of 15 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
src/utils/telemetry.ts	New centralized telemetry service with command wrapper and event constants
src/extension.ts	Telemetry initialization on activation and disposal on deactivation
package.json	Added `@vscode/extension-telemetry` dependency and hardcoded AI connection string
npm-shrinkwrap.json	Lock file updates for new telemetry dependencies
src/webview/PnPWebview.ts	Wrapped samples gallery command with telemetry
src/services/executeWrappers/TerminalCommandExecuter.ts	Wrapped all terminal commands with telemetry, including npm script tracking
src/services/actions/SpfxAppCLIActions.ts	Wrapped app catalog management commands with telemetry
src/services/actions/Scaffolder.ts	Wrapped project scaffolding commands with telemetry
src/services/actions/IncreaseVersionActions.ts	Wrapped version increment command with telemetry
src/services/actions/Dependencies.ts	Wrapped dependency validation/installation commands with telemetry
src/services/actions/CopilotActions.ts	Wrapped Copilot integration command with telemetry
src/services/actions/CliActions.ts	Wrapped CLI action commands with telemetry
src/providers/AuthProvider.ts	Wrapped login/logout commands with telemetry
src/panels/CommandPanel.ts	Fixed trailing spaces in action tree item labels
docs/src/content/docs/about/telemetry.mdx	Comprehensive documentation on telemetry collection and privacy

Files not reviewed (1)

npm-shrinkwrap.json: Language not supported

src/extension.ts

package.json

npm-shrinkwrap.json

src/utils/telemetry.ts

Adam-it · 2025-12-20T21:17:08Z

@Saurabh7019 you are going to kill me I know. We need to rebase again, sorry for that 😉

Adam-it

First of all sorry for the huge hold up 🙏
Second of all, awesome work 👏 I left a few comments we could recheck before we proceed

also:

Shouldn't we also add telemetry.json so that our extension will support retrieving a dump of all telemetry using vsce cli
Did you maybe checked that the extension does not log anything to telemetry if we switch it off in VS Code?
Should we consider our own setting for it so that for example someone would like to switch of telemetry only for SPFx Toolkit but leave it on for other extensions?
Internally, lets open a discussion (best on the related topic on Basecamp) to organize a tenant for SPFx Toolkit with App Insights and how we will maintain it and what costs limits we will set

docs/src/content/docs/about/telemetry.mdx

Adam-it · 2026-01-15T13:26:13Z

package.json

 	],
 	"license": "MIT",
 	"main": "./dist/extension.js",
+	"aiConnectionString": "InstrumentationKey=137865b8-3555-4866-a2c9-7e35ead3d578;IngestionEndpoint=https://northeurope-2.in.applicationinsights.azure.com/;LiveEndpoint=https://northeurope.livediagnostics.monitor.azure.com/;ApplicationId=d253fe5b-d38c-41f5-841d-afed5bcd9557",


What is the idea we have for it. I suggest we keep this empty and add this as a new encrypted secret in the repo which would be only populated before vsce package on the pipeline on release and pre release only.
Any alternatives?

I suggest we keep this empty and add this as a new encrypted secret in the repo which would be only populated before vsce package on the pipeline on release and pre release only.

This sounds perfect to me. I may need your help to implement it though :)

Adam-it · 2026-01-15T13:26:51Z

npm-shrinkwrap.json

 			"dev": true,
 			"license": "MIT"
 		},
+		"node_modules/@nevware21/ts-async": {


synce this is not a MS maintained package lets run a one time security scan for it

np audit doesn't flag this.

Adam-it · 2026-01-15T13:28:08Z

src/providers/AuthProvider.ts

+      commands.registerCommand(Commands.login, TelemetryService.withTelemetry(Commands.login, AuthProvider.signIn))
    );
    subscriptions.push(
-      commands.registerCommand(Commands.logout, AuthProvider.logout)
+      commands.registerCommand(Commands.logout, TelemetryService.withTelemetry(Commands.logout, AuthProvider.logout))


TBH i would not track those.
What would be the benefit of knowing this?
I think what is more important is what functionalities are used?
Do users use spfx chat participant, or SPFx move app... if not lets remove those

Adam-it · 2026-01-15T13:31:49Z

src/extension.ts

+	if (telemetryService) {
+		telemetryService.sendEvent('Extension Deactivated');
+		telemetryService.dispose();
+	}


This is cool.
TBH I wonder if, and when does it really happen 🤔. if at all 😜

It's only appropriate that deactivation is also tracked when activation is.. for the lifecycle. It will happen when VS code is closed or when the extension is uninstalled.

src/extension.ts

Saurabh7019 · 2026-01-26T14:35:59Z

Shouldn't we also add telemetry.json so that our extension will support retrieving a dump of all telemetry using vsce cli - DONE

Did you maybe checked that the extension does not log anything to telemetry if we switch it off in VS Code? - YES. Also documented how to disable it.

Should we consider our own setting for it so that for example someone would like to switch of telemetry only for SPFx Toolkit but leave it on for other extensions? - I think this is fine, also the recommendation is to If isTelemetryEnabled reports false, even if a custom setting is enabled, telemetry must not be sent.

Internally, lets open a discussion (best on the related topic on Basecamp) to organize a tenant for SPFx Toolkit with App Insights and how we will maintain it and what costs limits we will set - DONE https://3.basecamp.com/5946252/buckets/41828971/messages/9506881071

Adam-it · 2026-02-15T20:16:48Z

We just had a new minor release. We should rebase this branch before we proceed

## 🎯 Aim The scaffolding form is not adapted for all versions of SPFx, and most probably it will take a lot of time until we do that. For now the aim is to adapt the scaffolding options in order to inform the user that they only support latest version of SPFx ## 📷 Result <img width="1536" height="821" alt="image" src="https://github.com/user-attachments/assets/79322abb-dc38-40e8-9039-d74db80a6994" /> ## ✅ What was done - [X] Added notification to the scaffolding form - [X] Removed gulp fast serve - [X] Updated chat participant prompts ## 🔗 Related issue Closes pnp#667 --------- Co-authored-by: Saurabh Tripathi <saurabh7019@gmail.com>

…#701 (pnp#718) ## 🎯 Aim The aim of this PR is to update all `list` related CLI commands used by the LLM to base on CSV command output mode instead of JSON which is more optimal for LLM and reducses the amount of used context window. ## 📷 Result <img width="1536" height="817" alt="image" src="https://github.com/user-attachments/assets/b94bfd13-b8f4-4f6b-936d-25736e46d1e6" /> ## ✅ What was done - [X] updates output mode to CSV ## 🔗 Related issue Closes: pnp#701 --------- Co-authored-by: Saurabh Tripathi <saurabh7019@gmail.com>

## 🎯 Aim The aim of this PR clean up all left usage of yo-rc.json file so that SPFx Toolkit is ready for newer versions of SPFx ## 📷 Result <img width="1536" height="820" alt="image" src="https://github.com/user-attachments/assets/7b884b3b-0f2c-43b0-866e-b1ec1b478e24" /> ## ✅ What was done - [X] Removes ACE checker - [X] Refactors using yo-rc.json for getting solution name in ci-cd action to get the name from package.json ## 🔗 Related issue Closes: pnp#711

…np#716) ## 🎯 Aim The aim of this PR is to refactor the old `/setup` chat participant command to a dedicated Language Model Tool that will validate and install all needed dependencies for any version of SharePoint Framework ## 📷 Result <img width="1536" height="816" alt="image" src="https://github.com/user-attachments/assets/8d1205a2-0670-4690-ba9b-6d364a655d13" /> <img width="485" height="509" alt="image" src="https://github.com/user-attachments/assets/6ec798cb-4d31-4aa3-8f21-249d34757021" /> <img width="505" height="454" alt="image" src="https://github.com/user-attachments/assets/82341651-ce54-4cc3-b780-d1e41c31452b" /> <img width="492" height="636" alt="image" src="https://github.com/user-attachments/assets/ea7152bd-ea80-4051-8bd9-cd390dc591a2" /> <img width="504" height="438" alt="image" src="https://github.com/user-attachments/assets/0a808371-4ba7-40b5-8b4d-067fb7045d2d" /> <img width="501" height="652" alt="image" src="https://github.com/user-attachments/assets/1c802146-9e2e-425c-b406-4b23a4c27501" /> ## ✅ What was done - [X] Removed the `/setup` GitHub Copilot Chat command - [X] Added SPFx setup environment Language Model Tool ## 🔗 Related issue Closes: pnp#706 --------- Co-authored-by: Saurabh Tripathi <saurabh7019@gmail.com>

## 🎯 Aim This PR fixes issues in `prepare-sample-data.ps1` script to ensure all ACE samples are included in the `sp-dev-fx-samples.json`. ## ✅ What was done - [X] Better path handling for `.yo-rc.json` - [X] Appends results to `$samples` instead of overwriting `$output` ## 🔗 Related issue Closes: pnp#723

… (pnp#691) ## 🎯 Aim This PR fixes the `nvm use` error on Windows when creating a new SPFx project via the sample gallery. On Windows, `nvm-windows` does not automatically read the `.nvmrc`, so the command fails without a version argument. The fix parses the `.nvmrc` file and sends `nvm use <version>` with the version number for `nvm` users.. for `nvs` users it checks for both `.nvmrc` and `.node-version` files and sends `nvs use` which works on all platforms. ## 📷 Result NA ## ✅ What was done - [X] modified `createTerminal` method in `TerminalCommandExecuter.ts` as agreed in the issue ## 🔗 Related issue Closes: pnp#519

pre-release 4.16.1

…#726) ## 🎯 Aim The aim is to totally remove all related code regarding the /info SPFx Chat Participant command and, in the documentation, replace it with guidance recommending the use of the CLI for the Microsoft 365 MCP server instead. ## 📷 Result <img width="1380" height="902" alt="image" src="https://github.com/user-attachments/assets/41335670-ee0c-41a2-99fd-bc0dad5f512c" /> ## ✅ What was done - [X] Removed all code related to the /info SPFx Chat Participant command - [X] Cleaned up any references, registrations, and configuration linked to the /info command - [X] Updated documentation to recommend using the CLI for Microsoft 365 MCP server for tenant info and management ## 🔗 Related issue Closes: pnp#708 --------- Co-authored-by: Adam-it <adam.wojcik.it@gmail.com>

…np#697) ## 🎯 Aim The aim is to add a new action that will validate user local setup for the currently open SPFx project. For example someone opens some old SPFx sample that uses SPFx 1.16 and would like to recheck if the current version of node and local dependencies are valid for that project. The aciton uses `spfx doctor` command to get the report and similar like the setup action will install the correct version of node and global dependencies if needed based on the compatibility matrix ## 📷 Result <img width="1536" height="815" alt="image" src="https://github.com/user-attachments/assets/22d601c8-8b57-403b-84e5-33bb3cb4aab2" /> <img width="1536" height="810" alt="image" src="https://github.com/user-attachments/assets/fcd31e92-ee89-4cc2-a4e9-abdd631f22c7" /> ## ✅ What was done - [X] Added new action - [ ] Updated docs 👉// TODO ## 🔗 Related issue Closes: pnp#518 --------- Co-authored-by: Saurabh Tripathi <saurabh7019@gmail.com>

## 🎯 Aim I noticed we are missing some docs about recently added langauge model tools and this PR is suppose to fill this gap

## 🎯 Aim I know the related issue was still not agreed but this one is quite easy and hits on my nerve .😁 What I usually do is move the SPFx Toolkit Task Pane with heft and npm actions to the Explorer view so the tasks sit right along my files I modified. The problem is that this panel does not show up until the extension gets active so this happens only when we go to the SPFx Toolkit view. So this means every time I opened a SPFx project I had to go to the SPFx Toolkit view and then go back to the explorer view just to see the task panel. This is quite frustrating. Folks might have customized the look and feel of VS Code, like moving the login and management views to some other part of VS Code, In this case they would also need to go to the view to make them active. That Is why I propose to just check if a project has a config\package-solution.json file, and if so, we may assume it is a SPFx project and make the extension active. If it will still not be an SPFx project, this will be caught by the IsSpfx helper method in CommandPanels and will just load the SPFx Toolkit with the functionality when you are not in a SPFx project ## 📷 Result <img width="684" height="788" alt="image" src="https://github.com/user-attachments/assets/bc8f7617-93b4-496d-803c-3a970457c254" /> ## 🔗 Related issue Closes: pnp#733

## 🎯 Aim The aim is to fix the way we handle MSAL refresh token. Currently, in the extension, after we sign in, we are more or less logged in 1 hour and after that the access token expires, and we need to perform sign in again. I noticed it was due to a simple bugi and the main origin and fix up was done and already published in CLI SPFx Toolkit npm package. No the token should allow to retain sign in from 4-90 days (depending on tenant settings) In order to test this PR is best 1. to uninstall SPFx Toolkit if you have it. 2. Then use `vsce package` to build a local vscix package and install SPFx Toolkit from your local package. 3. Next perform sign in 4. Wait for 1-2 hours and restart VS Code. You should still be signed in ## 🔗 Related issue Closes: pnp#500

Adam-it · 2026-03-30T23:45:22Z

we had a new minor release again. Maybe lets keep this PR as draft until I setup a tenant for it and make the changes.
When we find time lets rebase against main

Saurabh7019 mentioned this pull request Nov 20, 2025

💡 [Feature]: Telemetry support for vscode.open actions and AI capabilities #646

Open

Adam-it self-assigned this Nov 22, 2025

Adam-it removed their assignment Dec 6, 2025

Adam-it force-pushed the dev branch 2 times, most recently from 2cc4605 to 4c4858b Compare December 8, 2025 22:51

Adam-it force-pushed the dev branch 2 times, most recently from 4ca0a6a to 8716fdf Compare December 17, 2025 22:10

Adam-it marked this pull request as draft December 17, 2025 22:11

Saurabh7019 force-pushed the issue-570 branch from 4510320 to c333458 Compare December 19, 2025 12:23

Saurabh7019 marked this pull request as ready for review December 19, 2025 12:27

Copilot AI review requested due to automatic review settings December 19, 2025 12:27

Copilot started reviewing on behalf of Saurabh7019 December 19, 2025 12:28 View session

Copilot AI reviewed Dec 19, 2025

View reviewed changes

Saurabh7019 force-pushed the issue-570 branch from 2f61fa0 to 697927c Compare December 19, 2025 13:38

Adam-it marked this pull request as draft December 20, 2025 21:17

Saurabh7019 force-pushed the issue-570 branch from 697927c to f17759b Compare December 21, 2025 19:17

Saurabh7019 marked this pull request as ready for review December 21, 2025 19:18

Adam-it self-assigned this Jan 15, 2026

Adam-it force-pushed the issue-570 branch from f17759b to 6a2e279 Compare January 15, 2026 12:45

Adam-it requested changes Jan 15, 2026

View reviewed changes

Adam-it marked this pull request as draft January 15, 2026 13:40

Saurabh7019 force-pushed the issue-570 branch from 6a2e279 to a1f018b Compare January 26, 2026 14:11

Saurabh7019 marked this pull request as ready for review January 26, 2026 14:36

Adam-it force-pushed the dev branch 2 times, most recently from 2c84a0b to fd39e59 Compare February 15, 2026 20:15

Adam-it marked this pull request as draft February 15, 2026 20:16

Adam-it and others added 20 commits February 16, 2026 13:57

pre-release 4.17.1

347a289

pre-release 4.17.2

19a5434

pre-release 4.17.3 (pnp#730)

dfe357e

pre-release 4.16.1

pre-release 4.17.4

3467bce

pre-release 4.17.5

5ed920f

Updates docs for language model tools (pnp#739)

a51a1f1

## 🎯 Aim I noticed we are missing some docs about recently added langauge model tools and this PR is suppose to fill this gap

pre-release 4.17.6

7204f71

pre-release 4.17.7

761a797

Telemetry support for SPFx Toolkit. Closes pnp#570

320fe4d

address code review feedback

75cbdff

Saurabh7019 force-pushed the issue-570 branch from a1f018b to 75cbdff Compare March 26, 2026 14:48

add telemetry for validateEnvironmentForProject

e2f3020

Saurabh7019 marked this pull request as ready for review March 26, 2026 19:51

Adam-it force-pushed the dev branch from a06fbb6 to 6fc9093 Compare March 30, 2026 23:43

Adam-it marked this pull request as draft March 30, 2026 23:44

Conversation

Saurabh7019 commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🎯 Aim

📷 Result

✅ What was done

✅ How to test

🔗 Related issue

Uh oh!

Adam-it commented Nov 17, 2025

Uh oh!

Adam-it commented Dec 17, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Adam-it commented Dec 20, 2025

Uh oh!

Adam-it left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Adam-it Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Saurabh7019 Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

Adam-it Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Saurabh7019 Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Adam-it Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Adam-it Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Saurabh7019 Jan 26, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Saurabh7019 commented Jan 26, 2026

Uh oh!

Adam-it commented Feb 15, 2026

Uh oh!

Adam-it commented Mar 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Saurabh7019 commented Nov 17, 2025 •

edited

Loading

Saurabh7019 Jan 26, 2026 •

edited

Loading