Skip to content

feat: VS Code extension MVP — ground-up rebuild with daemon architecture#559

Open
joshsmithxrm wants to merge 417 commits intomainfrom
feature/vscode-extension-mvp
Open

feat: VS Code extension MVP — ground-up rebuild with daemon architecture#559
joshsmithxrm wants to merge 417 commits intomainfrom
feature/vscode-extension-mvp

Conversation

@joshsmithxrm
Copy link
Owner

Summary

Complete rebuild of the VS Code extension from the ground up, replacing the self-contained approach with a thin UI layer that delegates all operations to the ppds serve daemon via JSON-RPC.

  • Profile management — create, delete, rename, select profiles from the sidebar
  • Environment discovery — browse and select Dataverse environments with status bar indicator
  • Solutions browser — explore solutions with expandable component groups
  • Dataverse notebooks (.ppdsnb) — SSMS-like query experience with SQL and FetchXML, IntelliSense, virtual scrolling, export to CSV/JSON
  • Data Explorer — webview panel for quick ad-hoc queries
  • Query history — automatic persistence of executed queries
  • Pre-release channel — version 0.5.0 using odd/even minor convention (odd = pre-release, even = stable)

Architecture

VS Code Extension (TypeScript) → JSON-RPC over stdio → ppds serve (daemon) → Application Services

All business logic stays in Application Services. The extension is UI-only.

Not yet ported from legacy (tracked for 0.6.0 stable)

  • Plugin Trace viewer
  • Connection References viewer
  • Environment Variables viewer
  • Metadata Browser
  • Web Resources viewer
  • Import Job viewer

Daemon-side changes

  • New RPC endpoints: auth/create, auth/delete, auth/rename, env/who, env/configure, query/complete, query/explain, solutions/list, solutions/components, schema/entities, schema/attributes
  • FetchXML completion engine
  • Query history persistence
  • TDS read replica routing support

Test plan

  • Install .vsix locally and verify extension activates
  • Create/select/rename/delete authentication profiles
  • Select environment, verify status bar updates
  • Create .ppdsnb notebook, execute SQL query, verify results render
  • Toggle cell language between SQL and FetchXML
  • Test IntelliSense for SQL and FetchXML
  • Export results to CSV and JSON
  • Open Data Explorer panel, run ad-hoc query
  • Browse solutions tree view
  • Verify 85 unit tests pass (npm test in extension/)
  • Verify vsce package --pre-release produces clean .vsix

🤖 Generated with Claude Code

@gemini-code-assist
Copy link

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request marks a significant architectural shift for the VS Code extension, moving to a daemon-based model for all core operations. This change enables a more robust and feature-rich development experience within VS Code, offering advanced querying capabilities, comprehensive environment and profile management, and a foundation for future tooling. The focus is on delivering a powerful, integrated environment for Power Platform developers.

Highlights

  • Ground-Up Rebuild with Daemon Architecture: The VS Code extension has been completely rebuilt to utilize a thin UI layer that delegates all operations to the ppds serve daemon via JSON-RPC. This shifts business logic to application services, making the extension UI-agnostic.
  • Core Feature Parity with TUI: Key functionalities from the Text User Interface (TUI) have been ported, including comprehensive profile management (create, delete, rename, select), environment discovery and selection, and a solutions browser with expandable component groups.
  • Dataverse Notebooks (.ppdsnb): Introduced a new SSMS-like query experience with Dataverse notebooks, supporting SQL and FetchXML queries, IntelliSense, virtual scrolling for large results, and export capabilities to CSV/JSON. Query history is automatically persisted.
  • IntelliSense for SQL and FetchXML: Added daemon-powered IntelliSense for both SQL and FetchXML, providing autocomplete for tables, columns, and FetchXML elements directly within notebook cells and query panels.
  • Data Explorer Webview Panel: A new webview panel, 'Data Explorer', offers a quick ad-hoc query experience, complementing the persistent workflows of Dataverse notebooks. It supports multi-tab instances and a TDS Read Replica toggle.
  • Enhanced Daemon RPC Endpoints: Numerous new RPC endpoints have been added to the ppds serve daemon to support the new extension features, including auth/create, auth/delete, auth/rename, env/who, env/configure, query/complete, query/explain, solutions/list, solutions/components, schema/entities, schema/attributes, and query history persistence.
  • Improved Testing Infrastructure: The build and test systems have been updated, transitioning from Webpack to esbuild for bundling and from Jest to Vitest for unit testing. Playwright E2E tests have been introduced to ensure end-to-end functionality.
Changelog
  • .gitignore
    • Updated ignored files to include Node.js modules and VS Code extension test artifacts.
  • docs/plans/2026-02-08-vscode-extension-mvp.md
    • Updated the architecture description to reflect the thin UI layer and daemon communication.
    • Changed the tech stack from Jest to Vitest for testing.
    • Reordered development phases, promoting notebooks as the primary experience and demoting the query panel.
    • Added new tasks for IntelliSense daemon endpoints, VS Code CompletionItemProvider, and auto-saving query history.
  • extension/package.json
    • Updated the extension version to 0.5.0 and revised keywords for better discoverability.
    • Modified activation events to trigger on view and notebook activation.
    • Added new VS Code contributions for views, commands, menus, notebooks, languages (FetchXML), and configuration settings.
    • Updated build scripts to use esbuild and test scripts to use Vitest and Playwright.
  • extension/src/daemonClient.ts
    • Implemented new RPC methods for authentication, environment management, query execution, history, export, explain, profile management, solutions, and schema.
    • Added a startup handshake and race condition fix to ensure stable daemon connection.
    • Introduced a _disposed flag and connectingPromise for robust connection management and disposal.
  • extension/src/extension.ts
    • Refactored the activation logic to initialize and register all new components, including tree views, notebook serializer and controller, environment status bar, and completion providers.
    • Integrated auto-start logic for the daemon based on user configuration.
  • src/PPDS.Cli/Commands/Serve/Handlers/RpcMethodHandler.cs
    • Implemented new RPC methods for environment configuration (env/config/get, env/config/set), profile CRUD operations (profiles/create, profiles/delete, profiles/rename), schema browsing (schema/entities, schema/attributes), and query IntelliSense (query/complete).
    • Added support for TDS Read Replica routing in query/sql.
    • Introduced auto-saving of query history during query/sql and query/fetch execution.
    • Refactored WithActiveProfileAsync to use the connection pool manager for improved performance and consistency.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This is an impressive and substantial pull request that successfully rebuilds the VS Code extension from the ground up with a new daemon-based architecture. The scope of work is extensive, introducing major features like Dataverse notebooks, a data explorer, and solution browsing, while also making significant architectural improvements for robustness and security. The code quality is high, and the detailed planning documents are a great addition. My review focuses on a few key areas, including resource management in asynchronous operations, the end-to-end testing setup, and data handling logic to ensure the new foundation is as solid as possible.

Note: Security Review did not run due to the size of the PR.

@codecov-commenter
Copy link

@joshsmithxrm joshsmithxrm force-pushed the feature/vscode-extension-mvp branch from 68e42e8 to 09eb073 Compare March 6, 2026 18:39
joshsmithxrm and others added 23 commits March 16, 2026 03:15
- SqlQueryService now throws DmlConfirmationRequired (not DmlBlocked)
  when safetyResult.RequiresConfirmation is true, eliminating the
  fragile message-sniff (ex.Message.Contains("--confirm")) in the handler
- RpcMethodHandler catches DmlConfirmationRequired and DmlBlocked as
  separate clauses with their correct error data shapes
- Add catch-all PpdsException clause that maps to ExecutionFailed so
  unexpected PpdsException codes don't propagate unhandled

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…/fetch

InjectTopAttribute is still used by the query/fetch RPC handler for raw
FetchXML TOP injection. The spec incorrectly listed it for deletion.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…asks 8-12)

Integrate the 8 ppds:* query hints into the shared SqlQueryService so
they work across TUI, CLI, and VS Code. QueryHintParser was already
complete and tested but never called in production code — this wires
it into PrepareExecutionAsync and ExplainAsync.

Changes:
- Add ForceClientAggregation/NoLock to QueryPlanOptions
- Create QueryExecutionOptions record (BypassPlugins/BypassFlows)
- Add ExecutionOptions to QueryPlanContext
- Add IQueryExecutor.ExecuteFetchXmlAsync overload with DIM pattern
- Override in QueryExecutor using RetrieveMultipleRequest for bypass params
- FetchXmlScanNode: inject no-lock="true", route to new executor overload
- ExecutionPlanBuilder: pass noLock, add ForceClientAggregation routing
  to PlanClientSideAggregate with ClientAggregateNode
- SqlQueryService: parse hints, apply USE_TDS/MAX_ROWS/MAXDOP/NOLOCK/
  HASH_GROUP/BYPASS_PLUGINS/BYPASS_FLOWS overrides to plan options

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… SqlQueryResult

New QueryDataSource record identifies environments that contributed data
to a query result. SqlQueryResult gains DataSources (list of contributing
environments) and AppliedHints (hint names that were active).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds envColor: string | null to QueryPanelHostToWebview and
SolutionsPanelHostToWebview updateEnvironment messages to carry the
configured environment color from host to webview.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…sPanel

Adds an environmentColor field to both panels, fetches it from
daemon.envConfigGet() after initialization and environment picker
selection, and includes it in all updateEnvironment postMessage calls.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…toolbars

Adds CSS rules in shared.css mapping envColor values to left-border
accents on .toolbar, and updates both webview updateEnvironment handlers
to set/remove the data-env-color attribute accordingly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…an tree

Implements Task 19 — walks the QueryPlanResult tree after planning to collect
RemoteScanNode labels into DataSources and emits active hint names in
AppliedHints on SqlQueryResult and SqlQueryStreamChunk (final chunk).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…reExecutionAsync/ExplainAsync

Covers AC-04 through AC-13 and AC-21 across three test files:
- SqlQueryServiceTests: USE_TDS routing, NOLOCK FetchXML injection, MAX_ROWS cap,
  MAXDOP pool cap, BATCH_SIZE passthrough, hint overrides caller settings,
  malformed/unknown hints silently ignored, ExplainAsync plan reflection
- QueryExecutorTests: BYPASS_PLUGINS sets BypassCustomPluginExecution parameter,
  BYPASS_FLOWS sets SuppressCallbackRegistrationExpanderJob parameter,
  null options falls through to standard RetrieveMultiple path
- ExecutionPlanBuilderTests: HASH_GROUP forces ClientAggregateNode over FetchXML
  aggregate pushdown, non-aggregate queries unaffected

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…cross-env banner

- Add QueryDataSourceDto DTO and DataSources/AppliedHints fields to QueryResultResponse (C#)
- Map SqlQueryResult.DataSources and AppliedHints into RPC response in QuerySqlAsync (only when 2+ sources)
- Add dataSources/appliedHints to QueryResultResponse TypeScript interface
- Add #data-source-banner element to QueryPanel HTML template
- Wire banner show/hide logic in handleQueryResult based on dataSources presence
- Add .data-source-banner/.data-source-label/.data-source-tag CSS classes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…eResolutionService

When environments.json has duplicate labels, ProfileResolutionService
throws. Catch the ArgumentException and degrade gracefully — cross-env
queries are disabled but single-env queries proceed normally.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…sages

VS Code sends internal messages like {vscodeScheduleAsyncWork: 1} to
webview iframes. These lack the 'command' property and hit the
assertNever exhaustive switch default, causing uncaught errors. Add a
guard to skip messages without 'command' before entering the switch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…top override

The extension sends top:100 as a default on every query. Previously,
TopOverride always clobbered the SQL's own TOP clause, so SELECT TOP 5
returned 100 rows. Now TopOverride only applies when the SQL has no
explicit TOP — the user's intent takes precedence.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…across all skills

Retrospective found the previous session shipped broken features (TDS
lies, TOP override bug) because it never visually verified its work —
only ran mock-based unit tests. These changes close that gap:

- NEW /qa command: dispatches a fresh agent with no source code access
  to test the running product via CDP/CLI/MCP. Loops until all checks pass.
- CLAUDE.md: add verification and QA gate rules to Workflow section
- /gates: add post-gate reminder that mechanical gates are insufficient for UI
- /implement: invoke /qa at phase gates for extension, CLI, and MCP changes
- /verify: make Phase B mandatory for query/data-display changes
- @webview-panels: add visual verification to quality gates, update file tree
  (error-handler.ts, selection-utils.ts), document data-env-color theming
- @retrospective: add $ARGUMENTS support, default to latest session instead
  of hard-coded "2 days ago"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…TOP in ExplainAsync

- QueryMode now always reports "dataverse" since ITdsQueryExecutor is never
  registered in DI — was falsely reporting "tds" based on user toggle
- Commented out TDS Read Replica menu item that exposed non-functional feature
- ExplainAsync now guards explicit TOP against MaxResultRows hint override,
  matching the existing guard in PrepareExecutionAsync

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ocking

When running in F5 development mode, the daemon now copies build output
to a temp directory before spawning. This prevents dotnet build from
failing with "file in use" errors while the daemon holds locks on the
original binaries. The temp directory is cleaned up on dispose.

Also update /gates with file-locking recovery: if solution build fails
due to locked files, retry with individual changed projects.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Move scripts/*.js → tools/*.mjs (ESM, consistent with webview-cdp.mjs)
- Rename tests/tui-e2e/ → tests/PPDS.Tui.E2eTests/ (matches PPDS.*.Tests convention)
- Move docs/specs/query-engine-v2.md → specs/ (one spec directory, not two)
- Remove empty docs/specs/ and scripts/ directories
- Update all references: package.json, CI workflow, skills, specs, CONTRIBUTING

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- specs/README.md: remove dead IMPLEMENTATION_PLAN.md link, fix broken
  vault relative path
- specs/query-parity.md: fix AC-27 contradiction (InjectTopAttribute is
  retained, not deleted)
- CLAUDE.md: remove stale docs/superpowers/plans/ warning
- PrefetchScanNodeTests: remove ConfigureAwait(false) (xUnit1030)
- FetchXmlGeneratorTests: discard unused expectedAttr param (xUnit1026)
- webview-cdp skill: remove stale agent-browser reference

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…implementation plan

- Add Phase 5 to specs/query-parity.md covering DI registration, execution mode
  reporting, TDS failure behavior (fail, don't fall back), and UI updates
- Add implementation plan at docs/plans/2026-03-16-tds-endpoint-wiring.md
- 18 acceptance criteria (AC-30 through AC-48)
- Design decision: fail with clear error when TDS can't be used, never silently
  substitute Dataverse

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… DI registration

Chunks 1-3 of TDS Endpoint Wiring plan:

- Add ErrorCodes.Query.TdsIncompatible and TdsConnectionFailed
- Add QueryErrorCode.TdsIncompatible and TdsConnectionFailed (Dataverse layer)
- Create QueryExecutionMode enum (Dataverse, Tds)
- Add ExecutionMode property to SqlQueryResult and SqlQueryStreamChunk
- Fix TdsQueryExecutor to throw QueryExecutionException (D4 compliance)
- Add TDS compatibility pre-check in SqlQueryService.PrepareExecutionAsync
- Set ExecutionMode from plan tree in ExecuteAsync and ExecuteStreamingAsync
- Catch TDS connection failures and wrap in PpdsException(TdsConnectionFailed)
- Register ITdsQueryExecutor in daemon DI (per-environment singleton)
- Register ITdsQueryExecutor in CLI DI (transient, same auth pattern)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…fix TUI status

Chunk 4 of TDS Endpoint Wiring plan:

- RpcMethodHandler maps SqlQueryResult.ExecutionMode to queryMode RPC field
- Add TdsIncompatible and TdsConnectionFailed catch clauses in RPC error chain
- Un-comment TDS Read Replica menu item in VS Code query panel
- TUI status label shows "via TDS" or "via Dataverse" from streaming chunk

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- AC-38: verify ExecutionMode.Tds maps to queryMode "tds" in RPC response
- AC-38: verify queryMode JSON serialization for tds/dataverse/null
- AC-44: verify streaming final chunk carries ExecutionMode
- AC-44: verify non-final chunks have null ExecutionMode
- AC-34/35: verify ExecutionMode set correctly for TDS and Dataverse routes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
TUI was missing DML confirmation handling — when SqlQueryService threw
PpdsException(DmlConfirmationRequired), it fell through to the generic
error handler showing raw "requires --confirm" message with no way to
confirm. Now catches DmlConfirmationRequired, shows a confirmation
dialog, and retries with IsConfirmed=true if user chooses "Execute".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@joshsmithxrm
Copy link
Owner Author

@gemini-code-assist review

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This is a massive and impressive pull request that rebuilds the VS Code extension from the ground up with a new daemon architecture. The changes include significant improvements to the development process, with new AI skills for quality gates, verification, and design. The code itself is being migrated to a more robust structure with TypeScript webviews and typed messaging. My review focuses on ensuring the new infrastructure is sound and consistent. I've found a few minor areas for improvement, mostly in the documentation and scripts.

joshsmithxrm and others added 5 commits March 16, 2026 03:27
…-check and connection failure tests

- Extract duplicated token provider wiring from DaemonConnectionPoolManager
  and ServiceRegistration into shared TdsQueryExecutorFactory.Create()
- Add 4 missing tests from TDS implementation plan:
  - ExecuteAsync_TdsWithDml_ThrowsTdsIncompatible
  - ExecuteAsync_TdsWithIncompatibleEntity_ThrowsTdsIncompatible
  - ExecuteAsync_NoTds_DmlDoesNotThrowTdsIncompatible
  - ExecuteAsync_TdsConnectionFails_ThrowsTdsConnectionFailed

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Regenerate package-lock.json to match package.json after rebase
(fixes npm ci failure in CI). Reduce Linux status line from two
node processes to one, using basename for directory extraction.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comprehensive spec covering 6 panels (Import Jobs, Connection
References, Environment Variables, Plugin Traces, Metadata Browser,
Web Resources) across all 4 PPDS surfaces (Daemon RPC, VS Code, TUI,
MCP). Includes 73 acceptance criteria, issue reconciliation mapping
27 existing issues, and phased work breakdown with worktree strategy.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add typecheck step to CI build and publish workflows (esbuild
  doesn't type-check, so type errors could slip through)
- Fix stale extension path in Claude pre-commit hook (src/extension
  → src/PPDS.Extension)
- Propagate OperationCanceledException in QueryExplainAsync instead
  of swallowing it in bare catch
- Clear ExplainDocumentProvider singleton on dispose to prevent
  stale reference after extension reload
- Add escapeHtml/escapeAttr to context menu rendering for
  defense-in-depth XSS protection

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add extension/TS paths to ci-gate.yml so extension-only PRs
  trigger required status checks
- Make discovery cache thread-safe with Volatile.Read/Write and
  store-list-before-expiry ordering
- Add 3-failure threshold to daemon heartbeat before kill to
  tolerate transient network hiccups
- Fix E2E smoke test: replace vacuous if-guard with explicit
  assertion, correct view container selector ID
- Wrap sync-over-async RemoteExecutorFactory in Task.Run to
  prevent thread pool starvation on cache miss

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

2 participants