Informational dashboard using duckdb-wasm + apache echarts#1185
Informational dashboard using duckdb-wasm + apache echarts#1185kevinschaper merged 22 commits intomainfrom
Conversation
Implements the foundational components for a Knowledge Graph dashboard with client-side SQL analytics, similar to evidence.dev architecture. Core Infrastructure: - DuckDB WASM integration with local file serving (fixes CORS issues) - SQL query composables with caching and error handling - KG data management composables for parquet file loading - Data processing pipeline to handle DuckDB data types (Uint32Array, BigInt) Dashboard Components: - BaseChart: Reusable chart container with SQL display functionality - KGDashboard: Provider component for shared data context - DataSource: Declarative component for registering parquet data sources - KGMetricCard: Displays formatted metrics from SQL queries Features Working: ✅ Client-side SQL queries against KG parquet files ✅ Metric cards displaying formatted KG statistics (nodes, edges, etc.) ✅ Proper data type processing and number formatting ✅ Declarative data source registration ✅ Vue 3 reactive data pipeline with provide/inject Still TODO: - Actual chart visualizations (BarChart, SankeyChart) - Chart component implementations with ECharts 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Create SankeyChart component with ECharts integration for knowledge graph flow visualization - Create ChordChart component with circular network layout for category connections - Add interactive SVG bar chart tooltips with detailed connection breakdowns - Implement smart radial tooltip positioning that avoids covering the chart - Add export functionality for multiple formats (PNG, SVG, high-res PNG, JSON) - Integrate both charts into PageKGDashboard with SQL-driven data queries - Support real-time data processing from edge_report parquet files 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add unit tests for DuckDB WASM data processing and type handling - Add tests for SQL query composable with error handling and caching - Add tests for KG data management and data source registration - Add tests for BaseChart component integration - Add tests for KGMetricCard value formatting and display - Tests cover JSON parsing, DuckDB data types, and edge cases - Developed using test-driven approach to solve data processing issues 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Create horizontal BarChart component with ECharts integration - Add flexible data column detection (category, name, count, value, etc.) - Implement colorful bars with staggered animations and hover effects - Add smart label formatting for long category names and large numbers - Include comprehensive export functionality (PNG, SVG, high-res PNG, JSON) - Integrate BarChart into KG Dashboard showing top 15 biomedical categories - Support both horizontal and vertical orientations via props - Add SQL query for node distribution from node_report parquet data - Update placeholder sections with future visualization ideas 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
✅ Deploy Preview for monarch-app ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1185 +/- ##
=======================================
Coverage 72.07% 72.07%
=======================================
Files 92 92
Lines 3255 3255
=======================================
Hits 2346 2346
Misses 909 909 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
- Add loadCSV function to DuckDB composable using read_csv_auto - Extend DataSourceConfig and DataSource component to support format prop - Configure gene_connection_report as CSV format to fix loading issues - Remove unused ortholog visualization imports from dashboard - Update KG source fallback to use monarch-kg-dev/latest - Handle API responses returning "unknown" for KG source 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
…hing is fully initialized
- Fixed JSDOM window.scrollTo polyfills in test setup - Updated BaseChart test mocks with correct AppButton props - Fixed KGMetricCard component mocks with proper Vue refs and computed values - Added missing LogLevel export to DuckDB WASM mock - Completed useSqlQuery mock interface with all required properties - Fixed API mocks in use-kg-data tests for proper fallback behavior All 148 unit tests now passing (100% success rate) 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add public/duckdb/ to .gitignore to prevent tracking build artifacts - Remove 40MB+ DuckDB WASM files from repository - Files are auto-generated by Vite plugin during build process - Reduces repository size and prevents merge conflicts on binary files - Verified build process and DuckDB functionality still work correctly 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Pull Request Overview
This PR introduces a comprehensive informational dashboard for the Monarch Knowledge Graph, implementing a data visualization system using DuckDB WASM and Apache ECharts. The dashboard provides interactive charts and metrics for exploring KG structure, data quality, and content statistics.
- Adds KG Dashboard infrastructure with DuckDB WASM integration for client-side SQL query execution
- Implements multiple chart types (Sankey, Chord, Bar, Donut, etc.) with export capabilities
- Provides comprehensive unit test coverage for all dashboard components and composables
Reviewed Changes
Copilot reviewed 29 out of 30 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
| frontend/vitest.config.ts | Adds test globals and NODE_ENV configuration |
| frontend/vite.config.mjs | Implements DuckDB WASM file copying plugin and build optimization |
| frontend/src/router/index.ts | Adds KG Dashboard route |
| frontend/src/pages/knowledgeGraph/PageKGDashboard.vue | Main dashboard page with metrics and visualizations |
| frontend/src/composables/*.ts | Core composables for DuckDB integration and SQL query execution |
| frontend/src/components/dashboard/*.vue | Chart components with export functionality |
| frontend/unit/*.test.ts | Comprehensive test suite for all dashboard functionality |
| frontend/src/util/*.ts | Utility improvements for date formatting and DOM handling |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| "x", | ||
| tempContainer.offsetHeight, | ||
| ); | ||
|
|
There was a problem hiding this comment.
Remove debug console.log statements from production code. These should be removed or replaced with proper logging.
| ); | ||
| let svgString = svgChart.renderToSVGString(); | ||
| console.log("SVG string length:", svgString.length); | ||
| console.log("SVG starts with:", svgString.substring(0, 200)); |
There was a problem hiding this comment.
Remove debug console.log statements from production code. These should be removed or replaced with proper logging.
| console.log("SVG starts with:", svgString.substring(0, 200)); | |
| // console.log("Export options:", exportOptions); // Removed for production | |
| svgChart.setOption(exportOptions); | |
| // Give the chart time to render and then export | |
| setTimeout(() => { | |
| try { | |
| // console.log( | |
| // "Chart size:", | |
| // svgChart.getWidth(), | |
| // "x", | |
| // svgChart.getHeight(), | |
| // ); // Removed for production | |
| let svgString = svgChart.renderToSVGString(); | |
| // console.log("SVG string length:", svgString.length); // Removed for production | |
| // console.log("SVG starts with:", svgString.substring(0, 200)); // Removed for production |
| ); | ||
| let svgString = svgChart.renderToSVGString(); | ||
| console.log("SVG string length:", svgString.length); | ||
| console.log("SVG starts with:", svgString.substring(0, 200)); |
There was a problem hiding this comment.
Remove debug console.log statements from production code. These should be removed or replaced with proper logging.
| console.log("SVG starts with:", svgString.substring(0, 200)); | |
| // Chart size and SVG string debug logs removed | |
| let svgString = svgChart.renderToSVGString(); |
| '<svg width="2400" height="900" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" baseProfile="full">', | ||
| ); | ||
|
|
||
| console.log("Modified SVG starts with:", svgString.substring(0, 200)); |
There was a problem hiding this comment.
Remove debug console.log statements from production code. These should be removed or replaced with proper logging.
| console.log("Modified SVG starts with:", svgString.substring(0, 200)); |
| join(duckdbDir, "duckdb-browser-mvp.worker.js"), | ||
| ); | ||
|
|
||
| console.log("✓ DuckDB WASM files copied to public/duckdb/"); |
There was a problem hiding this comment.
[nitpick] Consider using a proper logging mechanism instead of console.log for build output. This could be replaced with a build-time logger or removed if not essential.
| console.log("✓ DuckDB WASM files copied to public/duckdb/"); | |
| this.info("✓ DuckDB WASM files copied to public/duckdb/"); |
| const handleUnregisterDataSource = (name: string): void => { | ||
| // Note: The composable doesn't have an unregister method yet | ||
| // For now, we'll just log this - in practice, data sources usually persist | ||
| console.debug(`DataSource "${name}" component unmounted`); |
There was a problem hiding this comment.
[nitpick] Consider whether debug logging is necessary in production code. While console.debug is less intrusive than console.log, it may still clutter browser dev tools.
| console.debug(`DataSource "${name}" component unmounted`); | |
| if (isDevelopment.value) { | |
| console.debug(`DataSource "${name}" component unmounted`); | |
| } |
| // Silently handle initialization errors - they'll resolve once DuckDB is ready | ||
| return; | ||
| } | ||
| console.error("Auto-execute failed:", err); |
There was a problem hiding this comment.
Consider making this error message more specific to help with debugging. Include context about which SQL query failed and the data sources involved.
| console.error("Auto-execute failed:", err); | |
| console.error( | |
| `Auto-execute failed for SQL: "${sql}" with dataSources: [${dataSources.join(", ")}]`, | |
| err | |
| ); |
| if (err.message?.includes("DuckDB not initialized")) { | ||
| return; | ||
| } | ||
| console.error("Polling execution failed:", err); |
There was a problem hiding this comment.
Consider making this error message more specific to help with debugging. Include context about which SQL query failed during polling and the poll interval.
| console.error("Polling execution failed:", err); | |
| console.error( | |
| `Polling execution failed for SQL: "${sql}" (pollInterval: ${pollInterval}ms):`, | |
| err | |
| ); |
|
Works great and looks awesome!!. II’ve added a few minor questions/comments for clarity :-) |
Removed DonutChart, GroupedBarChart, and StackedBarChart components that were created but never used in the dashboard implementation. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Cleaned up development console.log and console.debug statements while preserving console.error for legitimate error handling. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Replaced emoji icons in export menus and debug info with AppIcon components: - 🖼️📷 → download icons for PNG exports - 📐 → code icon for SVG export - 📄 → file-lines icon for JSON export - 🔍 → magnifying-glass icon for debug info 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Removed redundant ECharts title configurations from chart components to eliminate duplicate titles. Each chart now shows only the BaseChart HTML title which provides better accessibility and consistent styling. Changes: - Removed ECharts title config from BarChart, ChordChart, SankeyChart - Updated dashboard section structure for cleaner layout - Maintained individual descriptive chart titles 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Removed unused 'originalX' variable that was causing ESLint warning. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
|
Cleaned up logging, unused charts and emojis and the listing is finally passing. I think this is ready to go! |
Adding an informational dashboard inspired by evidence.dev, but using the same underlying technologies in our web app (with heavy help from claude code)