Skip to content

Informational dashboard using duckdb-wasm + apache echarts#1185

Merged
kevinschaper merged 22 commits intomainfrom
informational-dashboard
Aug 28, 2025
Merged

Informational dashboard using duckdb-wasm + apache echarts#1185
kevinschaper merged 22 commits intomainfrom
informational-dashboard

Conversation

@kevinschaper
Copy link
Member

Adding an informational dashboard inspired by evidence.dev, but using the same underlying technologies in our web app (with heavy help from claude code)

  • Add KG Dashboard infrastructure with DuckDB WASM integration
  • Add Sankey and Chord Chart components to KG Dashboard
  • Add comprehensive unit tests for dashboard components
  • Add BarChart component for node distribution visualization

kevinschaper and others added 4 commits August 1, 2025 22:11
Implements the foundational components for a Knowledge Graph dashboard
with client-side SQL analytics, similar to evidence.dev architecture.

Core Infrastructure:
- DuckDB WASM integration with local file serving (fixes CORS issues)
- SQL query composables with caching and error handling
- KG data management composables for parquet file loading
- Data processing pipeline to handle DuckDB data types (Uint32Array, BigInt)

Dashboard Components:
- BaseChart: Reusable chart container with SQL display functionality
- KGDashboard: Provider component for shared data context
- DataSource: Declarative component for registering parquet data sources
- KGMetricCard: Displays formatted metrics from SQL queries

Features Working:
✅ Client-side SQL queries against KG parquet files
✅ Metric cards displaying formatted KG statistics (nodes, edges, etc.)
✅ Proper data type processing and number formatting
✅ Declarative data source registration
✅ Vue 3 reactive data pipeline with provide/inject

Still TODO:
- Actual chart visualizations (BarChart, SankeyChart)
- Chart component implementations with ECharts

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Create SankeyChart component with ECharts integration for knowledge graph flow visualization
- Create ChordChart component with circular network layout for category connections
- Add interactive SVG bar chart tooltips with detailed connection breakdowns
- Implement smart radial tooltip positioning that avoids covering the chart
- Add export functionality for multiple formats (PNG, SVG, high-res PNG, JSON)
- Integrate both charts into PageKGDashboard with SQL-driven data queries
- Support real-time data processing from edge_report parquet files

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add unit tests for DuckDB WASM data processing and type handling
- Add tests for SQL query composable with error handling and caching
- Add tests for KG data management and data source registration
- Add tests for BaseChart component integration
- Add tests for KGMetricCard value formatting and display
- Tests cover JSON parsing, DuckDB data types, and edge cases
- Developed using test-driven approach to solve data processing issues

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Create horizontal BarChart component with ECharts integration
- Add flexible data column detection (category, name, count, value, etc.)
- Implement colorful bars with staggered animations and hover effects
- Add smart label formatting for long category names and large numbers
- Include comprehensive export functionality (PNG, SVG, high-res PNG, JSON)
- Integrate BarChart into KG Dashboard showing top 15 biomedical categories
- Support both horizontal and vertical orientations via props
- Add SQL query for node distribution from node_report parquet data
- Update placeholder sections with future visualization ideas

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@netlify
Copy link

netlify bot commented Aug 2, 2025

Deploy Preview for monarch-app ready!

Name Link
🔨 Latest commit 9e288ba
🔍 Latest deploy log https://app.netlify.com/projects/monarch-app/deploys/68b0b75fd4c28e0008e9bb29
😎 Deploy Preview https://deploy-preview-1185--monarch-app.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@codecov
Copy link

codecov bot commented Aug 2, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.07%. Comparing base (e488355) to head (9e288ba).
⚠️ Report is 27 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1185   +/-   ##
=======================================
  Coverage   72.07%   72.07%           
=======================================
  Files          92       92           
  Lines        3255     3255           
=======================================
  Hits         2346     2346           
  Misses        909      909           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

kevinschaper and others added 5 commits August 3, 2025 10:23
- Add loadCSV function to DuckDB composable using read_csv_auto
- Extend DataSourceConfig and DataSource component to support format prop
- Configure gene_connection_report as CSV format to fix loading issues
- Remove unused ortholog visualization imports from dashboard
- Update KG source fallback to use monarch-kg-dev/latest
- Handle API responses returning "unknown" for KG source

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Fixed JSDOM window.scrollTo polyfills in test setup
- Updated BaseChart test mocks with correct AppButton props
- Fixed KGMetricCard component mocks with proper Vue refs and computed values
- Added missing LogLevel export to DuckDB WASM mock
- Completed useSqlQuery mock interface with all required properties
- Fixed API mocks in use-kg-data tests for proper fallback behavior

All 148 unit tests now passing (100% success rate)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@kevinschaper kevinschaper marked this pull request as ready for review August 21, 2025 17:08
kevinschaper and others added 4 commits August 21, 2025 11:01
- Add public/duckdb/ to .gitignore to prevent tracking build artifacts
- Remove 40MB+ DuckDB WASM files from repository
- Files are auto-generated by Vite plugin during build process
- Reduces repository size and prevents merge conflicts on binary files
- Verified build process and DuckDB functionality still work correctly

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces a comprehensive informational dashboard for the Monarch Knowledge Graph, implementing a data visualization system using DuckDB WASM and Apache ECharts. The dashboard provides interactive charts and metrics for exploring KG structure, data quality, and content statistics.

  • Adds KG Dashboard infrastructure with DuckDB WASM integration for client-side SQL query execution
  • Implements multiple chart types (Sankey, Chord, Bar, Donut, etc.) with export capabilities
  • Provides comprehensive unit test coverage for all dashboard components and composables

Reviewed Changes

Copilot reviewed 29 out of 30 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
frontend/vitest.config.ts Adds test globals and NODE_ENV configuration
frontend/vite.config.mjs Implements DuckDB WASM file copying plugin and build optimization
frontend/src/router/index.ts Adds KG Dashboard route
frontend/src/pages/knowledgeGraph/PageKGDashboard.vue Main dashboard page with metrics and visualizations
frontend/src/composables/*.ts Core composables for DuckDB integration and SQL query execution
frontend/src/components/dashboard/*.vue Chart components with export functionality
frontend/unit/*.test.ts Comprehensive test suite for all dashboard functionality
frontend/src/util/*.ts Utility improvements for date formatting and DOM handling

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

"x",
tempContainer.offsetHeight,
);

Copy link

Copilot AI Aug 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove debug console.log statements from production code. These should be removed or replaced with proper logging.

Suggested change

Copilot uses AI. Check for mistakes.
);
let svgString = svgChart.renderToSVGString();
console.log("SVG string length:", svgString.length);
console.log("SVG starts with:", svgString.substring(0, 200));
Copy link

Copilot AI Aug 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove debug console.log statements from production code. These should be removed or replaced with proper logging.

Suggested change
console.log("SVG starts with:", svgString.substring(0, 200));
// console.log("Export options:", exportOptions); // Removed for production
svgChart.setOption(exportOptions);
// Give the chart time to render and then export
setTimeout(() => {
try {
// console.log(
// "Chart size:",
// svgChart.getWidth(),
// "x",
// svgChart.getHeight(),
// ); // Removed for production
let svgString = svgChart.renderToSVGString();
// console.log("SVG string length:", svgString.length); // Removed for production
// console.log("SVG starts with:", svgString.substring(0, 200)); // Removed for production

Copilot uses AI. Check for mistakes.
);
let svgString = svgChart.renderToSVGString();
console.log("SVG string length:", svgString.length);
console.log("SVG starts with:", svgString.substring(0, 200));
Copy link

Copilot AI Aug 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove debug console.log statements from production code. These should be removed or replaced with proper logging.

Suggested change
console.log("SVG starts with:", svgString.substring(0, 200));
// Chart size and SVG string debug logs removed
let svgString = svgChart.renderToSVGString();

Copilot uses AI. Check for mistakes.
'<svg width="2400" height="900" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" baseProfile="full">',
);

console.log("Modified SVG starts with:", svgString.substring(0, 200));
Copy link

Copilot AI Aug 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove debug console.log statements from production code. These should be removed or replaced with proper logging.

Suggested change
console.log("Modified SVG starts with:", svgString.substring(0, 200));

Copilot uses AI. Check for mistakes.
join(duckdbDir, "duckdb-browser-mvp.worker.js"),
);

console.log("✓ DuckDB WASM files copied to public/duckdb/");
Copy link

Copilot AI Aug 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Consider using a proper logging mechanism instead of console.log for build output. This could be replaced with a build-time logger or removed if not essential.

Suggested change
console.log("✓ DuckDB WASM files copied to public/duckdb/");
this.info("✓ DuckDB WASM files copied to public/duckdb/");

Copilot uses AI. Check for mistakes.
const handleUnregisterDataSource = (name: string): void => {
// Note: The composable doesn't have an unregister method yet
// For now, we'll just log this - in practice, data sources usually persist
console.debug(`DataSource "${name}" component unmounted`);
Copy link

Copilot AI Aug 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] Consider whether debug logging is necessary in production code. While console.debug is less intrusive than console.log, it may still clutter browser dev tools.

Suggested change
console.debug(`DataSource "${name}" component unmounted`);
if (isDevelopment.value) {
console.debug(`DataSource "${name}" component unmounted`);
}

Copilot uses AI. Check for mistakes.
// Silently handle initialization errors - they'll resolve once DuckDB is ready
return;
}
console.error("Auto-execute failed:", err);
Copy link

Copilot AI Aug 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider making this error message more specific to help with debugging. Include context about which SQL query failed and the data sources involved.

Suggested change
console.error("Auto-execute failed:", err);
console.error(
`Auto-execute failed for SQL: "${sql}" with dataSources: [${dataSources.join(", ")}]`,
err
);

Copilot uses AI. Check for mistakes.
if (err.message?.includes("DuckDB not initialized")) {
return;
}
console.error("Polling execution failed:", err);
Copy link

Copilot AI Aug 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider making this error message more specific to help with debugging. Include context about which SQL query failed during polling and the poll interval.

Suggested change
console.error("Polling execution failed:", err);
console.error(
`Polling execution failed for SQL: "${sql}" (pollInterval: ${pollInterval}ms):`,
err
);

Copilot uses AI. Check for mistakes.
@varun-divya
Copy link
Contributor

Works great and looks awesome!!. II’ve added a few minor questions/comments for clarity :-)

kevinschaper and others added 9 commits August 26, 2025 09:49
Removed DonutChart, GroupedBarChart, and StackedBarChart components
that were created but never used in the dashboard implementation.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Cleaned up development console.log and console.debug statements while
preserving console.error for legitimate error handling.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Replaced emoji icons in export menus and debug info with AppIcon components:
- 🖼️📷 → download icons for PNG exports
- 📐 → code icon for SVG export
- 📄 → file-lines icon for JSON export
- 🔍 → magnifying-glass icon for debug info

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Removed redundant ECharts title configurations from chart components
to eliminate duplicate titles. Each chart now shows only the BaseChart
HTML title which provides better accessibility and consistent styling.

Changes:
- Removed ECharts title config from BarChart, ChordChart, SankeyChart
- Updated dashboard section structure for cleaner layout
- Maintained individual descriptive chart titles

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Removed unused 'originalX' variable that was causing ESLint warning.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@kevinschaper
Copy link
Member Author

Cleaned up logging, unused charts and emojis and the listing is finally passing. I think this is ready to go!

@kevinschaper kevinschaper merged commit f63ab53 into main Aug 28, 2025
12 checks passed
@kevinschaper kevinschaper deleted the informational-dashboard branch August 28, 2025 21:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants