Skip to content

feat(lineage): add lineage visualization across datasets, charts and dashboards - #40912

Draft
rusackas wants to merge 11 commits into
masterfrom
adopt/lineage-visualization
Draft

feat(lineage): add lineage visualization across datasets, charts and dashboards#40912
rusackas wants to merge 11 commits into
masterfrom
adopt/lineage-visualization

Conversation

@rusackas

@rusackas rusackas commented Jun 9, 2026

Copy link
Copy Markdown
Member

SUMMARY

Adopts #37701 by @qf-jonathan — thank you for the substantial work here! 🙏

Adds interactive Sankey-based lineage that lets users explore data relationships:

  • Dataset lineage — upstream database and downstream charts/dashboards
  • Chart lineage — database → dataset → chart → dashboards pipeline
  • Dashboard lineage — all upstream databases, datasets and charts feeding a dashboard

Implementation: three REST endpoints (/api/v1/lineage/{dataset|chart|dashboard}/<id>) with permissions + integration tests, and a LineageView/LineageModal frontend (ECharts Sankey) surfaced from the dataset editor and the chart/dashboard action menus.

What changed vs. the original PR

Rebased onto current master and updated for drift accumulated since the original branch (Feb):

  • Resolved merge conflicts in constants.py, ChartCard, useHeaderActionsDropdownMenu, and the chart integration tests.
  • Repointed the lineage imports from the now-removed @apache-superset/core/ui to @apache-superset/core/translation and /theme (the package layout changed).
  • Re-applied the dataset lineage tab onto the migrated (TypeScript) DatasourceEditor (it was .jsx when the original PR was written).

TESTING INSTRUCTIONS

Open a dataset → Lineage tab; open a chart/dashboard menu → Lineage; verify the Sankey renders upstream/downstream relationships.

Backend: pytest tests/integration_tests/{charts,dashboards,datasets}/api_tests.py -k lineage

Verification status (honest notes for reviewers)

  • ✅ Backend static checks pass locally (mypy, ruff, pylint, license headers).
  • ✅ Frontend: prettier clean; the adopted TS files type-check cleanly after the import fixes.
  • ⚠️ oxlint and the backend integration tests weren't run in my environment (no DB); CI will exercise both.
  • ⚠️ End-to-end feature behavior is best confirmed with a manual smoke test in a running instance.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration
  • Introduces new feature or API
  • Removes existing feature or API

Closes #37701

🤖 Generated with Claude Code

@github-actions github-actions Bot added the api Related to the REST API label Jun 9, 2026
@sadpandajoe sadpandajoe added the review:checkpoint Last PR reviewed during the daily review standup label Jun 9, 2026
Comment thread superset-frontend/src/features/lineage/LineageView.tsx Outdated
Comment thread superset/datasets/api.py
@dosubot dosubot Bot added change:backend Requires changing the backend change:frontend Requires changing the frontend viz:charts:sankey Related to the Sankey chart labels Jun 9, 2026
Comment thread superset-frontend/src/features/datasets/AddDataset/EditDataset/index.tsx Outdated
Comment thread superset-frontend/src/features/lineage/LineageModal.tsx
Comment thread superset-frontend/src/features/lineage/LineageView.tsx Outdated
Comment thread superset-frontend/src/features/lineage/LineageView.tsx Outdated
Comment thread superset/dashboards/api.py
Comment thread superset/dashboards/api.py
Comment thread superset/datasets/api.py Outdated
Comment thread superset/datasets/api.py Outdated
Comment thread tests/integration_tests/charts/api_tests.py Outdated
Comment thread superset-frontend/src/hooks/apiResources/lineage.ts Outdated
Comment thread superset/charts/api.py Outdated
@codecov

codecov Bot commented Jun 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 51.31004% with 223 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.34%. Comparing base (765a4ec) to head (f75d48d).

Files with missing lines Patch % Lines
...rset-frontend/src/features/lineage/LineageView.tsx 6.10% 200 Missing ⚠️
superset/charts/api.py 75.00% 4 Missing and 3 partials ⚠️
superset/datasets/api.py 80.00% 3 Missing and 3 partials ⚠️
...e/components/DatasourceEditor/DatasourceEditor.tsx 0.00% 5 Missing ⚠️
...set-frontend/src/features/lineage/LineageModal.tsx 85.71% 3 Missing ⚠️
...mponents/useExploreAdditionalActionsMenu/index.tsx 66.66% 1 Missing ⚠️
superset/dashboards/api.py 95.83% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #40912      +/-   ##
==========================================
- Coverage   79.41%   79.34%   -0.08%     
==========================================
  Files        2895     2898       +3     
  Lines      167947   168402     +455     
  Branches    38896    39003     +107     
==========================================
+ Hits       133382   133619     +237     
- Misses      32065    32276     +211     
- Partials     2500     2507       +7     
Flag Coverage Δ
hive 37.85% <66.66%> (+0.06%) ⬆️
javascript 74.77% <22.30%> (-0.17%) ⬇️
mysql 57.58% <92.59%> (+0.08%) ⬆️
postgres 57.62% <92.59%> (+0.08%) ⬆️
presto 39.73% <66.66%> (+0.06%) ⬆️
python 83.89% <92.59%> (+0.02%) ⬆️
sqlite 57.31% <92.59%> (+0.08%) ⬆️
unit 74.32% <66.66%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@bito-code-review bito-code-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Agent Run #9dcb5b

Actionable Suggestions - 4
  • superset/dashboards/api.py - 1
  • superset-frontend/src/features/lineage/LineageView.tsx - 2
  • superset-frontend/src/features/lineage/LineageModal.tsx - 1
    • Unconditional hook calls waste resources · Line 40-46
Additional Suggestions - 9
  • superset-frontend/src/features/lineage/LineageModal.tsx - 1
    • Missing unit tests for new component · Line 1-79
      The new LineageModal component lacks unit tests. Per project guidelines, new features should include tests covering success paths, error scenarios, and edge cases.
  • superset/charts/api.py - 1
    • CWE-285: Improper Authorization · Line 353-356
      ChartDAO.get_by_id_or_uuid() applies ChartFilter which returns no results for unauthorized users, causing ChartNotFoundError to be raised. This design conflates 'not found' with 'access denied'. Consider refactoring to raise ChartAccessDeniedError when filters deny access.
  • superset-frontend/src/components/Datasource/components/DatasourceEditor/DatasourceEditor.tsx - 1
    • Redundant Loading state · Line 432-438
      The `DatasetLineageTab` wrapper calls `useDatasetLineage(datasourceId ?? 0)` even when `datasourceId` is undefined, triggering a spurious `/api/v1/dataset/0/lineage` request. The check at line 434 then renders `` anyway, duplicating `LineageView`'s own loading state. Since the parent at line 2536 already has a non-null `datasource.id`, either pass the resource as a prop or return `null` for falsy `datasourceId`.
  • tests/integration_tests/charts/api_tests.py - 1
    • Unused import with noqa · Line 53-53
      The `client` import on line 53 has `# noqa: F401` but is not referenced anywhere in the diff. Either remove it or add a comment explaining why it's needed as a module-level import.
  • superset-frontend/src/features/charts/ChartCard.tsx - 1
    • Missing test identifier · Line 105-126
      The new lineage menu item is missing a `data-test` attribute, making it inconsistent with DashboardCard (which has `data-test="dashboard-card-option-lineage-button"` at line 112). This will hinder automated testing of the ChartCard component.
  • superset/dashboards/schemas.py - 1
    • Missing allow_none for optional field · Line 564-567
      The `slug` field should specify `allow_none=True` to match the API behavior where `dash.slug` can be None. Without this, marshmallow serialization may raise an error when encountering null values.
      Code suggestion
      --- a/superset/dashboards/schemas.py
      +++ b/superset/dashboards/schemas.py
       @@ -563,7 +563,7 @@ class DashboardLineageDashboardSchema(Schema):
        class DashboardLineageDashboardSchema(Schema):
            id = fields.Integer()
            title = fields.String()
      -    slug = fields.String()
      +    slug = fields.String(allow_none=True)
            published = fields.Boolean()
  • superset-frontend/src/explore/components/useExploreAdditionalActionsMenu/index.tsx - 1
    • Inconsistent test attribute naming · Line 1042-1042
      The `data-test` attribute `view-lineage-menu-item` conflicts with the existing convention in `DashboardCard.tsx` (line 112) which uses `dashboard-card-option-lineage-button` for similar lineage menu items. Use `view-lineage-menu-item-button` to maintain consistent test selector naming across the explore and dashboard features.
  • tests/integration_tests/dashboards/api_tests.py - 1
    • Unused import in test file · Line 72-72
      The `lineage_test_data` fixture is imported but never used in this file. The `inject_expected_dashboard_lineage` fixture (line 70-71) already depends on `lineage_test_data` internally, so this direct import is redundant.
      Code suggestion
      --- tests/integration_tests/dashboards/api_tests.py
      +++ tests/integration_tests/dashboards/api_tests.py
       @@ -68,8 +68,7 @@
        )
        from tests.integration_tests.fixtures.client import client  # noqa: F401
        from tests.integration_tests.fixtures.lineage import (
            inject_expected_dashboard_lineage,  # noqa: F401
      -    lineage_test_data,  # noqa: F401
        )
        from tests.integration_tests.fixtures.world_bank_dashboard import (
  • superset-frontend/src/hooks/apiResources/index.ts - 1
    • Missing unit tests for lineage hooks · Line 32-32
      The `lineage.ts` module exports 3 API hooks but lacks a corresponding test file. Other modules in this directory (dashboards, datasets, queries, etc.) have dedicated `.test.ts` files following the established pattern. Per BITO.md rule [11730], new tools should include comprehensive unit tests covering success paths, error scenarios, validation failures, and edge cases.
Filtered by Review Rules

Bito filtered these suggestions based on rules created automatically for your feedback. Manage rules.

  • superset/dashboards/api.py - 1
  • superset-frontend/src/features/datasets/AddDataset/EditDataset/index.tsx - 1
  • superset-frontend/src/features/lineage/LineageView.tsx - 2
  • superset/charts/api.py - 1
  • superset-frontend/src/dashboard/components/Header/useHeaderActionsDropdownMenu.tsx - 1
Review Details
  • Files reviewed - 23 · Commit Range: 23123ce..23123ce
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/DatasourceEditor.tsx
    • superset-frontend/src/dashboard/components/Header/useHeaderActionsDropdownMenu.tsx
    • superset-frontend/src/dashboard/types.ts
    • superset-frontend/src/explore/components/useExploreAdditionalActionsMenu/index.tsx
    • superset-frontend/src/features/charts/ChartCard.tsx
    • superset-frontend/src/features/dashboards/DashboardCard.tsx
    • superset-frontend/src/features/datasets/AddDataset/EditDataset/index.tsx
    • superset-frontend/src/features/lineage/LineageModal.tsx
    • superset-frontend/src/features/lineage/LineageView.tsx
    • superset-frontend/src/features/lineage/index.ts
    • superset-frontend/src/hooks/apiResources/index.ts
    • superset-frontend/src/hooks/apiResources/lineage.ts
    • superset/charts/api.py
    • superset/charts/schemas.py
    • superset/constants.py
    • superset/dashboards/api.py
    • superset/dashboards/schemas.py
    • superset/datasets/api.py
    • superset/datasets/schemas.py
    • tests/integration_tests/charts/api_tests.py
    • tests/integration_tests/dashboards/api_tests.py
    • tests/integration_tests/datasets/api_tests.py
    • tests/integration_tests/fixtures/lineage.py
  • Files skipped - 0
  • Tools
    • Eslint (Linter) - ✔︎ Successful
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

Comment thread superset/dashboards/api.py Outdated
Comment thread superset-frontend/src/features/lineage/LineageView.tsx
Comment thread superset-frontend/src/features/lineage/LineageView.tsx
Comment thread superset-frontend/src/features/lineage/LineageModal.tsx
@netlify

netlify Bot commented Jun 9, 2026

Copy link
Copy Markdown

Deploy Preview for superset-docs-preview ready!

Name Link
🔨 Latest commit f75d48d
🔍 Latest deploy log https://app.netlify.com/projects/superset-docs-preview/deploys/6a9a285279ef80000925e94f
😎 Deploy Preview https://deploy-preview-40912--superset-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@bito-code-review bito-code-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Agent Run #e7785a

Actionable Suggestions - 1
  • superset-frontend/src/hooks/apiResources/apiResources.ts - 1
    • Missing unit tests for skip parameter · Line 89-89
Additional Suggestions - 1
  • superset/charts/api.py - 1
    • Test coverage gap for permission filter · Line 389-395
      The existing test `test_get_chart_lineage` uses `ADMIN_USERNAME` and will continue to pass. Consider adding a test with a non-admin user who has access to only some dashboards to verify the permission filtering is effective.
Review Details
  • Files reviewed - 12 · Commit Range: 23123ce..f8f4010
    • superset-frontend/src/dashboard/components/Header/useHeaderActionsDropdownMenu.tsx
    • superset-frontend/src/features/datasets/AddDataset/EditDataset/index.tsx
    • superset-frontend/src/features/lineage/LineageView.tsx
    • superset-frontend/src/hooks/apiResources/apiResources.ts
    • superset-frontend/src/hooks/apiResources/lineage.ts
    • superset/charts/api.py
    • superset/dashboards/api.py
    • superset/datasets/api.py
    • tests/integration_tests/charts/api_tests.py
    • tests/integration_tests/dashboards/api_tests.py
    • tests/integration_tests/datasets/api_tests.py
    • superset-frontend/src/hooks/apiResources/lineage.test.ts
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • Eslint (Linter) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

Comment thread superset-frontend/src/hooks/apiResources/apiResources.ts
@bito-code-review

bito-code-review Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Code Review Agent Run #8416a4

Actionable Suggestions - 0
Additional Suggestions - 1
  • superset-frontend/src/hooks/apiResources/apiResources.test.ts - 1
    • Incomplete toggle scenario coverage · Line 118-141
      The toggle test only covers the happy path (true→false). Consider adding a symmetric test for skip=false→true to verify the hook correctly returns to loading state without triggering a fetch when skipping is re-enabled.
      Code suggestion
      --- superset-frontend/src/hooks/apiResources/apiResources.test.ts (lines 141-142)
       141:       });
       142:     });
      +   
      +   test('returns to loading when skip toggles from false to true', async () => {
      +     const fetchMock = jest.fn().mockResolvedValue(fakeApiResult);
      +     (makeApi as any).mockReturnValue(fetchMock);
      +     const { result, rerender } = renderHook(
      +       ({ skip }) => useApiResourceFullBody('/test/endpoint', skip),
      +       { initialProps: { skip: false } },
      +     );
      +     await act(async () => {
      +       jest.runAllTimers();
      +     });
      +     expect(fetchMock).toHaveBeenCalledTimes(1);
      +     expect(result.current.status).toEqual(ResourceStatus.Complete);
      +
      +     rerender({ skip: true });
      +     await act(async () => {
      +       jest.runAllTimers();
      +     });
      +     expect(fetchMock).toHaveBeenCalledTimes(1);
      +     expect(result.current).toEqual({
      +       status: ResourceStatus.Loading,
      +       result: null,
      +       error: null,
      +     });
      +   });
           });
Review Details
  • Files reviewed - 1 · Commit Range: f8f4010..766b4a8
    • superset-frontend/src/hooks/apiResources/apiResources.test.ts
  • Files skipped - 0
  • Tools
    • Eslint (Linter) - ✔︎ Successful
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

@apache apache deleted a comment from github-actions Bot Jun 10, 2026
@rusackas rusackas added the 🎪 ⚡ showtime-trigger-start Create new ephemeral environment for this PR label Jun 10, 2026
@github-actions github-actions Bot added 🎪 766b4a8 🚦 building 🎪 ⌛ 48h Environment expires after 48 hours (default) and removed 🎪 ⚡ showtime-trigger-start Create new ephemeral environment for this PR labels Jun 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🎪 Showtime is building environment on GHA for 766b4a8

@rusackas
rusackas requested review from Vitor-Avila and removed request for michael-s-molina June 10, 2026 23:54
@rusackas

Copy link
Copy Markdown
Member Author

This might warrant an extension, but the work was already part done, thus the adoption... we can consider merging this, or leaving it behind... or @qf-jonathan can reopen theirs and I can close this out if they want to pick it up and drive it.

Comment thread superset/dashboards/schemas.py Outdated
Comment thread superset/datasets/api.py Outdated
Comment thread superset-frontend/src/features/lineage/LineageView.tsx
Comment thread superset-frontend/src/features/lineage/LineageView.tsx Outdated

@bito-code-review bito-code-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Agent Run #7c9e20

Actionable Suggestions - 2
  • superset-frontend/src/features/lineage/LineageView.tsx - 2
Additional Suggestions - 10
  • superset-frontend/src/hooks/apiResources/apiResources.ts - 1
    • Missing test coverage for skip param · Line 191-198
      The `useApiV1Resource` hook now forwards `skip` to `useApiResourceFullBody` but lacks dedicated unit tests. While the underlying hook is tested, adaptive rule [11730] requires coverage for new API entry points. Add tests similar to lines 101-141: one for `skip=true` and one for the toggle behavior.
  • superset-frontend/src/features/lineage/LineageView.tsx - 2
    • CWE-835: Missing sink for optional relationship · Line 537-556
      When `chart.dataset_id` is undefined, the chart appears orphaned in the sankey even though the upstream dataset references it via `chart_ids`. Add fallback logic to connect charts via their containing dataset's `chart_ids` array.
    • CWE-477: Reload instead of router navigation · Line 633-645
      Line 637 uses `window.location.href = getEntityUrl(selectedNode)` to navigate after clicking Open. This causes a full page reload instead of a client-side navigation, resetting React state.
  • tests/integration_tests/datasets/api_tests.py - 1
    • Unused import dead code · Line 58-58
      The `client` fixture import is unused throughout this file. Only line 58 imports it, but grep confirms no reference to `client` exists beyond that import. Dead import increases maintenance burden and may confuse readers about actual test dependencies. Remove it.
  • superset/datasets/schemas.py - 1
    • Missing OpenAPI field metadata · Line 253-304
      New DatasetLineage schemas lack OpenAPI field descriptions (metadata). The file consistently uses `metadata={"description": "..."}` on all field definitions (e.g., lines 233-244), but lines 253-304 are missing this documentation, reducing OpenAPI spec quality.
  • tests/integration_tests/charts/api_tests.py - 1
    • Unused import introduced by diff · Line 53-53
      The module-level `client` fixture (from `tests.integration_tests.fixtures.client`) is not referenced by either of the new test methods — both use `self.client` inherited from the test class base. The import is dead code in the context of these additions.
  • superset-frontend/src/hooks/apiResources/apiResources.test.ts - 1
    • Missing test coverage for skip reset · Line 101-141
      Consider adding a test case for toggling skip back to true (false→true) to complement the existing test for true→false. This would cover the scenario where a user cancels an operation mid-fetch, ensuring the loading state is preserved and no stale data appears.
  • superset-frontend/src/features/lineage/LineageModal.tsx - 1
    • Add unit tests for new component · Line 1-79
      This new component lacks unit tests. Per project guidelines, comprehensive unit tests should cover success paths and error scenarios for all entity types the component supports.
  • superset-frontend/src/explore/components/useExploreAdditionalActionsMenu/index.tsx - 1
    • Missing test coverage · Line 1082-1097
      The new VIEW_LINEAGE feature is missing test coverage. The existing test file (useExploreAdditionalActionsMenu.test.tsx) has no tests for this functionality. Per project guidelines, comprehensive unit tests should cover success paths, error scenarios, and edge cases for new features. ([11730](https://cwe.mitre.org/data/definitions/11730.html))
  • superset/dashboards/schemas.py - 1
    • Missing OpenAPI field descriptions · Line 638-692
      The new lineage schemas lack OpenAPI field descriptions. Existing schemas in this file (e.g., `DashboardGetResponseSchema`) use `metadata={"description": ...}` for API documentation. Adding descriptions improves the OpenAPI spec generated for the `/dashboard/{id}/lineage` endpoint.
Filtered by Review Rules

Bito filtered these suggestions based on rules created automatically for your feedback. Manage rules.

  • superset/charts/schemas.py - 8
  • superset-frontend/src/components/Datasource/components/DatasourceEditor/DatasourceEditor.tsx - 1
  • superset-frontend/src/hooks/apiResources/apiResources.ts - 2
  • superset/datasets/api.py - 1
Review Details
  • Files reviewed - 26 · Commit Range: 60dacb5..ea5c33f
    • superset-frontend/src/components/Datasource/components/DatasourceEditor/DatasourceEditor.tsx
    • superset-frontend/src/dashboard/components/Header/useHeaderActionsDropdownMenu.tsx
    • superset-frontend/src/dashboard/types.ts
    • superset-frontend/src/explore/components/useExploreAdditionalActionsMenu/index.tsx
    • superset-frontend/src/features/charts/ChartCard.tsx
    • superset-frontend/src/features/dashboards/DashboardCard.tsx
    • superset-frontend/src/features/datasets/AddDataset/EditDataset/index.tsx
    • superset-frontend/src/features/lineage/LineageModal.tsx
    • superset-frontend/src/features/lineage/LineageView.tsx
    • superset-frontend/src/features/lineage/index.ts
    • superset-frontend/src/hooks/apiResources/apiResources.test.ts
    • superset-frontend/src/hooks/apiResources/apiResources.ts
    • superset-frontend/src/hooks/apiResources/index.ts
    • superset-frontend/src/hooks/apiResources/lineage.test.ts
    • superset-frontend/src/hooks/apiResources/lineage.ts
    • superset/charts/api.py
    • superset/charts/schemas.py
    • superset/constants.py
    • superset/dashboards/api.py
    • superset/dashboards/schemas.py
    • superset/datasets/api.py
    • superset/datasets/schemas.py
    • tests/integration_tests/charts/api_tests.py
    • tests/integration_tests/dashboards/api_tests.py
    • tests/integration_tests/datasets/api_tests.py
    • tests/integration_tests/fixtures/lineage.py
  • Files skipped - 0
  • Tools
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • Eslint (Linter) - ✔︎ Successful

Bito Usage Guide

Commands

Type the following command in the pull request comment and save the comment.

  • /review - Manually triggers a full AI review.

  • /pause - Pauses automatic reviews on this pull request.

  • /resume - Resumes automatic reviews.

  • /resolve - Marks all Bito-posted review comments as resolved.

  • /abort - Cancels all in-progress reviews.

Refer to the documentation for additional commands.

Configuration

This repository uses Superset You can customize the agent settings here or contact your Bito workspace admin at evan@preset.io.

Documentation & Help

AI Code Review powered by Bito Logo

Comment thread superset-frontend/src/features/lineage/LineageView.tsx
Comment thread superset-frontend/src/features/lineage/LineageView.tsx
@michael-s-molina

Copy link
Copy Markdown
Member

@rusackas At Airbnb, we have systems to display lineage information with much richer information. I wonder if this should be an extension of Superset instead of part of the core?

Comment thread superset/charts/api.py Outdated
Comment thread superset/dashboards/api.py Outdated
Comment thread superset/dashboards/api.py Outdated
Comment thread superset/dashboards/api.py Outdated
Comment thread superset/datasets/api.py Outdated
Comment thread superset/datasets/api.py Outdated
Comment thread superset/datasets/api.py Outdated
Comment thread superset/datasets/api.py Outdated
Comment thread superset/datasets/api.py Outdated
Comment thread superset-frontend/src/features/lineage/LineageView.tsx
Comment thread superset/dashboards/api.py
Comment thread superset/datasets/api.py
Comment thread superset/datasets/api.py
Comment thread superset/datasets/schemas.py Outdated
Comment thread superset/datasets/schemas.py Outdated
@rusackas
rusackas marked this pull request as draft June 29, 2026 20:12
@rusackas

Copy link
Copy Markdown
Member Author

Agreed @michael-s-molina - this probably ought to be an extension, and would warrant a SIP if not. I've converted it to draft for now.

If y'all have an Extension that's better and on track to be open-sourced (and doesn't require additional hosted tooling) then we can probably just close this effort in favor of that effort.

@sadpandajoe sadpandajoe added review:draft and removed review:checkpoint Last PR reviewed during the daily review standup labels Jun 30, 2026
@sha174n

sha174n commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

The chart lineage endpoint returns the upstream dataset block (schema, table_name, database_name, backend, ids) unconditionally, whereas the dashboard lineage endpoint in this same PR gates that block behind security_manager.can_access_datasource(...) and redacts those fields when the caller lacks access. Could we mirror the dashboard endpoint's gating on the chart endpoint so the two behave consistently?

@rusackas

Copy link
Copy Markdown
Member Author

@sha174n good catch, fixed. The chart endpoint now redacts schema/table/database the same way the dashboard one does when the caller can't access the dataset.

@rusackas
rusackas force-pushed the adopt/lineage-visualization branch from 867327c to 08cb0f6 Compare August 20, 2026 04:26
@github-actions github-actions Bot added the requires:rebase Requires rebasing on top of current master label Sep 3, 2026
claude and others added 11 commits September 3, 2026 18:59
…dashboards

Adopts #37701 by @qf-jonathan. Adds interactive Sankey-based lineage that shows
the data relationships for a dataset, chart, or dashboard: three REST endpoints
(/api/v1/lineage/{dataset|chart|dashboard}/<id>) plus a LineageView/LineageModal
frontend surfaced from the dataset editor and the chart/dashboard menus.

Rebased onto current master and updated for drift since the original branch:
- repointed lineage imports from the removed '@apache-superset/core/ui' to
  '@apache-superset/core/translation' and '/theme'
- re-applied the dataset lineage tab onto the migrated (TypeScript) DatasourceEditor

Closes #37701

Co-authored-by: Jonathan Alberth Quispe Fuentes <qf.jonathan@gmail.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Wrap untranslated "Details" JSX text in t() (fixes pre-commit custom
  rules + babel-extract regression)
- Filter dataset/chart/dashboard lineage endpoints by the current user's
  permissions so they never expose charts, dashboards, or datasource
  metadata the user cannot access (mirrors existing related_objects /
  get_datasets redaction patterns)
- Register DashboardLineageResponseSchema in openapi_spec_component_schemas
  to resolve the dangling $ref in the generated API spec
- Key Sankey graph nodes and the node-details map by a stable unique
  identity (type:id) so entities sharing a display name no longer collapse
  into a single node; keep the human-readable title as a separate label
- Add a skip mechanism to useApiV1Resource / lineage hooks so empty-id
  callers (e.g. LineageModal) no longer fire requests against invalid
  endpoints like /api/v1/chart//lineage
- Defer the dataset edit page lineage fetch until the Lineage tab is active
- Make "View lineage" available in dashboard view mode, not only edit mode
- Compare data["result"] in lineage integration tests to match the
  result-wrapped API contract

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace `Record<string, any>` on additionalInfo with an explicit value
union, and type the Sankey node click handler params instead of `any`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pass '' instead of 0 as the fallback so isEmptyId skips the request
rather than calling /api/v1/dataset/0/lineage.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- mark redacted dashboard lineage dataset fields nullable in the schema
- filter dataset lineage chart_ids by chart access
- responsive Sankey width and fallback dataset->dashboard edges
- empty state for unsaved datasets; i18n legend labels; typed test mocks

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Mark database_name/slug as allow_none in lineage response schemas to
  match the nullable values the endpoints actually return.
- Defer LineageModal's lineage fetch until the modal opens (beforeOpen +
  the hooks' skip flag) so rendering it inside a dropdown no longer fires
  the endpoint on menu open.
- selectinload Slice.dashboards / Dashboard.slices in get_related_objects
  to avoid N+1 queries when building dataset lineage.
- Drop the unreachable dataset branch in getEntityUrl (no standalone
  dataset page) and add type hints to the new lineage locals.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mirrors the dashboard lineage endpoint's gating: schema/table/database
name are only exposed when the caller can access the underlying
datasource, so the chart endpoint no longer leaks that info
unconditionally.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Merges duplicate @superset-ui/core/components import, replaces direct
window.location navigation with the redirect() helper and drops the
hard-coded /superset/ dashboard path per navigationUtils invariants,
and fixes new drill_info RBAC tests calling insert_dataset() with a
nonexistent owners kwarg instead of editor_user_ids.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
These four tests were unrelated to this PR's lineage feature, called
insert_dashboard(roles=...) which no longer exists (roles was replaced
by the Subject-based editors/viewers model upstream), and asserted
DASHBOARD_RBAC-gated behavior that can_drill_dataset_via_dashboard_access
never implements (it only checks EMBEDDED_SUPERSET guest access and
ENABLE_VIEWERS promiscuous mode).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@rusackas
rusackas force-pushed the adopt/lineage-visualization branch from 32877a4 to f75d48d Compare September 4, 2026 02:09

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.10.

Benchmark suite Current: f75d48d Previous: 4a18556 Ratio
spa entrypoint (JS) 9277950 bytes 8332413 bytes 1.11

This comment was automatically generated by workflow using github-action-benchmark.

@github-actions github-actions Bot removed the requires:rebase Requires rebasing on top of current master label Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Related to the REST API change:backend Requires changing the backend change:frontend Requires changing the frontend review:draft size/XXL viz:charts:sankey Related to the Sankey chart 🎪 ⌛ 48h Environment expires after 48 hours (default)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants