Skip to content

Conversation

jbat
Copy link
Member

@jbat jbat commented Nov 1, 2024

https://jobready.atlassian.net/browse/NEP-18768

  • Add report class to identify data sovereignty issue for a list of dashboards

Example

new_boards = Superset::Dashboard::List.new(title_contains: 'Anledning till avbrott').rows.map(&:first)
=> [113, 114, 112, 111, 110, 109]

Superset::Services::DashboardReport.new(dashboard_ids: new_boards, report_on_data_sovereignty_only: true).perform
D, [2025-04-03T09:09:41.148230 #79232] DEBUG -- : Happi: GET https://ready-superset-eu.jobready.io/api/v1/dashboard/113, {}
D, [2025-04-03T09:09:44.310250 #79232] DEBUG -- : Happi: GET https://ready-superset-eu.jobready.io/api/v1/dashboard/113/datasets, {}
D, [2025-04-03T09:09:46.862876 #79232] DEBUG -- : Happi: GET https://ready-superset-eu.jobready.io/api/v1/dashboard/114, {}
D, [2025-04-03T09:09:49.644554 #79232] DEBUG -- : Happi: GET https://ready-superset-eu.jobready.io/api/v1/dashboard/114/datasets, {}
D, [2025-04-03T09:09:52.347466 #79232] DEBUG -- : Happi: GET https://ready-superset-eu.jobready.io/api/v1/dashboard/112, {}
D, [2025-04-03T09:09:54.883160 #79232] DEBUG -- : Happi: GET https://ready-superset-eu.jobready.io/api/v1/dashboard/112/datasets, {}
D, [2025-04-03T09:09:57.951083 #79232] DEBUG -- : Happi: GET https://ready-superset-eu.jobready.io/api/v1/dashboard/111, {}
D, [2025-04-03T09:10:00.617901 #79232] DEBUG -- : Happi: GET https://ready-superset-eu.jobready.io/api/v1/dashboard/111/datasets, {}
D, [2025-04-03T09:10:03.714419 #79232] DEBUG -- : Happi: GET https://ready-superset-eu.jobready.io/api/v1/dashboard/110, {}
D, [2025-04-03T09:10:07.166893 #79232] DEBUG -- : Happi: GET https://ready-superset-eu.jobready.io/api/v1/dashboard/110/datasets, {}
D, [2025-04-03T09:10:09.890921 #79232] DEBUG -- : Happi: GET https://ready-superset-eu.jobready.io/api/v1/dashboard/109, {}
D, [2025-04-03T09:10:12.798621 #79232] DEBUG -- : Happi: GET https://ready-superset-eu.jobready.io/api/v1/dashboard/109/datasets, {}
Data Sovereignty Report
-----------------------
Possible Invalid Dashboards: 0
=> []


def order_by
# by default, we will order by changed_on as per the GUI
",order_column:changed_on,order_direction:desc"
Copy link
Member Author

@jbat jbat Nov 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this default value is removed in a later commit that sets default to nil
as unfortunately Superset does not have a consistent naming convention for changed_on date
so we can not use this across all child classes

@jbat jbat force-pushed the NEP-18768-create-dashboard-report branch from 00b07f5 to 2c89619 Compare April 2, 2025 22:14
end

# add ERROR msg if multiple chart dataset schemas are found, ie all datasets should be sourced from the same db schema
chart_dataset_schemas = dashboard[:datasets][:chart_datasets].map{|d| d[:schema]}.uniq
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also include the filter datasets here to see if filter dataset schema is different from chart dataset schema ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@soundarya-mv nice pickup, and yes we should.
see last commit

@data_sovereignty_issues ||= begin
@report.map do |dashboard|
reasons = []
chart_dataset_ids = dashboard[:datasets][:chart_datasets].map{|d| d[:id]}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If chart_datasets also contains filter_datasets, then won't the below condition be false always ?

@jbat jbat changed the title NEP-18768 API Dashboard Report NEP-18768 API Compare Dashboard Report Jun 21, 2025
@jbat jbat changed the title NEP-18768 API Compare Dashboard Report API Compare Dashboard Report Jun 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants