Skip to content

Fix: Add tags_json to msih_view and make tag columns null-safe#1443

Open
ihartzaws wants to merge 1 commit intoaws-solutions-library-samples:mainfrom
ihartzaws:patch-3
Open

Fix: Add tags_json to msih_view and make tag columns null-safe#1443
ihartzaws wants to merge 1 commit intoaws-solutions-library-samples:mainfrom
ihartzaws:patch-3

Conversation

@ihartzaws
Copy link
Copy Markdown

The Media Services Insights dashboard deployment fails with CONTEXTUAL_UNKNOWN_SYMBOL error on tag_cost_center because the Athena view doesn't include tags_json, which the QuickSight dataset definition depends on.

Changes:

  1. msih_view SQL — Added tags_json column derived from CUR2 resource_tags MAP, using json_format for Athena VARCHAR compatibility: sql IF(cardinality(MAP_FILTER(resource_tags, (k, v) -> v IS NOT NULL AND v <> '')) > 0, json_format(CAST(MAP_FILTER(resource_tags, (k, v) -> v IS NOT NULL AND v <> '') AS JSON)), NULL) "tags_json"

  2. QuickSight dataset definition — Made tag_cost_center and tag_business_unit calculated columns null-safe to handle customers with no tags:

    ifelse(isNull({tags_json}) OR {tags_json} = '{}', NULL, parseJson({tags_json}, '$.tag_cost_center'))

Root cause:

The Athena view (msih_view) did not include tags_json in its SELECT, but the QuickSight dataset unconditionally referenced it via parseJson, causing CreateDataSet to fail for any customer where the view lacked tag data.

Notes:

  • Empty tag maps return NULL instead of '{}' to avoid Athena cast errors on empty JSON objects
  • Athena views don't support JSON column types, so json_format() is used to output VARCHAR
  • The null-safe ifelse in QuickSight handles both NULL and empty '{}' cases

Testing:

Verified against a customer deployment where the original error occurred. Dashboard deploys and SPICE syncs successfully after the fix.

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

The Media Services Insights dashboard deployment fails with CONTEXTUAL_UNKNOWN_SYMBOL error on tag_cost_center because the Athena view doesn't include tags_json, which the QuickSight dataset definition depends on.

Changes:

1. msih_view SQL — Added tags_json column derived from CUR2 resource_tags MAP, using json_format for Athena VARCHAR compatibility:
  sql
   IF(cardinality(MAP_FILTER(resource_tags, (k, v) -> v IS NOT NULL AND v <> '')) > 0,
     json_format(CAST(MAP_FILTER(resource_tags, (k, v) -> v IS NOT NULL AND v <> '') AS JSON)),
     NULL) "tags_json"
   

2. QuickSight dataset definition — Made tag_cost_center and tag_business_unit calculated columns null-safe to handle customers with no tags:
  
   ifelse(isNull({tags_json}) OR {tags_json} = '{}', NULL, parseJson({tags_json}, '$.tag_cost_center'))
   

Root cause:

The Athena view (msih_view) did not include tags_json in its SELECT, but the QuickSight dataset unconditionally referenced it via parseJson, causing CreateDataSet to fail for any customer where the view lacked tag data.

Notes:
- Empty tag maps return NULL instead of '{}' to avoid Athena cast errors on empty JSON objects
- Athena views don't support JSON column types, so json_format() is used to output VARCHAR
- The null-safe ifelse in QuickSight handles both NULL and empty '{}' cases

Testing:

Verified against a customer deployment where the original error occurred. Dashboard deploys and SPICE syncs successfully after the fix.
@ihartzaws
Copy link
Copy Markdown
Author

Update:

The Media Services Insights dashboard deployment fails with CONTEXTUAL_UNKNOWN_SYMBOL error on tag_cost_center because the Athena view doesn't include tags_json, which the QuickSight dataset definition depends on. Additionally, the
CreateColumnsOperation expressions for tag columns don't handle null values, causing failures for customers without tags.

Changes:

  1. msih_view SQL — Added tags_json column derived from CUR2 resource_tags MAP, using json_format for Athena VARCHAR compatibility:
    sql
    IF(cardinality(MAP_FILTER(resource_tags, (k, v) -> v IS NOT NULL AND v <> '')) > 0,
    json_format(CAST(MAP_FILTER(resource_tags, (k, v) -> v IS NOT NULL AND v <> '') AS JSON)),
    NULL) "tags_json"

  2. QuickSight CreateColumnsOperation for tag_cost_center and tag_business_unit — The original Expression in both CreateColumnsOperation blocks did not handle null tags_json values, causing dataset creation to fail for customers without tags.
    Changed from:

    Expression: parseJson(tags_json, '$.tag_cost_center')

To:

Expression: ifelse(isNull({tags_json}) OR {tags_json} = '{}', NULL, parseJson({tags_json}, '$.tag_cost_center'))

Notes:

  • Empty tag maps return NULL to avoid Athena cast errors on empty JSON objects
  • Athena views don't support JSON column types, so json_format() is used to output VARCHAR
  • The null-safe ifelse in the CreateColumnsOperation expressions handles both NULL and empty '{}' cases

Testing:

Verified against a customer deployment where the original errors occurred. Dashboard deploys and SPICE syncs successfully after the fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants