Skip to content

Commit 8b9d192

Browse files
authored
BUG: ignore manifest lines that do not contain s3 URLs (#139)
This came up while trying to use manifests generated using Looker Studio
1 parent 417ffe0 commit 8b9d192

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

idc_index/index.py

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -836,7 +836,9 @@ def _validate_update_manifest_and_get_download_size(
836836
REGEXP_EXTRACT(manifest_cp_cmd, '(?:.*?\\/){{3}}([^\\/?#]+)', 1) AS manifest_crdc_series_uuid,
837837
REGEXP_EXTRACT(manifest_cp_cmd, 's3://\\S+') AS s3_url,
838838
FROM
839-
manifest_df )
839+
manifest_df
840+
WHERE
841+
REGEXP_EXTRACT(manifest_cp_cmd, 's3://\\S+') IS NOT NULL)
840842
SELECT
841843
seriesInstanceuid,
842844
index_crdc_series_uuid,

0 commit comments

Comments
 (0)