Skip to content

Commit 93cbcbd

Browse files
committed
Ingest: pull dcas ipis from inbox
1 parent beb7f27 commit 93cbcbd

File tree

1 file changed

+6
-3
lines changed

1 file changed

+6
-3
lines changed

ingest_templates/dcas_ipis.yml

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,11 +16,14 @@ attributes:
1616
1717
ingestion:
1818
source:
19-
type: local_file
20-
path: ./tmp/upload/COLPDATA.csv
19+
type: s3
20+
bucket: edm-recipes
21+
key: inbox/dcas/ipis/{{ version }}/dcas_ipis.csv
2122
file_format:
2223
type: csv
23-
encoding: ISO-8859-1
24+
# encoding: ISO-8859-1
25+
# I think we needed the above in the past, but including it now results in an invalid column header: bbl
26+
# Leaving it here just in case this weird behavior is reintroduced
2427
processing_steps:
2528
- name: clean_column_names
2629
args: {"replace": {" ": "_", "-": "_"}, "lower": True}

0 commit comments

Comments
 (0)