Skip to content

Commit df2dee3

Browse files
committed
use s3 and original excel files for CBBR source data
1 parent 2b0ede5 commit df2dee3

File tree

2 files changed

+13
-7
lines changed

2 files changed

+13
-7
lines changed

ingest_templates/dcp_cbbr_requests.yml

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,10 +17,13 @@ attributes:
1717
1818
ingestion:
1919
source:
20-
type: local_file
21-
path: ./dcp_cbbr_requests.csv
20+
type: s3
21+
bucket: edm-recipes
22+
key: inbox/dcp/dcp_cbbr_requests/{{ version }} BRs for OMB (All Boards).xlsx
2223
file_format:
23-
type: csv
24+
type: xlsx
25+
sheet_name: OMB Template
26+
dtype: str
2427
processing_steps:
2528
- name: clean_column_names
26-
args: { "replace": { " -": "", "-": "_", " ": "_" }, "lower": True }
29+
args: { "replace": { "-": "_", " ": "_" }, "lower": True }

ingest_templates/omb_cbbr_agency_responses.yml

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,10 +15,13 @@ attributes:
1515
1616
ingestion:
1717
source:
18-
type: local_file
19-
path: ./omb_cbbr_agency_responses.csv
18+
type: s3
19+
bucket: edm-recipes
20+
key: inbox/dcp/omb_cbbr_agency_responses/{{ version }} OMB Raw Export.xlsx
2021
file_format:
21-
type: csv
22+
type: xlsx
23+
sheet_name: AGCY reponses
24+
dtype: str
2225
processing_steps:
2326
- name: clean_column_names
2427
args: { "replace": { " ": "_" }, "lower": True }

0 commit comments

Comments
 (0)