Our import script can import training documents

[dcollie2]

Description

Our import currently does not handle training documents. We need it to process the cme_file_obfuscated.csv file, associate each file with the appropriate resource, and upload it to our app's storage. This could be handled in our data_import script but given the differences it probably should be developed separately.

Running this import should not in any way alter the files stored in Azure.

We will use the Azure interface Dmitry developed to read the files for importing into S3.

See requirements.

When writing this, look for any places where we can and should alter created_at and updated_at dates to reflect the dates when the files were uploaded. We have not finished our synching requirements yet, so if the answers here are not obvious, please do not worry about them. We will handle them in a follow-up task once we know the details of synching with Azure.

Dependencies

Dependent on #30, #104, and #118

Tasks

• Training documents are imported from import_files/cme_file_obfuscated.csv
• data_import:import_all triggers the file import
• data_import:destroy_all_data removes the file associations and deletes the files from storage (dependent: destroy on Topics has_many_attached)

[dcollie2]

Some notes:

• When cycling through the CSV, you will need to look up each topic using the topic_id
• Use the topic to determine where to find the file:
• Language and state will determine location
• If it's English, it will be in CMES-Pi/assets/Content
• If it's Spanish, it will be in SP_CMES-Pi/assets/Content
• If it's archived, it will be the appropriate language archive directory at the root. So: CMES-Pi_Archive