Skip to content

Switch to CSVImports for Redshift upload #218

@patr1ckm

Description

@patr1ckm

In general:

  1. Add a utility get_headers.
  2. Switch from imports_post to imports_post_files_csv in write_civis.character and write_civis.numeric.
  3. Before uploading, files need to be cleaned withfiles_post_process_csv. Note in the reference PR the steps that are needed to process the cleaning results.
  4. The client will need to be regenerated to access the new endpoints.
  5. Support multiple file ids in write_civis.numeric. If multiple files are uploaded, check that the schema, headers, delimiter, and compression are consistent across files. Upload the files in parallel.
  6. Document the upsert option to if_exists.

See civisanalytics/civis-python#328 for all details.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions