Update to new HCA metadata schema and bundle structure (#70)

* Update requirements and setup.py to install and include the metadata-api library properly.

* Update .ignore files.

* Update dcp_utils and removed unused functions, remove redundant test cases too.

* A big refactor to input_utils so that we can take advantage of the metadata-api library while not using the DSS client.

* Refactor the create_analysis_json so we create analysis_process and analysis_protocol separately.

* Refactor the create_envelope.py to perform_submission.py so we create analysis_process and analysis_protocol separately in the submission envelope.

* Update setup.py, fix styles and update entry_points.

* Update requirements, especially the pytest version.

* Update get_analysis_metadata, so that we can use pipeline version as the analysis protocol_id.

* Update submit WDL, so that it accepts different versions of the analysis process and protocol.

* Update adapter WDL for SS2 pipeline.

* Update adapter WDL for Optimus 3-prime pipeline.

* update the outdated documentation.

* Get rid of the deprecated 10x adapter workflow and its dependencies.

* Update the version of pipeline-tools in adapter wdls.

* Fix an issue that's caused by outdated cromwell-metadata docker image.

* Update the dockerfile of cromwell-metadata image for development.

* Fix issues with installing the hca-metadata-api library in pipeline-tools.

* Remove unused testing fixtures.

* Update Travis CI config for unittests and testing requirements.

* Migrate existing test cases to use pytest, update fixtures and lint with PEP 8.

* Refactor a function in input_utils.py so it's more testable and following the best practices.

* Update test cases for input_utils.py

* Update test cases for get_analysis_metadata.py

* Fix bugs within the submission steps, lint code and respond to comments.

* Update requirements as suggested in comments.

* Update unittest data.

* Add a new parameter cromwell_url to the adapter workflow, change all of the associated places.

* Also change the Optimus while I can still remember it.

* Update the metadata schema version we pin to.

* Update the versions for metadata-api and the docker images everywhere.

Author: rexwangcc

.dockerignore

@@ -1 +1,6 @@
 adapter_pipelines
+.eggs
+.pytest_cache
+pipeline_tools.egg-info
+cromwell_credentials.txt
+caas_key.json

.gitignore

@@ -1,8 +1,9 @@
 # Byte-compiled / optimized / DLL files
 __pycache__/
 *.pyc
-docker/cromwell-metadata/cromwell_credentials.txt
-docker/cromwell-metadata/caas_key.json
+.pytest_cache
+cromwell_credentials.txt
+caas_key.json
 
 # Sphinx documentation
 docs/_build/

.travis.yml

@@ -1,6 +1,5 @@
 language: python
 python:
-- '2.7'
 - '3.6'
 install: pip install -r requirements.txt -r test-requirements.txt
 env:

Dockerfile

@@ -1,13 +1,13 @@
-FROM python:2.7
+FROM python:3.6
 
 LABEL maintainer="Mint Team <mintteam@broadinstitute.org>" \
   software="Python" \
-  description="Python2.7 library used for processing notifications from HCA DCP and doing submissions."
+  description="Python3 library used for processing notifications from HCA DCP and doing submissions."
 
 RUN mkdir /tools
 
 WORKDIR /tools
 
 COPY . .
 
-RUN pip install . --process-dependency-links
+RUN pip install . --process-dependency-links --trusted-host github.com