Skip to content

Commit 15d6fd6

Browse files
committed
deploy: 86be501
1 parent a7418df commit 15d6fd6

File tree

3 files changed

+12
-6
lines changed

3 files changed

+12
-6
lines changed

_sources/step_1_configuration.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,10 @@ For more information and examples, see [External Buckets](external_buckets.md).
2828
This is generally the bucket in the account in which you are running compute.
2929
* **SOURCE_BUCKET:** The bucket where the image files you will be reading are.
3030
Often, this is the same as AWS_BUCKET.
31-
* **WORKSPACE:** The bucket where non-image files you will be reading are (e.g. pipeline, load_data.csv, etc.).
31+
These files can be downloaded or read directly off the bucket (see `DOWNLOAD_FILES` below for more).
32+
* **WORKSPACE_BUCKET:** The bucket where non-image files you will be reading are (e.g. pipeline, load_data.csv, etc.).
3233
Often, this is the same as AWS_BUCKET.
34+
Workspace files will always be automatically downloaded to your EC2 instance (as of v2.2.1).
3335
* **DESTINATION_BUCKET:** The bucket where you want to write your output files.
3436
Often, this is the same as AWS_BUCKET.
3537
* **UPLOAD_FLAGS:** If you need to add flags to an AWS CLI command to upload flags to your DESTINATION_BUCKET, this is where you enter them.
@@ -57,6 +59,7 @@ If you have multiple Dockers running per machine, each Docker will have access t
5759
This typically requires a larger EBS volume (depending on the size of your image sets, and how many sets are processed per group), but avoids occasional issues with S3FS that can crop up on longer runs.
5860
By default, DCP uses S3FS to mount the S3 `SOURCE_BUCKET` as a pseudo-file system on each EC2 instance in your spot fleet to avoid file download.
5961
If you are unable to mount the `SOURCE_BUCKET` (perhaps because of a permissions issue) you should proceed with `DOWNLOAD_FILES = 'True'`.
62+
Note that as of v2.2.1, all non-image files (e.g. load_data.csv's and pipelines) are downloaded regardless of this setting and regardless of whether `SOURCE_BUCKET` and `WORKSPACE_BUCKET` are the same.
6063
* **ASSIGN_IP:** Whether or not to assign an a public IPv4 address to each instance in the spot fleet.
6164
If set to 'False' will overwrite whatever is in the Fleet file.
6265
If set to 'True' will respect whatever is in the Fleet file.

0 commit comments

Comments
 (0)