updates documentation

Signed-off-by: Nell Shamrell <[email protected]>

Author: nellshamrell

README.md

@@ -7,7 +7,6 @@ development environment for Clearly Defined including:
 * [crawler](https://github.com/clearlydefined/crawler)
 * harvest store (mounted as a volume in the service container)
 * definitions and curations mongo DB databases
-* queues
 
 We do this through [Docker](https://www.docker.com/) and [Docker Compose](https://docs.docker.com/compose/)
 
@@ -58,11 +57,18 @@ services:
     ports:
       - "4000:4000"
     env_file: .env
+    volumes:
+      - ./harvested_data:/tmp/harvested_data/
     links:
       - clearlydefined_mongo_db
+      - crawler
   crawler:
-    context: <path-to-crawler-repo-on-your-system>
+    build:
+      context: <path-to-crawler-repo-on-your-system>
+      dockerfile: DevDockerfile
     env_file: .env
+    volumes:
+      - ./harvested_data:/tmp/harvested_data/
     ports:
       - "5000:5000"
 ```
@@ -89,16 +95,13 @@ And add in appropriate values to the .env file:
 
 (You can use the same GitHub token for both CURATION_GITHUB_TOKEN and CRAWLER_GITHUB_TOKEN)
 
-
 **.env**
 ```
 # Curation GitHub Info
-CURATION_GITHUB_REPO="sample-curated-data"
 CURATION_GITHUB_BRANCH="master"
 CURATION_GITHUB_OWNER="clearlydefined"
 CURATION_GITHUB_REPO="curated-data-dev"
-CURATION_GITHUB_TOKEN="<Your GitHub Personal Access Token>
-
+CURATION_GITHUB_TOKEN="<your GitHub token>"
 
 # Curation Store Info
 CURATION_MONGO_CONNECTION_STRING="mongodb://clearlydefined_mongo_db"
@@ -123,7 +126,12 @@ HARVEST_STORE_PROVIDER="file"
 FILE_STORE_LOCATION="/tmp/harvested_data"
 
 # Crawler Info
-CRAWLER_GITHUB_TOKEN="<Your GitHub Personal Access Token>"
+CRAWLER_API_URL="http://crawler:5000"
+CRAWLER_GITHUB_TOKEN="<your GitHub token>"
+CRAWLER_DEADLETTER_PROVIDER=cd(file)
+CRAWLER_NAME=cdcrawlerlocal
+CRAWLER_QUEUE_PROVIDER=memory
+CRAWLER_STORE_PROVIDER=cd(file)
 ```
 
 Now, from withing your **docker_dev_env_experiment** directory, run:
@@ -214,8 +222,99 @@ The database contains two collections:
 
 The reason the definitions database is called definitions-paged is because, previously, the definitions collection was not paged. The pagination was added in [this January 2019 pull request](https://github.com/clearlydefined/service/pull/364). Our production Azure setup includes both definitions and definitions-paged collections - the definitions-paged collection is the one that is actively used. This development environment includes the definitions-paged collection in order to more closely mirror production.
 
-These collections are seeded using the following container.
+These collections are seeded using the Clearly Defined Mongo Seed container.
+
+If you have [mongodb](https://docs.mongodb.com/manual/installation/) installed on your local system, you can attach to the Mongo database with:
+
+```bash
+$ mongo mongodb://localhost:27017
+```
+
+You can also do this through the [Docker desktop client](https://www.docker.com/products/docker-desktop).
+
 
 ### Clearly Defined Mongo Seed
 
-This container exists only to seed initial data into the Clearly Defined Mongo DB. It populates both the collections and definitions-paged collections with sample data.
+This container exists only to seed initial data into the Clearly Defined Mongo DB. It populates both the collections and definitions-paged collections with sample data.
+
+## Using
+
+As noted above, you can start all of the containers with:
+
+```bash
+$ docker-compose up
+```
+
+This will show all of the logs from all of the container in your current shell.
+
+### Rebuilding after changes
+
+When you make changes to one of the code bases, you do need to rebuild the Docker images.
+
+If you were to make a change to the website, and want to rebuild only that container, you can do so by running:
+
+```bash
+$ docker-compose up --detach --build web
+```
+
+The same will work for the service and the crawler:
+
+```bash
+$ docker-compose up --detach --build service
+```
+
+```bash
+$ docker-compose up --detach --build crawler
+```
+
+### Stopping containers
+
+You can stop and destroy the containers in the shell where you ran `docker-compose up` with CTRL c.
+
+You can also do this through the [Docker desktop client](https://www.docker.com/products/docker-desktop) or through the [Docker CLI](https://docs.docker.com/engine/reference/commandline/cli/).
+
+Note that in order to re-seed the database you must completely destroy the containers (this can also be done through both the Desktop client and the CLI).
+
+### Attaching to a container
+
+You can attach to a container either through using the [Docker desktop client](https://www.docker.com/products/docker-desktop) or the [Docker CLI](https://docs.docker.com/engine/reference/commandline/attach/).
+
+### Running only certain containers
+
+You can choose to run or not run any of the containers listed in [docker-compose.yml]. If you don't wish to run one of the containers, comment it out like this:
+
+**docker-compose.yml**
+```bash
+version: "3.8"
+services:
+#  web:
+#    build:
+#      context: <path-to-website-repo-on-your-system>
+#      dockerfile: DevDockerfile
+#    ports: 
+#      - "3000:3000"
+#    stdin_open: true
+  service:
+    build:
+      context: <path-to-service-repo-on-your-system>
+      dockerfile: DevDockerfile
+    ports:
+      - "4000:4000"
+    env_file: .env
+    volumes:
+      - ./harvested_data:/tmp/harvested_data/
+    links:
+      - clearlydefined_mongo_db
+      - crawler
+  crawler:
+    build:
+      context: <path-to-crawler-repo-on-your-system>
+      dockerfile: DevDockerfile
+    env_file: .env
+    volumes:
+      - ./harvested_data:/tmp/harvested_data/
+    ports:
+      - "5000:5000"
+```
+
+If you run `docker-compose up` after making these changes to the file, it will start all containers except the web container.