From 4803c8a5c1f000f2b6813accd281d9f7562ccfdb Mon Sep 17 00:00:00 2001 From: tschaffter Date: Sat, 5 Jun 2021 15:59:42 -0700 Subject: [PATCH 01/12] Start updating docker-compose.yml --- .../CODE_OF_CONDUCT.md | 0 .github/CONTRIBUTING.md | 135 ++++++++++++++++++ docker-compose.yml | 52 +++++-- 3 files changed, 172 insertions(+), 15 deletions(-) rename CODE_OF_CONDUCT.md => .github/CODE_OF_CONDUCT.md (100%) create mode 100644 .github/CONTRIBUTING.md diff --git a/CODE_OF_CONDUCT.md b/.github/CODE_OF_CONDUCT.md similarity index 100% rename from CODE_OF_CONDUCT.md rename to .github/CODE_OF_CONDUCT.md diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md new file mode 100644 index 0000000..ca5e099 --- /dev/null +++ b/.github/CONTRIBUTING.md @@ -0,0 +1,135 @@ +# Contributing + +A big welcome and thank you for considering contributing to this project. + +This project is a community effort and lives off your contributions, be it in +the form of bug reports, feature requests, discussions, or fixes and other +code changes. + +Reading and following these guidelines will help us make the contribution +process easy and effective for everyone involved. It also communicates that you +agree to respect the time of the developers managing and developing these open +source projects. In return, we will reciprocate that respect by addressing your +issue, assessing changes, and helping you finalize your pull requests. + +## Quicklinks + +- [Code of Conduct](#code-of-conduct) +- [Getting Started](#getting-started) + - [Issues](#issues) + - [Pull Requests](#pull-requests) + - [Setup Development Environment](#setup-development-environment) + - [Testing](#testing) +- [Release Procedure](#release-procedure) +- [Getting Help](#getting-help) + +## Code of Conduct + +We take our open source community seriously and hold ourselves and other +contributors to high standards of communication. By participating and +contributing to this project, you agree to uphold our [Code of Conduct]. + +## Getting Started + +Contributions are made to this repo via Issues and Pull Requests (PRs). A few +general guidelines that cover both: + +- Search for existing Issues and PRs before creating your own. +- We work hard to makes sure issues are handled in a timely manner but, + depending on the impact, it could take a while to investigate the root cause. + A friendly ping in the comment thread to the submitter or a contributor can + help draw attention if your issue is blocking. + +### Issues + +Issues should be used to report problems with this project, request a new +feature, or to discuss potential changes before a PR is created. When you +create a new Issue, a template will be loaded that will guide you through +collecting and providing the information we need to investigate. + +If you find an Issue that addresses the problem you're having, please add your +own reproduction information to the existing issue rather than creating a new +one. Adding a [reaction] can also help be indicating to our maintainers that a +particular problem is affecting more than just the reporter. + +### Pull Requests + +PRs to our repositories are always welcome and can be a quick way to get your +fix or improvement slated for the next release. In general, PRs should: + +- Only fix/add the functionality in question **OR** address wide-spread + whitespace/style issues, not both. +- Add unit or integration tests for fixed or changed functionality + (if a test suite already exists). +- Address a single concern in the least number of changed lines as possible. +- Include documentation in the repo or on our [docs site]. +- Be accompanied by a complete Pull Request template (loaded automatically + when a PR is created). + +For changes that address core functionality or would require breaking changes +(e.g. a major release), it's best to open an Issue to discuss your proposal +first. This is not required but can save time creating and reviewing changes. + +In general, we follow the [Forking Workflow]: + +1. Fork the repository to your own Github account +2. Clone the project to your machine +3. Create a branch locally with a succinct but descriptive name +4. Commit changes to the branch +5. Following any formatting and testing guidelines specific to this repo +6. Push changes to your fork +7. Open a PR in our repository and follow the PR template so that we can + efficiently review the changes. + +We recommend that you add this repository as an [upstream remote] to your local +git repository so that you can fetch the latest updates. + +On your local machine make sure you have the latest version of the `develop` +branch from this upstream repository: + + git checkout develop + git pull upstream develop + +### Setup Development Environment + +This project relies on Node tools and project-specific commands listed in the +file [package.json] to streamline the development and testing of this project. +The command below will install the required development tools. + + npm ci + +### Testing + +Please add tests for new code. These might include unit tests (to test specific +functionality of code that was added to support fixing the bug or feature), +integration tests (to test that the feature is usable - e.g., it should have +complete the expected behavior as reported in the feature request or bug +report), or both. + +Before submitting a PR, please check that the content of the branch that you +plan to submit passes with the tests defined for this project: + + npm run lint + npm run test + +## Release Procedure + +Maintainers are required to follow the procedure below when creating a new +release. + +TBA + +## Getting Help + +Join us on the [NLP Sandbox Discord server] and post your question to the +channel that best matches the topic of your request. + + + +[Code of Conduct]: CODE_OF_CONDUCT.md +[upstream remote]: https://help.github.com/en/articles/configuring-a-remote-for-a-fork +[reaction]: https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/ +[docs site]: https://nlpsandbox.io +[Forking Workflow]: https://www.atlassian.com/git/tutorials/comparing-workflows/forking-workflow +[package.json]: ../package.json +[NLP Sandbox Discord server]: https://nlpsandbox.io/discord diff --git a/docker-compose.yml b/docker-compose.yml index a8891c9..a4af5d6 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -2,28 +2,42 @@ version: "3.8" services: date-annotator: - image: nlpsandbox/date-annotator-example:1.0.1 + image: docker.synapse.org/syn22277123/date-annotator-example:1.1.2 container_name: date-annotator restart: always networks: - - nlp-sandbox-internal + - nlpsandbox-internal person-name-annotator: - image: nlpsandbox/person-name-annotator-example:1.0.1 + image: docker.synapse.org/syn22277123/person-name-annotator-example:1.1.2 container_name: person-name-annotator restart: always networks: - - nlp-sandbox-internal + - nlpsandbox-internal physical-address-annotator: - image: nlpsandbox/physical-address-annotator-example:1.0.1 + image: docker.synapse.org/syn22277123/physical-address-annotator-example:1.1.2 container_name: physical-address-annotator restart: always networks: - - nlp-sandbox-internal + - nlpsandbox-internal + + contact-annotator: + image: docker.synapse.org/syn22277123/contact-annotator-example:1.1.2 + container_name: contact-annotator + restart: always + networks: + - nlpsandbox-internal + + id-annotator: + image: docker.synapse.org/syn22277123/id-annotator-example:1.1.2 + container_name: id-annotator + restart: always + networks: + - nlpsandbox-internal phi-deidentifier: - image: nlpsandbox/phi-deidentifier:1.0.1 + image: docker.synapse.org/syn22277123/phi-deidentifier:1.1.0 container_name: phi-deidentifier restart: always environment: @@ -33,19 +47,27 @@ services: - PERSON_NAME_ANNOTATOR_PORT=8080 - PHYSICAL_ADDRESS_ANNOTATOR_HOST=physical-address-annotator - PHYSICAL_ADDRESS_ANNOTATOR_PORT=8080 + - CONTACT_ANNOTATOR_HOST=contact-annotator + - CONTACT_ANNOTATOR_PORT=8080 + - ID_ANNOTATOR_HOST=id-annotator + - ID_ANNOTATOR_PORT=8080 networks: - - nlp-sandbox-internal + - nlpsandbox-internal depends_on: - date-annotator - person-name-annotator - physical-address-annotator + - contact-annotator + - id-annotator phi-deidentifier-app: - image: nlpsandbox/phi-deidentifier-app:1.0.0 + image: docker.synapse.org/syn22277123/phi-deidentifier-app:1.0.0 build: context: client dockerfile: Dockerfile container_name: phi-deidentifier-app + env_file: + - .env environment: - SERVER_PORT=${SERVER_PORT} - SERVER_PROTOCOL=${SERVER_PROTOCOL} @@ -53,7 +75,7 @@ services: - SERVER_API_PATH=${SERVER_API_PATH} restart: always networks: - - nlp-sandbox-internal + - nlpsandbox-internal depends_on: - phi-deidentifier @@ -70,14 +92,14 @@ services: - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro - ./nginx/templates:/etc/nginx/templates:ro networks: - - nlp-sandbox - - nlp-sandbox-internal + - nlpsandbox + - nlpsandbox-internal ports: - - "80:80" + - "${SERVER_PORT}:80" depends_on: - phi-deidentifier-app networks: - nlp-sandbox: - nlp-sandbox-internal: + nlpsandbox: + nlpsandbox-internal: internal: true From 7e186b7ab959aaf843bccc2d81e3061ab404e461 Mon Sep 17 00:00:00 2001 From: tschaffter Date: Sat, 5 Jun 2021 16:04:51 -0700 Subject: [PATCH 02/12] Update dependabot config --- .github/dependabot.yml | 10 ---------- .github/workflows/lock.yml | 2 +- 2 files changed, 1 insertion(+), 11 deletions(-) diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 9485ddf..4dd6365 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -6,37 +6,27 @@ updates: schedule: interval: "monthly" target-branch: "main" - reviewers: - - "nlpsandbox/maintainers" - package-ecosystem: "docker" directory: "/client" schedule: interval: "monthly" target-branch: "main" - reviewers: - - "nlpsandbox/maintainers" - package-ecosystem: "github-actions" directory: "/" schedule: interval: "monthly" target-branch: "develop" - reviewers: - - "nlpsandbox/maintainers" - package-ecosystem: "npm" directory: "/" schedule: interval: "monthly" target-branch: "main" - reviewers: - - "nlpsandbox/maintainers" - package-ecosystem: "npm" directory: "/client" schedule: interval: "monthly" target-branch: "main" - reviewers: - - "nlpsandbox/maintainers" diff --git a/.github/workflows/lock.yml b/.github/workflows/lock.yml index a1d5c74..13f1619 100644 --- a/.github/workflows/lock.yml +++ b/.github/workflows/lock.yml @@ -1,4 +1,4 @@ -name: 'Lock Threads' +name: Lock threads on: schedule: From 83831247a5164322fd7cb2dc970cd9fae3f55bd6 Mon Sep 17 00:00:00 2001 From: tschaffter Date: Sat, 5 Jun 2021 16:11:55 -0700 Subject: [PATCH 03/12] Update CI workflow --- .github/workflows/ci.yml | 106 ++++++++++++++++++++++++--------------- 1 file changed, 66 insertions(+), 40 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 77db040..a0660ef 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -15,25 +15,10 @@ env: docker_repository: nlpsandbox/phi-deidentifier-app jobs: - build: - env: - working-directory: ./client - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v2 - - name: Install modules - working-directory: ${{env.working-directory}} - run: npm ci - - name: Run ESLint - working-directory: ${{env.working-directory}} - run: npm run lint - docker: - # needs: [test] + configure: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v2 - - name: Prepare - id: prep + - name: Create configuration run: | DOCKER_IMAGE=${{ env.docker_repository }} VERSION=noop @@ -54,27 +39,75 @@ jobs: fi TAGS="${DOCKER_IMAGE}:${VERSION}" if [[ $VERSION =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then - MINOR=${VERSION%.*} - MAJOR=${MINOR%.*} - TAGS="$TAGS,${DOCKER_IMAGE}:${MINOR},${DOCKER_IMAGE}:${MAJOR},${DOCKER_IMAGE}:latest" + TAGS="$TAGS,${DOCKER_IMAGE}:latest" PUSH=true - # elif [ "${{ github.event_name }}" = "push" ]; then - # TAGS="$TAGS,${DOCKER_IMAGE}:sha-${GITHUB_SHA::8}" fi - echo ::set-output name=version::${VERSION} - echo ::set-output name=tags::${TAGS} - echo ::set-output name=created::$(date -u +'%Y-%m-%dT%H:%M:%SZ') - echo ::set-output name=push::${PUSH} + printf "%s\n" \ + "version=${VERSION}" \ + "tags=${TAGS}" \ + "created=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" \ + "push=${PUSH}" | tee -a .github_env + + - name: Upload .github_env + uses: actions/upload-artifact@v2 + with: + name: GITHUB_ENV + path: .github_env + + lint: + needs: [configure] + runs-on: ubuntu-latest + env: + working-directory: ./client + steps: + - uses: actions/checkout@v2 + + - name: Lint Dockerfiles + uses: docker://hadolint/hadolint:latest + with: + entrypoint: hadolint + args: server/Dockerfile + + - name: Validate docker-compose.yml + run: | + docker-compose -f docker-compose.yml config >/dev/null + + - name: Install dependencies + working-directory: ${{env.working-directory}} + run: npm ci + + - name: Run ESLint + working-directory: ${{env.working-directory}} + run: npm run lint + + docker: + needs: [lint] + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v2 + + - name: Download .github_env + uses: actions/download-artifact@v2 + with: + name: GITHUB_ENV + path: . + + - name: Restore $GITHUB_ENV + run: cat .github_env >> $GITHUB_ENV + - name: Set up QEMU uses: docker/setup-qemu-action@v1 + - name: Set up Docker Buildx uses: docker/setup-buildx-action@v1 - - name: Login to DockerHub - if: steps.prep.outputs.push == 'true' + + - name: Login to Docker Hub Registry + if: env.push == 'true' uses: docker/login-action@v1 with: username: ${{ secrets.DOCKERHUB_USERNAME }} - password: ${{ secrets.DOCKERHUB_PASSWORD }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + - name: Build and push id: docker_build uses: docker/build-push-action@v2 @@ -82,18 +115,11 @@ jobs: context: client file: client/Dockerfile platforms: linux/amd64 - push: ${{ steps.prep.outputs.push }} - tags: ${{ steps.prep.outputs.tags }} + push: ${{ env.push }} + tags: ${{ env.tags }} labels: | - org.opencontainers.image.created=${{ steps.prep.outputs.created }} + org.opencontainers.image.created=${{ env.created }} org.opencontainers.image.source=${{ github.repositoryUrl }} - org.opencontainers.image.version=${{ steps.prep.outputs.version }} + org.opencontainers.image.version=${{ env.version }} org.opencontainers.image.revision=${{ github.sha }} org.opencontainers.image.licenses=${{ github.event.repository.license.name }} - - name: Push README to DockerHub - if: steps.prep.outputs.push == 'true' - uses: peter-evans/dockerhub-description@v2 - with: - username: ${{ secrets.DOCKERHUB_USERNAME }} - password: ${{ secrets.DOCKERHUB_PASSWORD }} - repository: ${{ env.docker_repository }} \ No newline at end of file From 6664ed010096fea0d570082e64286cc435c2a9d5 Mon Sep 17 00:00:00 2001 From: tschaffter Date: Sat, 5 Jun 2021 16:15:38 -0700 Subject: [PATCH 04/12] Fix hadlint path to Dockerfile --- .github/workflows/ci.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a0660ef..9ec59ee 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -66,7 +66,7 @@ jobs: uses: docker://hadolint/hadolint:latest with: entrypoint: hadolint - args: server/Dockerfile + args: client/Dockerfile - name: Validate docker-compose.yml run: | From 8d8ad7230956b6a30716cca247c79fc540a6100a Mon Sep 17 00:00:00 2001 From: tschaffter Date: Sat, 5 Jun 2021 16:21:15 -0700 Subject: [PATCH 05/12] Improve Dockerfile --- client/Dockerfile | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/client/Dockerfile b/client/Dockerfile index 2d8c2f4..200e2ec 100644 --- a/client/Dockerfile +++ b/client/Dockerfile @@ -1,13 +1,14 @@ # pull official node base image FROM node:16.2.0-alpine3.12 as build + WORKDIR /app ENV PATH /app/node_modules/.bin:$PATH COPY package.json ./ COPY package-lock.json ./ -RUN npm ci --silent -RUN npm install react-scripts@3.4.1 -g --silent -COPY . ./ -RUN npm run build +COPY . ./ # TODO Either copy file individually or use this line + .gitignore +RUN npm ci --silent \ + && npm install react-scripts@3.4.1 -g --silent \ + && npm run build # production environment FROM nginx:1.21.0-alpine From ae92822ce77167358383c5605e19572f3dc5e133 Mon Sep 17 00:00:00 2001 From: tschaffter Date: Sat, 5 Jun 2021 16:22:17 -0700 Subject: [PATCH 06/12] Improve Dockerfile --- client/Dockerfile | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/client/Dockerfile b/client/Dockerfile index 200e2ec..2842830 100644 --- a/client/Dockerfile +++ b/client/Dockerfile @@ -5,7 +5,10 @@ WORKDIR /app ENV PATH /app/node_modules/.bin:$PATH COPY package.json ./ COPY package-lock.json ./ -COPY . ./ # TODO Either copy file individually or use this line + .gitignore + +# TODO Either copy file individually or use this line + .gitignore +COPY . ./ + RUN npm ci --silent \ && npm install react-scripts@3.4.1 -g --silent \ && npm run build From 7d17c19aecda3b80ef85078e89c3bb10a563a162 Mon Sep 17 00:00:00 2001 From: tschaffter Date: Sat, 5 Jun 2021 16:22:58 -0700 Subject: [PATCH 07/12] Ignore apk package version warning --- client/Dockerfile | 1 + 1 file changed, 1 insertion(+) diff --git a/client/Dockerfile b/client/Dockerfile index 2842830..52aeba7 100644 --- a/client/Dockerfile +++ b/client/Dockerfile @@ -23,6 +23,7 @@ WORKDIR /usr/share/nginx/html COPY env.sh . COPY env_vars . +# hadolint ignore=DL3008 RUN apk add --no-cache bash RUN chmod +x env.sh From f2faa7020c622d04a8944a5988f2de76c3928c0d Mon Sep 17 00:00:00 2001 From: tschaffter Date: Sat, 5 Jun 2021 16:25:02 -0700 Subject: [PATCH 08/12] Ignore apk package version warning --- client/Dockerfile | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/client/Dockerfile b/client/Dockerfile index 52aeba7..681848e 100644 --- a/client/Dockerfile +++ b/client/Dockerfile @@ -23,9 +23,9 @@ WORKDIR /usr/share/nginx/html COPY env.sh . COPY env_vars . -# hadolint ignore=DL3008 -RUN apk add --no-cache bash -RUN chmod +x env.sh +# hadolint ignore=DL3018 +RUN apk add --no-cache bash \ + && chmod +x env.sh EXPOSE 80 CMD ["/bin/bash", "-c", "/usr/share/nginx/html/env.sh && nginx -g \"daemon off;\""] From 3320453e407e47b3c9e8a2c217fa453d55264468 Mon Sep 17 00:00:00 2001 From: tschaffter Date: Sat, 5 Jun 2021 16:26:43 -0700 Subject: [PATCH 09/12] Comment out validation of docker-compose.yml --- .github/workflows/ci.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 9ec59ee..76e3abc 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -68,9 +68,9 @@ jobs: entrypoint: hadolint args: client/Dockerfile - - name: Validate docker-compose.yml - run: | - docker-compose -f docker-compose.yml config >/dev/null + # - name: Validate docker-compose.yml + # run: | + # docker-compose -f docker-compose.yml config >/dev/null - name: Install dependencies working-directory: ${{env.working-directory}} From 90c8bc3dc286a00dd8320c1a54f2dfe7c93a319d Mon Sep 17 00:00:00 2001 From: thomasyu888 Date: Tue, 8 Jun 2021 15:54:22 +0800 Subject: [PATCH 10/12] Add in -example --- docker-compose.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker-compose.yml b/docker-compose.yml index 3240be5..093fa45 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -37,7 +37,7 @@ services: - nlpsandbox-internal phi-deidentifier: - image: docker.synapse.org/syn22277123/phi-deidentifier:1.1.0 + image: docker.synapse.org/syn22277123/phi-deidentifier-example:1.1.0 container_name: phi-deidentifier restart: always environment: From 7b448bf6cf59b0880b74fe7e813527f11898f690 Mon Sep 17 00:00:00 2001 From: Connor Boyle Date: Tue, 30 Nov 2021 12:26:38 -0800 Subject: [PATCH 11/12] Remove Consecutive RUNs in Dockerfile --- client/Dockerfile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/client/Dockerfile b/client/Dockerfile index 2f0607b..5924eca 100644 --- a/client/Dockerfile +++ b/client/Dockerfile @@ -6,8 +6,8 @@ ENV PATH /app/node_modules/.bin:$PATH COPY package.json ./ COPY package-lock.json ./ -RUN npm ci --silent -RUN npm install react-scripts@3.4.1 -g --silent +RUN npm ci --silent \ + && npm install react-scripts@3.4.1 -g --silent COPY src ./src COPY public ./public RUN npm run build From 6fa7b3e3bb6d9ae84dc2fb3273b97e096f916a66 Mon Sep 17 00:00:00 2001 From: Connor Boyle Date: Tue, 30 Nov 2021 12:35:09 -0800 Subject: [PATCH 12/12] 'physical-address' -> 'location' --- docker-compose.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker-compose.yml b/docker-compose.yml index bfc738e..4dab9c1 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -56,7 +56,7 @@ services: depends_on: - date-annotator - person-name-annotator - - physical-address-annotator + - location-annotator - contact-annotator - id-annotator