Formatting and linting improvements. (#887)

* Add black for code formatting. Remove some linting exclusions. Fix errors.

* Fix path.

* Don't install quality_checks package.

* Fix commands.

* Update pre-commit script.

* Add lint-staged to lint and format staged files only.

* Add husky to manage commit hooks.

* Add prettier for MD / YAML / JSON formatting.

* Resolve some Sonar warnings.

* Add .prettierignore and run prettier on all supported files.

* Revert auto formatting of jsc files.

* Fix lint error.

* Format Terraform.

Author: mfjarvis

.editorconfig

@@ -11,18 +11,8 @@ end_of_line = lf
 [Makefile]
 indent_style = tab
 
-[*.{xml,js,json,yaml}]
+[*.{xml,js,json,yaml,yml}]
 indent_size = 2
 
 [*.postman_collection.json]
 indent_style = tab
-
-[*.yaml]
-trim_trailing_whitespace = false
-
-[*.js]
-ij_javascript_force_semicolon_style = true
-ij_javascript_use_semicolon_after_statement = false
-
-[*.md]
-trim_trailing_whitespace = false

.github/ISSUE_TEMPLATE/bug_report.md

@@ -1,17 +1,17 @@
 ---
 name: Bug report
 about: Create a report to help us improve
-title: ''
-labels: ''
-assignees: ''
-
+title: ""
+labels: ""
+assignees: ""
 ---
 
 **Describe the bug**
 A clear and concise description of what the bug is.
 
 **To Reproduce**
 Steps to reproduce the behavior:
+
 1. Go to '...'
 2. Click on '....'
 3. Scroll down to '....'
@@ -24,15 +24,17 @@ A clear and concise description of what you expected to happen.
 If applicable, add screenshots to help explain your problem.
 
 **Desktop (please complete the following information):**
- - OS: [e.g. iOS]
- - Browser: [e.g. chrome, safari]
- - Version: [e.g. 22]
+
+- OS: [e.g. iOS]
+- Browser: [e.g. chrome, safari]
+- Version: [e.g. 22]
 
 **Smartphone (please complete the following information):**
- - Device: [e.g. iPhone6]
- - OS: [e.g. iOS8.1]
- - Browser: [e.g. stock browser, safari]
- - Version: [e.g. 22]
+
+- Device: [e.g. iPhone6]
+- OS: [e.g. iOS8.1]
+- Browser: [e.g. stock browser, safari]
+- Version: [e.g. 22]
 
 **Additional context**
 Add any other context about the problem here.

.github/ISSUE_TEMPLATE/feature_request.md

@@ -1,10 +1,9 @@
 ---
 name: Feature request
 about: Suggest an idea for this project
-title: ''
-labels: ''
-assignees: ''
-
+title: ""
+labels: ""
+assignees: ""
 ---
 
 **Is your feature request related to a problem? Please describe.**

.github/pull_request_template.md

@@ -1,23 +1,24 @@
 ## Summary
-* Routine Change
-* :exclamation: Breaking Change
-* :robot: Operational or Infrastructure Change
-* :sparkles: New Feature
-* :warning: Potential issues that might be caused by this change
 
-Add any other relevant notes or explanations here. **Remove this line if you have nothing to add.**
+- Routine Change
+- :exclamation: Breaking Change
+- :robot: Operational or Infrastructure Change
+- :sparkles: New Feature
+- :warning: Potential issues that might be caused by this change
 
+Add any other relevant notes or explanations here. **Remove this line if you have nothing to add.**
 
 ## Reviews Required
-* [x] Dev
-* [ ] Test
-* [ ] Tech Author
-* [ ] Product Owner
 
+- [x] Dev
+- [ ] Test
+- [ ] Tech Author
+- [ ] Product Owner
 
 ## Review Checklist
+
 :information_source: This section is to be filled in by the **reviewer**.
 
-* [ ] I have reviewed the changes in this PR and they fill all or part of the acceptance criteria of the ticket, and the code is in a mergeable state.
-* [ ] If there were infrastructure, operational, or build changes, I have made sure there is sufficient evidence that the changes will work.
-* [ ] I have ensured the changelog has been updated by the submitter, if necessary.
+- [ ] I have reviewed the changes in this PR and they fill all or part of the acceptance criteria of the ticket, and the code is in a mergeable state.
+- [ ] If there were infrastructure, operational, or build changes, I have made sure there is sufficient evidence that the changes will work.
+- [ ] I have ensured the changelog has been updated by the submitter, if necessary.

.github/workflows/create-release-tag.yml

@@ -12,15 +12,15 @@ jobs:
       - name: Checkout
         uses: actions/checkout@v5
         with:
-          fetch-depth: 0  # This causes all history to be fetched, which is required for calculate-version to function	
+          fetch-depth: 0 # This causes all history to be fetched, which is required for calculate-version to function
 
       - name: Install Python 3.9
         uses: actions/setup-python@v6
         with:
           python-version: 3.9
 
       - name: Upgrade python pip
-        run: python -m pip install --upgrade pip 
+        run: python -m pip install --upgrade pip
 
       - name: Install git
         run: pip install gitpython

.github/workflows/deploy-backend.yml

@@ -136,11 +136,11 @@ jobs:
         if: ${{ inputs.environment == 'dev' && inputs.create_mns_subscription }}
         with:
           python-version: 3.11
-          cache: 'poetry'
+          cache: "poetry"
 
       - name: Create MNS Subscription
         if: ${{ inputs.environment == 'dev' && inputs.create_mns_subscription }}
-        working-directory: './lambdas/mns_subscription'
+        working-directory: "./lambdas/mns_subscription"
         env:
           APIGEE_ENVIRONMENT: ${{ inputs.apigee_environment }}
           SQS_ARN: ${{ env.ID_SYNC_QUEUE_ARN }}

.github/workflows/pr-teardown.yml

@@ -23,7 +23,7 @@ jobs:
     permissions:
       id-token: write
       contents: read
-  
+
     steps:
       - name: Connect to AWS
         uses: aws-actions/configure-aws-credentials@a03048d87541d1d9fcf2ecf528a4a65ba9bd7838
@@ -37,7 +37,7 @@ jobs:
 
       - name: Checkout
         uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8
-        
+
       - uses: hashicorp/setup-terraform@b9cd54a3c349d3f38e8881555d616ced269862dd
         with:
           terraform_version: "1.12.2"
@@ -55,10 +55,10 @@ jobs:
       - uses: actions/setup-python@v6
         with:
           python-version: 3.11
-          cache: 'poetry'
+          cache: "poetry"
 
       - name: Unsubscribe MNS
-        working-directory: './lambdas/mns_subscription'
+        working-directory: "./lambdas/mns_subscription"
         env:
           SQS_ARN: ${{ env.ID_SYNC_QUEUE_ARN }}
         run: |

.github/workflows/quality-checks.yml

@@ -12,8 +12,26 @@ env:
   LAMBDA_PATH: ${{ github.workspace }}/lambdas
 
 jobs:
-  lint:
-    name: Lint specification and Python projects
+  lint-specification:
+    name: Lint specification
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v5
+
+      - uses: actions/setup-node@v5
+        with:
+          node-version: "23.11.0"
+          cache: "npm"
+
+      - name: Install linting dependencies
+        run: make install-node
+
+      - name: Lint
+        run: make lint
+
+  lint-python:
+    name: Lint Python projects
     runs-on: ubuntu-latest
 
     steps:
@@ -22,23 +40,22 @@ jobs:
       - name: Install poetry
         run: pip install poetry==2.1.4
 
-      # Base linting requires 3.8 due to APIM package dependencies. See root README for details under linting.
-      # Consider upgrading this and poetry deps if we move away from Azure DevOps to using the Proxygen tool.
       - uses: actions/setup-python@v6
         with:
-          python-version: 3.8
-          cache: 'poetry'
-
-      - uses: actions/setup-node@v5
-        with:
-          node-version: '23.11.0'
-          cache: 'npm'
+          python-version: 3.11
+          cache: "poetry"
 
       - name: Install linting dependencies
-        run: make install-node && poetry install --no-root
+        run: poetry install --no-root
+        working-directory: quality_checks
 
       - name: Lint
-        run: make lint
+        run: poetry run make lint
+        working-directory: quality_checks
+
+      - name: Check formatting
+        run: poetry run make format-check
+        working-directory: quality_checks
 
   testcoverage_and_sonarcloud:
     name: Test Coverage and SonarCloud
@@ -53,7 +70,7 @@ jobs:
       - uses: actions/setup-python@v6
         with:
           python-version: 3.11
-          cache: 'poetry'
+          cache: "poetry"
 
       - name: Set up AWS credentials
         env:
@@ -110,7 +127,7 @@ jobs:
         working-directory: delta_backend
         id: delta
         env:
-            PYTHONPATH: delta_backend/src:delta_backend/tests
+          PYTHONPATH: delta_backend/src:delta_backend/tests
         continue-on-error: true
         run: |
           poetry install
@@ -144,9 +161,9 @@ jobs:
           PYTHONPATH: ${{ env.LAMBDA_PATH }}/ack_backend/src:${{ github.workspace }}/ack_backend/tests
         continue-on-error: true
         run: |
-            poetry install
-            poetry run coverage run --source=src -m unittest discover || echo "ack-lambda tests failed" >> ../../failed_tests.txt
-            poetry run coverage xml -o ../../ack-lambda-coverage.xml
+          poetry install
+          poetry run coverage run --source=src -m unittest discover || echo "ack-lambda tests failed" >> ../../failed_tests.txt
+          poetry run coverage xml -o ../../ack-lambda-coverage.xml
 
       - name: Run unittest with coverage-mns-subscription
         working-directory: lambdas/mns_subscription

.husky/pre-commit

@@ -0,0 +1 @@
+npx lint-staged

.prettierignore

@@ -0,0 +1,7 @@
+# This is a template, not a valid YAML file
+/manifest_template.yml
+
+# We hit compile errors in Apigee if these are auto formatted
+# TODO - investigate
+/proxies/live/apiproxy/resources/jsc/
+/proxies/sandbox/apiproxy/resources/jsc/