a bit of refactoring on utils and pipelines

Author: iakov-gan

.github/workflows/python.yml

@@ -22,4 +22,4 @@ jobs:
           pip install cfn-flip pylint urllib3 boto3 bandit
       - name: Pylint all
         run: |
-          python data-collection/utils/pylint.py
+          python utils/pylint.py

.github/workflows/security-scan.yml

@@ -30,4 +30,4 @@ jobs:
           pip install --no-cache-dir packaging cyclonedx-python-lib=='5.2.0' #https://github.com/bridgecrewio/checkov/issues/5841
       - name: Scan all templates
         run: |
-          data-collection/utils/lint.sh
+          utils/lint.sh

data-collection/CONTRIBUTING.md

@@ -57,19 +57,19 @@ aws s3api create-bucket --bucket $bucket
 
 Cloud Formation:
 ```bash
-./data-collection/utils/lint.sh
+./utils/lint.sh
 ```
 
 Pylint:
 ```bash
-python3 ./data-collection/utils/pylint.py
+python3 ./utils/pylint.py
 ```
 
 
 3. Upload the code to a bucket and run integration tests in your testing environment
 
 ```bash
-./data-collection/test/run-test-from-scratch.sh --no-teardown
+./test/run-test-from-scratch.sh --no-teardown
 ```
 
 The test will install stacks from scratch in a single account, then it will check the presence of Athena tables. After running tests, it will delete the stacks and all artifacts that are not deleted by CFN. You can avoid teardown by providing a flag `--no-teardown`.

data-collection/utils/lint.sh

@@ -7,7 +7,7 @@ GREEN='\033[0;32m'
 YELLOW='\033[0;33m'
 NC='\033[0m' # No Color
 
-folder=$(git rev-parse --show-toplevel)/data-collection/deploy
+folder=$(git rev-parse --show-toplevel)
 success_count=0
 failure_count=0