Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
3c6050a
get an acceptance test working that include an e2e run of the pipeline
eveleighoj Feb 8, 2026
dfe511c
add pre-commit config
eveleighoj Feb 8, 2026
7f7ed94
lint and format code, improve makefile so that it can be used from wi…
eveleighoj Feb 8, 2026
7b4d6d7
correct error in how centroid is calculated
eveleighoj Feb 8, 2026
a2b1f34
remove unused functions
eveleighoj Feb 17, 2026
e65ec27
remove unwanted function
eveleighoj Feb 17, 2026
b12bdcc
run test on all branches except main
eveleighoj Feb 17, 2026
91b7715
migrate to a new way of working forcing users to setup a .venv
eveleighoj Feb 17, 2026
596aa0b
correct black version for python 3.9
eveleighoj Feb 17, 2026
3f5c07a
run black after locking version
eveleighoj Feb 17, 2026
ca4a1e9
fix flake8 errors
eveleighoj Feb 17, 2026
ce8cc00
clear up poor code quality
eveleighoj Feb 18, 2026
113e900
add pandas back to local requirements, may be needed in production re…
eveleighoj Feb 18, 2026
21064c0
big cleanup and add logging
eveleighoj Feb 19, 2026
c8e2f37
add additiona tests to cofnirm presence of qualitty column in entity
eveleighoj Feb 19, 2026
a0249da
remove load-type
eveleighoj Feb 20, 2026
84cbe92
fix s3 error and add tests
eveleighoj Feb 20, 2026
b4589de
correct issue pipeline as after the separation its missing the column…
eveleighoj Feb 21, 2026
95cbbef
improve path handling to reduce errors
eveleighoj Feb 21, 2026
ca934dc
fix clean_up_datasets code and alter writing code
eveleighoj Feb 22, 2026
8e3234a
add some extra docs forr adding futurre pipelines
eveleighoj Feb 22, 2026
97730df
correct geojson production
eveleighoj Feb 23, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .flake8
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[flake8]
max-line-length = 180
ignore = W291, E203, W503, E501, F401, F841, F821, F402, F811
ignore = E203, W503, E501
per-file-ignores =
examples/*:F401,F821,F841,E501,E402
*reconciliation*:F401,F821,F841,E501,E402,E226
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ jobs:

- name: Build artifacts
run: |
bash build_aws_package.sh
bash bin/build_aws_package.sh

- name: Upload build artifacts
uses: actions/upload-artifact@v4
Expand Down
4 changes: 3 additions & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,15 @@ run-name: Test - ${{ github.head_ref || github.ref_name }} by @${{ github.actor

on:
push:
branches: [main]
branches-ignore: [main]
workflow_dispatch:
workflow_call:

jobs:
lint:
runs-on: ubuntu-latest
env:
ENVIRONMENT: test
steps:
- uses: actions/checkout@v4
with:
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,7 @@ dependencies.tar.gz
# Configuration files with potentially sensitive data
*config*.json
*config*.yaml
!.pre-commit-config.yaml
*config*.yml
application.properties
spark-defaults.conf
Expand Down
9 changes: 9 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
repos:
- repo: https://github.com/psf/black
rev: 25.11.0
hooks:
- id: black
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.3.0
hooks:
- id: flake8
Loading