Skip to content

Commit 62761bc

Browse files
robbibtrobbibt
authored
Update packaging (#144)
* Add code for preprocessing ELVIS paths * Add experimental eodatasets code * Add all validation notebooks * Add all modified files * Clean notebook * Update requirements * Point tests to new path * Add intertidal change notebook * Add OWS styling notebook * Add recent changes * Update ensemble code, relocate notebooks * Clean tide gauge notebook * Update for FES2022 * Add updated notebooks * Add cost distance notebook * Update notebook to latest version * Update * Update notebooks * Add experimental extents notebook * Incremental changes * Update clipping notebook * Update changes * Update tide corr rankings notebook * Clean and prep notebook * Update supplementary notebooks * Update requirements * Update scripts to use eo-tides * Update notebooks * Update product def * Update to latest versions * Update notebooks * Fix Datacube 1.9 bug * Formatting * Update pixel function * Update to match unstable Sandbox * Update elevation notebook * Remove not needed files * Automatically update integration test validation results * Add pyproject.toml, clean files * Fix package install * Test using uv for Dockerfile setup * Add Git * Update lockfile * Update Dockerfile to include build-essential and libpq-dev for psycopg2 installation * Fix formatting * Add python3-pip * Attempt to fix * Try adding package to path * Remove system command * Try different path * Fix missing entrypoints * Use uv to run tests * Try using system Python * Test what is available * Revert * Try to use existing venv * Try to avoid overwriting app * Remove unnecessary files, add uv.lock * Pin DEA Tools * Update lockfile * Automatically update integration test validation results * Remove old metadata file * Fix merge issue * Automatically update integration test validation results --------- Co-authored-by: robbibt <robbibt@users.noreply.github.com>
1 parent 3086602 commit 62761bc

36 files changed

+8817
-21869
lines changed

‎.github/workflows/dea-intertidal-image-dev.yml‎

Lines changed: 0 additions & 103 deletions
This file was deleted.

‎.github/workflows/dea-intertidal-image.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ jobs:
8181
timeout-minutes: 30
8282
shell: bash
8383
run: |
84-
docker build -t dea_intertidal --build-arg REQUIREMENTS_IN=requirements.in .
84+
docker build -t dea_intertidal .
8585
8686
- name: Run integration tests
8787
run: |
@@ -101,9 +101,9 @@ jobs:
101101
--env AWS_ACCESS_KEY_ID \
102102
--env AWS_SECRET_ACCESS_KEY \
103103
--env AWS_SESSION_TOKEN \
104-
--volume ${GITHUB_WORKSPACE}:/app \
104+
--volume ${GITHUB_WORKSPACE}/tests:/app/tests \
105105
--volume ${GITHUB_WORKSPACE}/tide_models:/var/share/tide_models \
106-
dea_intertidal pytest -v --cov=intertidal --cov-report=xml tests
106+
dea_intertidal uv run pytest -v --cov=intertidal --cov-report=xml tests
107107
108108
# Commit validation results produced by integration tests back into repo
109109
# (relies on having previously added a volume to access outputs outside of the container)

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@
88
!*.sh
99
!*.yaml
1010
!*.yml
11+
!*.toml
12+
!*.lock
1113
!*.in
1214
!*.txt
1315
!**.github/workflows

‎Dockerfile‎

Lines changed: 25 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -4,39 +4,32 @@
44
# - GDAL 3.7.3, released 2023/10/30
55
FROM ghcr.io/osgeo/gdal:ubuntu-small-3.7.3
66

7-
ENV DEBIAN_FRONTEND=noninteractive \
8-
LC_ALL=C.UTF-8 \
9-
LANG=C.UTF-8
10-
11-
# Apt installation
12-
RUN apt-get update && \
13-
apt-get install -y \
14-
build-essential \
15-
git \
16-
python3-pip \
17-
libpq-dev \
18-
&& apt-get autoclean && \
19-
apt-get autoremove && \
20-
rm -rf /var/lib/{apt,dpkg,cache,log}
21-
22-
# Set up working directory
7+
# curl and certificates are required to download uv,
8+
# build-essential, libpq-dev and python3-dev are needed for psycopg2,
9+
# git is needed for hatchling version control
10+
RUN apt-get update && apt-get install -y --no-install-recommends \
11+
curl \
12+
ca-certificates \
13+
build-essential \
14+
libpq-dev \
15+
python3-dev \
16+
git && \
17+
rm -rf /var/lib/apt/lists/*
18+
19+
# Download, run and remove uv installer, and ensure it is on path
20+
ADD https://astral.sh/uv/0.8.22/install.sh /uv-installer.sh
21+
RUN sh /uv-installer.sh && rm /uv-installer.sh
22+
ENV PATH="/root/.local/bin/:$PATH"
23+
24+
# Copy project files into image, and set as working directory
25+
ADD . /app
2326
WORKDIR /app
2427

25-
# Accept build-time argument for requirements file
26-
ARG REQUIREMENTS_IN=requirements.in
27-
28-
# Copy requirements file first
29-
COPY ${REQUIREMENTS_IN} /app/requirements.in
30-
31-
# Install uv and requirements
32-
RUN pip install uv && \
33-
uv pip compile /app/requirements.in -o /app/requirements.txt --emit-find-links && \
34-
uv pip install -r /app/requirements.txt --system
28+
# Sync project into a new virtual environment based on uv.lock
29+
RUN uv sync --locked --extra datacube
3530

36-
# Now copy the rest of the files
37-
COPY . /app
31+
# Make uv virtual environment accessible
32+
ENV PATH="/app/.venv/bin:$PATH"
3833

39-
# Install DEA Intertidal with postgres extra and verify installation
40-
RUN uv pip install .[postgres] --system && \
41-
uv pip check && \
42-
dea-intertidal --help
34+
# Verify installation
35+
RUN uv pip check && dea-intertidal --help

‎data/raw/tide_correlation_points_input.geojson‎

Lines changed: 63 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{
22
"type": "FeatureCollection",
3+
"name": "tide_correlation_points_input",
34
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:OGC:1.3:CRS84" } },
45
"features": [
56
{ "type": "Feature", "properties": { "id": 0 }, "geometry": { "type": "Point", "coordinates": [ 136.774801932213336, -14.176371834023156 ] } },
@@ -1685,7 +1686,7 @@
16851686
{ "type": "Feature", "properties": { "id": 1680 }, "geometry": { "type": "Point", "coordinates": [ 139.534508608537919, -16.410002898051516 ] } },
16861687
{ "type": "Feature", "properties": { "id": 1681 }, "geometry": { "type": "Point", "coordinates": [ 134.247169671512069, -11.990908055922581 ] } },
16871688
{ "type": "Feature", "properties": { "id": 1682 }, "geometry": { "type": "Point", "coordinates": [ 125.304407009058565, -15.07371860596518 ] } },
1688-
{ "type": "Feature", "properties": { "id": 1683 }, "geometry": { "type": "Point", "coordinates": [ 117.729516684433207, -20.680647080573543 ] } },
1689+
{ "type": "Feature", "properties": { "id": 3000 }, "geometry": { "type": "Point", "coordinates": [ 145.404767084759186, -38.374788580360807 ] } },
16891690
{ "type": "Feature", "properties": { "id": 1684 }, "geometry": { "type": "Point", "coordinates": [ 149.347455024535179, -37.775665668686017 ] } },
16901691
{ "type": "Feature", "properties": { "id": 1685 }, "geometry": { "type": "Point", "coordinates": [ 142.249573344477056, -10.9089616827903 ] } },
16911692
{ "type": "Feature", "properties": { "id": 1686 }, "geometry": { "type": "Point", "coordinates": [ 136.262056823313799, -13.45126658368744 ] } },
@@ -3001,6 +3002,66 @@
30013002
{ "type": "Feature", "properties": { "id": 2996 }, "geometry": { "type": "Point", "coordinates": [ 129.183090987411106, -14.981958110707408 ] } },
30023003
{ "type": "Feature", "properties": { "id": 2997 }, "geometry": { "type": "Point", "coordinates": [ 136.685704428846549, -11.824890686651131 ] } },
30033004
{ "type": "Feature", "properties": { "id": 2998 }, "geometry": { "type": "Point", "coordinates": [ 124.707594388604122, -15.830218240321535 ] } },
3004-
{ "type": "Feature", "properties": { "id": 2999 }, "geometry": { "type": "Point", "coordinates": [ 148.347020608424913, -39.948653977801442 ] } }
3005+
{ "type": "Feature", "properties": { "id": 2999 }, "geometry": { "type": "Point", "coordinates": [ 148.347020608424913, -39.948653977801442 ] } },
3006+
{ "type": "Feature", "properties": { "id": 3001 }, "geometry": { "type": "Point", "coordinates": [ 145.541279842960876, -38.386908871976836 ] } },
3007+
{ "type": "Feature", "properties": { "id": 3002 }, "geometry": { "type": "Point", "coordinates": [ 145.372233670421366, -38.260602675135992 ] } },
3008+
{ "type": "Feature", "properties": { "id": 3003 }, "geometry": { "type": "Point", "coordinates": [ 145.42773184782115, -38.44878615022715 ] } },
3009+
{ "type": "Feature", "properties": { "id": 3004 }, "geometry": { "type": "Point", "coordinates": [ 145.753384946241624, -38.637288580360845 ] } },
3010+
{ "type": "Feature", "properties": { "id": 3005 }, "geometry": { "type": "Point", "coordinates": [ 147.53761945413973, -42.785777340993377 ] } },
3011+
{ "type": "Feature", "properties": { "id": 3006 }, "geometry": { "type": "Point", "coordinates": [ 148.091484885488541, -39.860162213410867 ] } },
3012+
{ "type": "Feature", "properties": { "id": 3007 }, "geometry": { "type": "Point", "coordinates": [ 150.676934460215563, -35.06850060952182 ] } },
3013+
{ "type": "Feature", "properties": { "id": 3008 }, "geometry": { "type": "Point", "coordinates": [ 153.412930815015528, -27.73987059736988 ] } },
3014+
{ "type": "Feature", "properties": { "id": 3009 }, "geometry": { "type": "Point", "coordinates": [ 153.373380389742124, -27.674484813651762 ] } },
3015+
{ "type": "Feature", "properties": { "id": 3010 }, "geometry": { "type": "Point", "coordinates": [ 153.410698129717844, -27.637486028718584 ] } },
3016+
{ "type": "Feature", "properties": { "id": 3011 }, "geometry": { "type": "Point", "coordinates": [ 153.174033488162479, -27.432716891415996 ] } },
3017+
{ "type": "Feature", "properties": { "id": 3012 }, "geometry": { "type": "Point", "coordinates": [ 153.093656817445577, -27.259205348281096 ] } },
3018+
{ "type": "Feature", "properties": { "id": 3013 }, "geometry": { "type": "Point", "coordinates": [ 153.236867631540434, -27.396356016567871 ] } },
3019+
{ "type": "Feature", "properties": { "id": 3014 }, "geometry": { "type": "Point", "coordinates": [ 153.404637983909851, -27.420596599799957 ] } },
3020+
{ "type": "Feature", "properties": { "id": 3015 }, "geometry": { "type": "Point", "coordinates": [ 153.120289563496698, -27.04598390235153 ] } },
3021+
{ "type": "Feature", "properties": { "id": 3016 }, "geometry": { "type": "Point", "coordinates": [ 152.505503719025228, -25.007382748037688 ] } },
3022+
{ "type": "Feature", "properties": { "id": 3017 }, "geometry": { "type": "Point", "coordinates": [ 149.855625225707598, -22.431820779628989 ] } },
3023+
{ "type": "Feature", "properties": { "id": 3018 }, "geometry": { "type": "Point", "coordinates": [ 149.933450256084285, -22.163898543905976 ] } },
3024+
{ "type": "Feature", "properties": { "id": 3019 }, "geometry": { "type": "Point", "coordinates": [ 149.664571155233688, -21.976671933942395 ] } },
3025+
{ "type": "Feature", "properties": { "id": 3020 }, "geometry": { "type": "Point", "coordinates": [ 149.86168537151562, -22.001231472217 ] } },
3026+
{ "type": "Feature", "properties": { "id": 3021 }, "geometry": { "type": "Point", "coordinates": [ 149.714966051953041, -21.448801338559559 ] } },
3027+
{ "type": "Feature", "properties": { "id": 3022 }, "geometry": { "type": "Point", "coordinates": [ 149.899959976618931, -21.01310875046714 ] } },
3028+
{ "type": "Feature", "properties": { "id": 3023 }, "geometry": { "type": "Point", "coordinates": [ 146.008070547699731, -18.190356624099707 ] } },
3029+
{ "type": "Feature", "properties": { "id": 3024 }, "geometry": { "type": "Point", "coordinates": [ 146.141712710518675, -18.245216891414415 ] } },
3030+
{ "type": "Feature", "properties": { "id": 3025 }, "geometry": { "type": "Point", "coordinates": [ 143.129102595086806, -12.233711727379349 ] } },
3031+
{ "type": "Feature", "properties": { "id": 3026 }, "geometry": { "type": "Point", "coordinates": [ 141.871143907358714, -12.707997875617593 ] } },
3032+
{ "type": "Feature", "properties": { "id": 3027 }, "geometry": { "type": "Point", "coordinates": [ 142.00191547479497, -12.7255404029566 ] } },
3033+
{ "type": "Feature", "properties": { "id": 3028 }, "geometry": { "type": "Point", "coordinates": [ 136.720338925583832, -15.758483902349582 ] } },
3034+
{ "type": "Feature", "properties": { "id": 3029 }, "geometry": { "type": "Point", "coordinates": [ 136.794336495450182, -15.73296749894739 ] } },
3035+
{ "type": "Feature", "properties": { "id": 3030 }, "geometry": { "type": "Point", "coordinates": [ 136.934038804077176, -15.881600548765155 ] } },
3036+
{ "type": "Feature", "properties": { "id": 3031 }, "geometry": { "type": "Point", "coordinates": [ 136.937228354502452, -15.690865433333775 ] } },
3037+
{ "type": "Feature", "properties": { "id": 3032 }, "geometry": { "type": "Point", "coordinates": [ 136.650168816227819, -15.673641861037295 ] } },
3038+
{ "type": "Feature", "properties": { "id": 3033 }, "geometry": { "type": "Point", "coordinates": [ 136.705666993627574, -15.888617559700759 ] } },
3039+
{ "type": "Feature", "properties": { "id": 3034 }, "geometry": { "type": "Point", "coordinates": [ 136.606153020359017, -15.70681318546014 ] } },
3040+
{ "type": "Feature", "properties": { "id": 3035 }, "geometry": { "type": "Point", "coordinates": [ 136.550973798001792, -15.644616952167301 ] } },
3041+
{ "type": "Feature", "properties": { "id": 3036 }, "geometry": { "type": "Point", "coordinates": [ 136.83101632534084, -15.55530954025963 ] } },
3042+
{ "type": "Feature", "properties": { "id": 3037 }, "geometry": { "type": "Point", "coordinates": [ 136.54969797783167, -15.532982687282713 ] } },
3043+
{ "type": "Feature", "properties": { "id": 3038 }, "geometry": { "type": "Point", "coordinates": [ 131.619928840528331, -12.286020354353845 ] } },
3044+
{ "type": "Feature", "properties": { "id": 3039 }, "geometry": { "type": "Point", "coordinates": [ 131.688823129714251, -12.286020354353845 ] } },
3045+
{ "type": "Feature", "properties": { "id": 3040 }, "geometry": { "type": "Point", "coordinates": [ 131.750062497879526, -12.273262152652748 ] } },
3046+
{ "type": "Feature", "properties": { "id": 3041 }, "geometry": { "type": "Point", "coordinates": [ 132.057535158875908, -12.306433477075597 ] } },
3047+
{ "type": "Feature", "properties": { "id": 3042 }, "geometry": { "type": "Point", "coordinates": [ 132.426885098122625, -12.178851460064642 ] } },
3048+
{ "type": "Feature", "properties": { "id": 3043 }, "geometry": { "type": "Point", "coordinates": [ 132.64122288670103, -11.825449272944287 ] } },
3049+
{ "type": "Feature", "properties": { "id": 3044 }, "geometry": { "type": "Point", "coordinates": [ 131.292043056810172, -11.720513063952779 ] } },
3050+
{ "type": "Feature", "properties": { "id": 3045 }, "geometry": { "type": "Point", "coordinates": [ 131.498088014282871, -11.472684995908994 ] } },
3051+
{ "type": "Feature", "properties": { "id": 3046 }, "geometry": { "type": "Point", "coordinates": [ 131.24866517102646, -12.172472359214098 ] } },
3052+
{ "type": "Feature", "properties": { "id": 3047 }, "geometry": { "type": "Point", "coordinates": [ 131.056654235424958, -12.160352067598053 ] } },
3053+
{ "type": "Feature", "properties": { "id": 3048 }, "geometry": { "type": "Point", "coordinates": [ 123.023771489372521, -12.280917073673402 ] } },
3054+
{ "type": "Feature", "properties": { "id": 3049 }, "geometry": { "type": "Point", "coordinates": [ 123.111803081110082, -12.278046478290655 ] } },
3055+
{ "type": "Feature", "properties": { "id": 3050 }, "geometry": { "type": "Point", "coordinates": [ 144.764385098124848, -40.694070172098755 ] } },
3056+
{ "type": "Feature", "properties": { "id": 3051 }, "geometry": { "type": "Point", "coordinates": [ 144.889734429838114, -40.658666162378211 ] } },
3057+
{ "type": "Feature", "properties": { "id": 3052 }, "geometry": { "type": "Point", "coordinates": [ 144.875062497881856, -40.473034327627268 ] } },
3058+
{ "type": "Feature", "properties": { "id": 3053 }, "geometry": { "type": "Point", "coordinates": [ 144.649880237857502, -38.302864218270884 ] } },
3059+
{ "type": "Feature", "properties": { "id": 3054 }, "geometry": { "type": "Point", "coordinates": [ 144.713671246362992, -38.14211087683708 ] } },
3060+
{ "type": "Feature", "properties": { "id": 3055 }, "geometry": { "type": "Point", "coordinates": [ 144.357717418902411, -38.134455955816421 ] } },
3061+
{ "type": "Feature", "properties": { "id": 3056 }, "geometry": { "type": "Point", "coordinates": [ 145.183173068963299, -38.398550731029104 ] } },
3062+
{ "type": "Feature", "properties": { "id": 3057 }, "geometry": { "type": "Point", "coordinates": [ 137.795536373943804, -32.867870293604071 ] } },
3063+
{ "type": "Feature", "properties": { "id": 3058 }, "geometry": { "type": "Point", "coordinates": [ 133.633810979046359, -32.282268835523766 ] } },
3064+
{ "type": "Feature", "properties": { "id": 3059 }, "geometry": { "type": "Point", "coordinates": [ 150.859934915865892, -23.468903100406809 ] } },
3065+
{ "type": "Feature", "properties": { "id": 3060 }, "geometry": { "type": "Point", "coordinates": [ 150.925001744541504, -23.553107231634041 ] } }
30053066
]
30063067
}

‎dev-requirements.in‎

Lines changed: 0 additions & 38 deletions
This file was deleted.

0 commit comments

Comments
 (0)