Skip to content

Commit 117e44d

Browse files
scott-lewis-nsidcscott-lewis-nsidc
authored
Merge pull request #14 from nsidc/scott-work
Initial work to update docker stack and reverse proxy
2 parents 82c6e8c + 630d7d6 commit 117e44d

File tree

10 files changed

+225
-14
lines changed

10 files changed

+225
-14
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,6 @@ jobs:
4949
--durations=20
5050
5151
- name: Upload coverage report
52-
uses: codecov/codecov-action@v4.5.0
52+
uses: codecov/codecov-action@v5.1.2
5353
with:
5454
token: ${{ secrets.CODECOV_TOKEN }}

‎Dockerfile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ FROM python:3.12-alpine
22

33
# git: for vcs-awareness during install
44
# build-base, etc.: for building jupyterlab deps
5-
RUN apk add git build-base musl-dev linux-headers
5+
RUN apk add git build-base musl-dev linux-headers gdal-dev
66

77
WORKDIR /app
88
ADD . .

‎README.md‎

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ for more details!**
3737
Set up the development compose configuration to be automatically loaded:
3838

3939
```bash
40-
ln -s compose.dev.yml compose.override.dev.yml
40+
ln -s compose.dev.yml compose.override.yml
4141
```
4242

4343

@@ -116,6 +116,10 @@ The stack is configured within `compose.yml` and includes containers:
116116
* `aross-stations-admin`: An [Adminer](https://www.adminer.org/) container for
117117
inspecting the database in the browser.
118118
* `aross-stations-api`: An HTTP API for accessing data in the database.
119+
* `aross-stations-ui` : The front-end with map and user interactivity for accessing data
120+
121+
The UI and API containers are served behind a traefik reverse proxy, allowing for SSL,
122+
as well as having them to use the same port and path structure.
119123

120124
```bash
121125
docker compose --profile ui up --pull=always --detach
@@ -215,7 +219,15 @@ services.
215219

216220
### View UI
217221

218-
Navigate to `http://localhost:80`.
222+
Navigate to `https://localhost/apps/aross-stations`. (Uses port 443)
223+
224+
### Access API
225+
226+
Use `https://localhost/api/aross-stations/v1/...`
227+
228+
The API itself runs on port 8000 of its container, but with the `traefik` reverse proxy
229+
you can just use the same port (but with the `/api/aross-stations` path prefix) and it
230+
will direct the traffic there.
219231

220232

221233
### Experiment in JupyterLab
@@ -224,6 +236,8 @@ This repository provides a demo notebook to experiment with the API. In your bro
224236
navigate to `http://localhost:8888`. The password is the same as the database password
225237
you set earlier.
226238

239+
Note: This container is NOT handled by the traefix reverse proxy; you can just access it
240+
using the non-secure port shown here.
227241

228242
## Cleanup
229243

‎compose.dev.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,8 @@ services:
1010
api:
1111
<<: *dev-common
1212
command: ["dev", "--host", "0.0.0.0", "./src/aross_stations_db/api"]
13+
# ports:
14+
# - "8000:8000"
1315
# NOTE: In place of the above command, which uses the image's default
1416
# `fastapi` entrypoint, you can run the container with no server (using
1517
# "sleep" instead):
@@ -24,6 +26,6 @@ services:
2426

2527
cli:
2628
<<: *dev-common
27-
29+
2830
jupyterlab:
2931
<<: *dev-common

‎compose.integration.yml‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
2+
x-dev-common: &dev-common
3+
image: "nsidc/aross-stations-db:dev"
4+
build: "."
5+
volumes:
6+
- "${PWD}:/app"
7+
8+
9+
services:
10+
ui:
11+
labels:
12+
- "traefik.enable=true"
13+
14+
- "traefik.http.routers.aross-ui.rule=(Host(`integration.aross-stations.apps.int.nsidc.org`) || Host(`integration.nsidc.org`)) && PathPrefix(`/apps/aross-stations`)"
15+
- "traefik.http.routers.aross-ui.entrypoints=websecure"
16+
- "traefik.http.routers.aross-ui.tls=true"
17+
- "traefik.http.routers.aross-ui.tls.certresolver=letsencrypt"
18+
- "traefik.http.routers.aross-ui.middlewares=aross-ui"
19+
- "traefik.http.routers.aross-ui.service=aross-ui"
20+
- "traefik.http.middlewares.aross-ui.stripprefix.prefixes=/apps/aross-stations"
21+
- "traefik.http.services.aross-ui.loadbalancer.server.port=80"
22+
23+
24+
api:
25+
<<: *dev-common
26+
command: ["dev", "--host", "0.0.0.0", "./src/aross_stations_db/api"]
27+
28+
labels:
29+
- "traefik.enable=true"
30+
31+
# local
32+
- "traefik.http.routers.aross-api.rule=(Host(`integration.aross-stations.apps.int.nsidc.org`) || Host(`integration.nsidc.org`)) && PathPrefix(`/api/aross-stations`)"
33+
- "traefik.http.routers.aross-api.entrypoints=websecure"
34+
- "traefik.http.routers.aross-api.tls=true"
35+
- "traefik.http.routers.aross-api.tls.certresolver=letsencrypt"
36+
- "traefik.http.routers.aross-api.middlewares=aross-api"
37+
- "traefik.http.routers.aross-api.service=aross-api"
38+
- "traefik.http.middlewares.aross-api.stripprefix.prefixes=/api/aross-stations"
39+
- "traefik.http.services.aross-api.loadbalancer.server.port=8000"
40+
41+
42+
cli:
43+
<<: *dev-common
44+
45+
jupyterlab:
46+
<<: *dev-common

‎compose.qa.yml‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
2+
x-dev-common: &dev-common
3+
image: "nsidc/aross-stations-db:dev"
4+
build: "."
5+
volumes:
6+
- "${PWD}:/app"
7+
8+
9+
services:
10+
ui:
11+
labels:
12+
- "traefik.enable=true"
13+
14+
- "traefik.http.routers.aross-ui.rule=(Host(`qa.aross-stations.apps.int.nsidc.org`) || Host(`qa.nsidc.org`)) && PathPrefix(`/apps/aross-stations`)"
15+
- "traefik.http.routers.aross-ui.entrypoints=websecure"
16+
- "traefik.http.routers.aross-ui.tls=true"
17+
- "traefik.http.routers.aross-ui.tls.certresolver=letsencrypt"
18+
- "traefik.http.routers.aross-ui.middlewares=aross-ui"
19+
- "traefik.http.routers.aross-ui.service=aross-ui"
20+
- "traefik.http.middlewares.aross-ui.stripprefix.prefixes=/apps/aross-stations"
21+
- "traefik.http.services.aross-ui.loadbalancer.server.port=80"
22+
23+
24+
api:
25+
<<: *dev-common
26+
command: ["dev", "--host", "0.0.0.0", "./src/aross_stations_db/api"]
27+
28+
labels:
29+
- "traefik.enable=true"
30+
31+
# local
32+
- "traefik.http.routers.aross-api.rule=(Host(`qa.aross-stations.apps.int.nsidc.org`) || Host(`qa.nsidc.org`)) && PathPrefix(`/api/aross-stations`)"
33+
- "traefik.http.routers.aross-api.entrypoints=websecure"
34+
- "traefik.http.routers.aross-api.tls=true"
35+
- "traefik.http.routers.aross-api.tls.certresolver=letsencrypt"
36+
- "traefik.http.routers.aross-api.middlewares=aross-api"
37+
- "traefik.http.routers.aross-api.service=aross-api"
38+
- "traefik.http.middlewares.aross-api.stripprefix.prefixes=/api/aross-stations"
39+
- "traefik.http.services.aross-api.loadbalancer.server.port=8000"
40+
41+
42+
cli:
43+
<<: *dev-common
44+
45+
jupyterlab:
46+
<<: *dev-common

‎compose.staging.yml‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
2+
x-dev-common: &dev-common
3+
image: "nsidc/aross-stations-db:dev"
4+
build: "."
5+
volumes:
6+
- "${PWD}:/app"
7+
8+
9+
services:
10+
ui:
11+
labels:
12+
- "traefik.enable=true"
13+
14+
- "traefik.http.routers.aross-ui.rule=(Host(`staging.aross-stations.apps.int.nsidc.org`) || Host(`staging.nsidc.org`)) && PathPrefix(`/apps/aross-stations`)"
15+
- "traefik.http.routers.aross-ui.entrypoints=websecure"
16+
- "traefik.http.routers.aross-ui.tls=true"
17+
- "traefik.http.routers.aross-ui.tls.certresolver=letsencrypt"
18+
- "traefik.http.routers.aross-ui.middlewares=aross-ui"
19+
- "traefik.http.routers.aross-ui.service=aross-ui"
20+
- "traefik.http.middlewares.aross-ui.stripprefix.prefixes=/apps/aross-stations"
21+
- "traefik.http.services.aross-ui.loadbalancer.server.port=80"
22+
23+
24+
api:
25+
<<: *dev-common
26+
command: ["dev", "--host", "0.0.0.0", "./src/aross_stations_db/api"]
27+
28+
labels:
29+
- "traefik.enable=true"
30+
31+
# local
32+
- "traefik.http.routers.aross-api.rule=(Host(`staging.aross-stations.apps.int.nsidc.org`) || Host(`staging.nsidc.org`)) && PathPrefix(`/api/aross-stations`)"
33+
- "traefik.http.routers.aross-api.entrypoints=websecure"
34+
- "traefik.http.routers.aross-api.tls=true"
35+
- "traefik.http.routers.aross-api.tls.certresolver=letsencrypt"
36+
- "traefik.http.routers.aross-api.middlewares=aross-api"
37+
- "traefik.http.routers.aross-api.service=aross-api"
38+
- "traefik.http.middlewares.aross-api.stripprefix.prefixes=/api/aross-stations"
39+
- "traefik.http.services.aross-api.loadbalancer.server.port=8000"
40+
41+
42+
cli:
43+
<<: *dev-common
44+
45+
jupyterlab:
46+
<<: *dev-common

‎compose.yml‎

Lines changed: 60 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,12 +8,54 @@ x-common: &common
88
x-image: &image "nsidc/aross-stations-db"
99

1010
services:
11+
traefik:
12+
image: traefik:v3.0
13+
container_name: traefik
14+
command:
15+
- --entrypoints.web.address=:80
16+
- --entrypoints.web.http.redirections.entryPoint.to=websecure
17+
- --entrypoints.web.http.redirections.entryPoint.scheme=https
18+
- --entrypoints.websecure.address=:443
19+
- --entrypoints.api.address=:8000
20+
- --certificatesresolvers.letsencrypt.acme.tlschallenge=true
21+
- --certificatesresolvers.letsencrypt.acme.email=scott.lewis@colorado.edu
22+
- --certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json
23+
- --providers.docker=true
24+
- --log.level=INFO
25+
ports:
26+
- "8000:8000"
27+
- "80:80"
28+
- "443:443"
29+
volumes:
30+
- /var/run/docker.sock:/var/run/docker.sock:ro
31+
- ./config:/letsencrypt
32+
labels:
33+
- "traefik.enable=true"
34+
restart: unless-stopped
35+
1136
ui:
1237
container_name: "aross-stations-ui"
1338
depends_on: ["api"]
1439
image: "nsidc/aross-stations-ui:latest"
15-
ports:
16-
- "80:80"
40+
labels:
41+
- "traefik.enable=true"
42+
43+
- "traefik.http.routers.aross-ui.rule=Host(`localhost`) && PathPrefix(`/apps/aross-stations`)"
44+
- "traefik.http.routers.aross-ui.entrypoints=websecure"
45+
- "traefik.http.routers.aross-ui.tls=true"
46+
- "traefik.http.routers.aross-ui.tls.certresolver=letsencrypt"
47+
- "traefik.http.routers.aross-ui.middlewares=aross-ui"
48+
- "traefik.http.routers.aross-ui.service=aross-ui"
49+
- "traefik.http.middlewares.aross-ui.stripprefix.prefixes=/apps/aross-stations"
50+
- "traefik.http.services.aross-ui.loadbalancer.server.port=80"
51+
52+
# Production domain (nsidc.org)
53+
# - "traefik.http.routers.aross-prod.rule=Host(`nsidc.org`) && PathPrefix(`/apps/aross`)"
54+
# - "traefik.http.routers.aross-prod.entrypoints=websecure"
55+
# - "traefik.http.routers.aross-prod.tls.certresolver=letsencrypt"
56+
# - "traefik.http.routers.aross-prod.middlewares=strip-aross-path"
57+
58+
restart: unless-stopped
1759
profiles: ["ui"]
1860

1961

@@ -25,8 +67,20 @@ services:
2567

2668
entrypoint: "fastapi"
2769
command: ["run", "--host", "0.0.0.0", "./src/aross_stations_db/api"]
28-
ports:
29-
- "8000:8000"
70+
71+
labels:
72+
- "traefik.enable=true"
73+
74+
# local
75+
- "traefik.http.routers.aross-api.rule=Host(`localhost`) && PathPrefix(`/api/aross-stations`)"
76+
- "traefik.http.routers.aross-api.entrypoints=websecure"
77+
- "traefik.http.routers.aross-api.tls=true"
78+
- "traefik.http.routers.aross-api.tls.certresolver=letsencrypt"
79+
- "traefik.http.routers.aross-api.middlewares=aross-api"
80+
- "traefik.http.routers.aross-api.service=aross-api"
81+
- "traefik.http.middlewares.aross-api.stripprefix.prefixes=/api/aross-stations"
82+
- "traefik.http.services.aross-api.loadbalancer.server.port=8000"
83+
3084
environment:
3185
AROSS_DB_CONNSTR: null
3286

@@ -40,6 +94,8 @@ services:
4094
POSTGRES_DB: "aross"
4195
POSTGRES_USER: "aross"
4296
POSTGRES_PASSWORD: null
97+
ports:
98+
- "5432:5432"
4399
volumes:
44100
- "./_data:/var/lib/postgresql/data"
45101

‎src/aross_stations_db/api/__init__.py‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
"Rain on snow events detected by"
99
" Automated Surface Observing System (ASOS) stations"
1010
),
11+
openapi_url="/api/aross-stations/openapi.json"
1112
)
1213
api.include_router(v1_router, prefix="/v1", tags=["v1"])
1314
api.add_middleware(
@@ -20,7 +21,7 @@
2021

2122

2223
@api.get("/")
23-
def get() -> dict[str, str]:
24+
def get_root() -> dict[str, str]:
2425
return {
25-
"Hello": "The root of this API doesn't do anything. Please check out '/docs'!"
26+
"Hello": "The root of this API doesn't do anything. Please check out '/docs' or something!"
2627
}

‎src/aross_stations_db/config.py‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ class Settings(BaseSettings):
1616

1717
DB_CONNSTR: PostgresDsn
1818

19-
@computed_field # type:ignore[misc]
19+
@computed_field # type: ignore[prop-decorator]
2020
@cached_property
2121
def db_engine(self) -> Engine:
2222
return create_engine(str(self.DB_CONNSTR))
@@ -33,12 +33,12 @@ class CliLoadSettings(Settings):
3333
# TODO: Specifically ignore this type of error instead of using type-ignore; but
3434
# mypy doesn't yet categorize this error in its own type, so we need to wait for a
3535
# release, likely 1.11: https://github.com/python/mypy/pull/16571/files
36-
@computed_field # type:ignore[misc]
36+
@computed_field # type: ignore[prop-decorator]
3737
@cached_property
3838
def events_dir(self) -> DirectoryPath:
3939
return self.DATA_BASEDIR / "events"
4040

41-
@computed_field # type:ignore[misc]
41+
@computed_field # type: ignore[prop-decorator]
4242
@cached_property
4343
def stations_metadata_filepath(self) -> FilePath:
4444
return self.DATA_BASEDIR / "metadata" / "aross.asos_stations.metadata.csv"

0 commit comments

Comments
 (0)