Skip to content

Commit f300973

Browse files
vincentsaragovincentsarago
authored
switch docker compose file to default to official runtimes (#152)
1 parent adcb499 commit f300973

File tree

7 files changed

+367
-296
lines changed

7 files changed

+367
-296
lines changed

‎.github/workflows/ci.yml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ jobs:
3838
pre-commit run --all-files
3939
4040
- name: Launch services
41-
run: docker compose up -d stac raster vector
41+
run: docker compose -f docker-compose.custom.yml --profile gunicorn up -d
4242

4343
- name: install lib postgres
4444
run: |
@@ -67,8 +67,8 @@ jobs:
6767
# see https://github.com/developmentseed/tipg/issues/37
6868
- name: Restart the Vector service
6969
run: |
70-
docker compose stop vector
71-
docker compose up -d vector
70+
docker compose -f docker-compose.custom.yml --profile gunicorn stop vector
71+
docker compose -f docker-compose.custom.yml --profile gunicorn up -d vector
7272
7373
- name: Sleep for 10 seconds
7474
run: sleep 10s
@@ -78,7 +78,7 @@ jobs:
7878
run: python -m pytest .github/workflows/tests/
7979

8080
- name: Stop services
81-
run: docker compose stop
81+
run: docker compose -f docker-compose.custom.yml stop
8282

8383

8484
publish-docker:

‎README.md

Lines changed: 60 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -60,35 +60,80 @@ Then you can start exploring your dataset with:
6060

6161
If you've added a vector dataset to the `public` schema in the Postgres database, they will be available through the **Vector** service at [http://localhost:8083](http://localhost:8083).
6262

63-
Alternatively, you may launch the application locally:
63+
## Deployment
64+
65+
This repository has current runtimes that are consistently updated with new functionality.
66+
67+
The services can be deployed locally via docker with `docker compose up`.
68+
69+
Two Infrastructure as Code (IaC) repositories are available:
70+
- [eoapi-cdk](https://github.com/developmentseed/eoapi-cdk): A set of AWS CDK constructs to deploy eoAPI services
71+
- [eoapi-k8s](https://github.com/developmentseed/eoapi-k8s): IaC and Helm charts for deploying eoAPI services on AWS and GCP
72+
73+
Finally, [eoapi-template](https://github.com/developmentseed/eoapi-template) is an AWS CDK app that shows how to configure the eoapi-cdk constructs.
74+
75+
Alternatively, you may install the libraries locally:
76+
77+
<details>
78+
6479
```bash
6580
python -m pip install --upgrade virtualenv
6681
virtualenv .venv
6782
source .venv/bin/activate
6883

69-
python -m pip install "psycopg[binary,pool]" uvicorn
70-
python -m pip install runtime/eoapi/{SERVICE} # SERVICE should be one of `raster, vector, stac.`
71-
7284
export DATABASE_URL=postgresql://username:[email protected]:5439/postgis # Connect to the database of your choice
7385

74-
.venv/bin/uvicorn eoapi.{SERVICE}.app:app --port 8000 --reload
86+
python -m pip install uvicorn
87+
88+
###############################################################################
89+
# Install and launch the application
90+
# Select one of the following
91+
92+
###############################################################################
93+
# STAC
94+
python -m pip install "psycopg[binary,pool]" stac-fastapi-pgstac
95+
.venv/bin/uvicorn stac_fastapi.pgstac.app:app --port 8081 --reload
96+
97+
###############################################################################
98+
# RASTER
99+
python -m pip install "psycopg[binary,pool]" titiler-pgstac
100+
.venv/bin/uvicorn titiler.pgstac.main:app --port 8082 --reload
101+
102+
###############################################################################
103+
# VECTOR
104+
python -m pip install tipg
105+
.venv/bin/uvicorn tipg.main:app --port 8083 --reload
75106
```
76107

77-
Note: services might have incompatible dependencies, which you can resolve by using a virtual environment for each service
108+
Note: python libraries might have incompatible dependencies, which you can resolve by using a virtual environment for each ones
78109

79-
---
110+
</details>
80111

81-
## Deployment
112+
## Custom runtimes
82113

83-
This repository has current runtimes that are consistently updated with new functionality.
114+
The eoAPI repository hosts customized versions of each base service which can work in parallel or in combination with each other.
84115

85-
The services can be deployed locally via docker with `docker-compose up`. The official runtimes can be launched with `docker compose -f docker-compose.official.yml up stac-fastapi titiler-pgstac tipg`.
116+
eoAPI custom runtimes can be launched with docker
86117

87-
Two Infrastructure as Code (IaC) repositories are available:
88-
- [eoapi-cdk](https://github.com/developmentseed/eoapi-cdk): A set of AWS CDK constructs to deploy eoAPI services
89-
- [eoapi-k8s](https://github.com/developmentseed/eoapi-k8s): IaC and Helm charts for deploying eoAPI services on AWS and GCP
118+
```
119+
docker compose -f docker-compose.custom.yml --profile gunicorn up
120+
```
90121

91-
Finally, [eoapi-template](https://github.com/developmentseed/eoapi-template) is an AWS CDK app that shows how to configure the eoapi-cdk constructs.
122+
Alternatively, you may launch the application locally:
123+
```bash
124+
python -m pip install --upgrade virtualenv
125+
virtualenv .venv
126+
source .venv/bin/activate
127+
128+
python -m pip install "psycopg[binary,pool]" uvicorn
129+
python -m pip install runtime/eoapi/{SERVICE} # SERVICE should be one of `raster, vector, stac.`
130+
131+
export DATABASE_URL=postgresql://username:[email protected]:5439/postgis # Connect to the database of your choice
132+
133+
.venv/bin/uvicorn eoapi.{SERVICE}.app:app --port 8000 --reload
134+
```
135+
136+
Note: services might have incompatible dependencies, which you can resolve by using a virtual environment for each service
92137

93138
## Contribution & Development
94139

@@ -113,4 +158,4 @@ See [contributors](https://github.com/developmentseed/eoAPI/graphs/contributors)
113158

114159
## Changes
115160

116-
See [CHANGES.md](https://github.com/developmentseed/eoAPI/blob/main/CHANGES.md).
161+
See [CHANGES.md](https://github.com/developmentseed/eoAPI/blob/main/CHANGES.md).

‎docker-compose.custom.yml

Lines changed: 264 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,264 @@
1+
version: '3'
2+
3+
services:
4+
stac:
5+
container_name: eoapi.stac
6+
profiles:
7+
- gunicorn
8+
build:
9+
context: .
10+
dockerfile: dockerfiles/Dockerfile.stac
11+
ports:
12+
- "${MY_DOCKER_IP:-127.0.0.1}:8081:8081"
13+
environment:
14+
- APP_HOST=0.0.0.0
15+
- APP_PORT=8081
16+
- HOST=0.0.0.0
17+
- PORT=8081
18+
- ENVIRONMENT=local
19+
# https://github.com/tiangolo/uvicorn-gunicorn-docker#web_concurrency
20+
- WEB_CONCURRENCY=10
21+
# https://github.com/tiangolo/uvicorn-gunicorn-docker#workers_per_core
22+
# - WORKERS_PER_CORE=1
23+
# https://github.com/tiangolo/uvicorn-gunicorn-docker#max_workers
24+
# - MAX_WORKERS=10
25+
# Postgres connection
26+
- POSTGRES_USER=username
27+
- POSTGRES_PASS=password
28+
- POSTGRES_DBNAME=postgis
29+
- POSTGRES_HOST_READER=database
30+
- POSTGRES_HOST_WRITER=database
31+
- POSTGRES_PORT=5432
32+
- DB_MIN_CONN_SIZE=1
33+
- DB_MAX_CONN_SIZE=10
34+
# https://github.com/developmentseed/eoAPI/issues/16
35+
# - TITILER_ENDPOINT=raster
36+
- TITILER_ENDPOINT=http://127.0.0.1:8082
37+
depends_on:
38+
- database
39+
- raster
40+
command:
41+
bash -c "bash /tmp/scripts/wait-for-it.sh -t 120 -h database -p 5432 && /start.sh"
42+
volumes:
43+
- ./dockerfiles/scripts:/tmp/scripts
44+
45+
raster:
46+
container_name: eoapi.raster
47+
profiles:
48+
- gunicorn
49+
build:
50+
context: .
51+
dockerfile: dockerfiles/Dockerfile.raster
52+
# At the time of writing, rasterio and psycopg wheels are not available for arm64 arch
53+
# so we force the image to be built with linux/amd64
54+
platform: linux/amd64
55+
ports:
56+
- "${MY_DOCKER_IP:-127.0.0.1}:8082:8082"
57+
environment:
58+
# Application
59+
- HOST=0.0.0.0
60+
- PORT=8082
61+
# https://github.com/tiangolo/uvicorn-gunicorn-docker#web_concurrency
62+
- WEB_CONCURRENCY=1
63+
# https://github.com/tiangolo/uvicorn-gunicorn-docker#workers_per_core
64+
- WORKERS_PER_CORE=1
65+
# https://github.com/tiangolo/uvicorn-gunicorn-docker#max_workers
66+
- MAX_WORKERS=10
67+
# Postgres connection
68+
- POSTGRES_USER=username
69+
- POSTGRES_PASS=password
70+
- POSTGRES_DBNAME=postgis
71+
- POSTGRES_HOST=database
72+
- POSTGRES_PORT=5432
73+
- DB_MIN_CONN_SIZE=1
74+
- DB_MAX_CONN_SIZE=10
75+
# - DB_MAX_QUERIES=10
76+
# - DB_MAX_IDLE=10
77+
# GDAL Config
78+
- CPL_TMPDIR=/tmp
79+
- GDAL_CACHEMAX=75%
80+
- GDAL_INGESTED_BYTES_AT_OPEN=32768
81+
- GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR
82+
- GDAL_HTTP_MERGE_CONSECUTIVE_RANGES=YES
83+
- GDAL_HTTP_MULTIPLEX=YES
84+
- GDAL_HTTP_VERSION=2
85+
- VSI_CACHE=TRUE
86+
- VSI_CACHE_SIZE=536870912
87+
# TiTiler Config
88+
- MOSAIC_CONCURRENCY=1
89+
# AWS S3 endpoint config
90+
- AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
91+
- AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
92+
# API Config
93+
- EOAPI_RASTER_ENABLE_MOSAIC_SEARCH=TRUE
94+
depends_on:
95+
- database
96+
command:
97+
bash -c "bash /tmp/scripts/wait-for-it.sh -t 120 -h database -p 5432 && /start.sh"
98+
volumes:
99+
- ./dockerfiles/scripts:/tmp/scripts
100+
101+
vector:
102+
container_name: eoapi.vector
103+
profiles:
104+
- gunicorn
105+
build:
106+
context: .
107+
dockerfile: dockerfiles/Dockerfile.vector
108+
ports:
109+
- "${MY_DOCKER_IP:-127.0.0.1}:8083:8083"
110+
environment:
111+
# Application
112+
- HOST=0.0.0.0
113+
- PORT=8083
114+
# https://github.com/tiangolo/uvicorn-gunicorn-docker#web_concurrency
115+
- WEB_CONCURRENCY=10
116+
# https://github.com/tiangolo/uvicorn-gunicorn-docker#workers_per_core
117+
# - WORKERS_PER_CORE=1
118+
# https://github.com/tiangolo/uvicorn-gunicorn-docker#max_workers
119+
# - MAX_WORKERS=10
120+
# Postgres connection
121+
- POSTGRES_USER=username
122+
- POSTGRES_PASS=password
123+
- POSTGRES_DBNAME=postgis
124+
- POSTGRES_HOST=database
125+
- POSTGRES_PORT=5432
126+
- DB_MIN_CONN_SIZE=1
127+
- DB_MAX_CONN_SIZE=10
128+
command:
129+
bash -c "bash /tmp/scripts/wait-for-it.sh -t 120 -h database -p 5432 && /start.sh"
130+
depends_on:
131+
- database
132+
volumes:
133+
- ./dockerfiles/scripts:/tmp/scripts
134+
135+
stac-uvicorn:
136+
container_name: eoapi.stac-uvicorn
137+
profiles:
138+
- uvicorn
139+
build:
140+
context: .
141+
dockerfile: dockerfiles/Dockerfile.stac-uvicorn
142+
ports:
143+
- "${MY_DOCKER_IP:-127.0.0.1}:8081:8081"
144+
environment:
145+
- APP_HOST=0.0.0.0
146+
- APP_PORT=8081
147+
- HOST=0.0.0.0
148+
- PORT=8081
149+
- ENVIRONMENT=local
150+
# Postgres connection
151+
- POSTGRES_USER=username
152+
- POSTGRES_PASS=password
153+
- POSTGRES_DBNAME=postgis
154+
- POSTGRES_HOST_READER=database
155+
- POSTGRES_HOST_WRITER=database
156+
- POSTGRES_PORT=5432
157+
- DB_MIN_CONN_SIZE=1
158+
- DB_MAX_CONN_SIZE=10
159+
# https://github.com/developmentseed/eoAPI/issues/16
160+
# - TITILER_ENDPOINT=raster
161+
- TITILER_ENDPOINT=http://127.0.0.1:8082
162+
depends_on:
163+
- database
164+
- raster-uvicorn
165+
command:
166+
bash -c "bash /tmp/scripts/wait-for-it.sh -t 120 -h database -p 5432 && uvicorn eoapi.stac.app:app --host 0.0.0.0 --port 8081"
167+
volumes:
168+
- ./dockerfiles/scripts:/tmp/scripts
169+
170+
raster-uvicorn:
171+
container_name: eoapi.raster-uvicorn
172+
profiles:
173+
- uvicorn
174+
build:
175+
context: .
176+
dockerfile: dockerfiles/Dockerfile.raster-uvicorn
177+
# At the time of writing, rasterio and psycopg wheels are not available for arm64 arch
178+
# so we force the image to be built with linux/amd64
179+
platform: linux/amd64
180+
ports:
181+
- "${MY_DOCKER_IP:-127.0.0.1}:8082:8082"
182+
environment:
183+
# Application
184+
- HOST=0.0.0.0
185+
- PORT=8082
186+
# Postgres connection
187+
- POSTGRES_USER=username
188+
- POSTGRES_PASS=password
189+
- POSTGRES_DBNAME=postgis
190+
- POSTGRES_HOST=database
191+
- POSTGRES_PORT=5432
192+
- DB_MIN_CONN_SIZE=1
193+
- DB_MAX_CONN_SIZE=10
194+
# GDAL Config
195+
- CPL_TMPDIR=/tmp
196+
- GDAL_CACHEMAX=75%
197+
- GDAL_INGESTED_BYTES_AT_OPEN=32768
198+
- GDAL_DISABLE_READDIR_ON_OPEN=EMPTY_DIR
199+
- GDAL_HTTP_MERGE_CONSECUTIVE_RANGES=YES
200+
- GDAL_HTTP_MULTIPLEX=YES
201+
- GDAL_HTTP_VERSION=2
202+
- VSI_CACHE=TRUE
203+
- VSI_CACHE_SIZE=536870912
204+
# TiTiler Config
205+
- MOSAIC_CONCURRENCY=1
206+
# AWS S3 endpoint config
207+
- AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
208+
- AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
209+
# API Config
210+
- EOAPI_RASTER_ENABLE_MOSAIC_SEARCH=TRUE
211+
depends_on:
212+
- database
213+
command:
214+
bash -c "bash /tmp/scripts/wait-for-it.sh -t 120 -h database -p 5432 && uvicorn eoapi.raster.app:app --host 0.0.0.0 --port 8082"
215+
volumes:
216+
- ./dockerfiles/scripts:/tmp/scripts
217+
218+
vector-uvicorn:
219+
container_name: eoapi.vector-uvicorn
220+
profiles:
221+
- uvicorn
222+
build:
223+
context: .
224+
dockerfile: dockerfiles/Dockerfile.vector-uvicorn
225+
ports:
226+
- "${MY_DOCKER_IP:-127.0.0.1}:8083:8083"
227+
environment:
228+
# Application
229+
- HOST=0.0.0.0
230+
- PORT=8083
231+
# Postgres connection
232+
- POSTGRES_USER=username
233+
- POSTGRES_PASS=password
234+
- POSTGRES_DBNAME=postgis
235+
- POSTGRES_HOST=database
236+
- POSTGRES_PORT=5432
237+
- DB_MIN_CONN_SIZE=1
238+
- DB_MAX_CONN_SIZE=10
239+
command:
240+
bash -c "bash /tmp/scripts/wait-for-it.sh -t 120 -h database -p 5432 && uvicorn eoapi.vector.app:app --host 0.0.0.0 --port 8083"
241+
depends_on:
242+
- database
243+
volumes:
244+
- ./dockerfiles/scripts:/tmp/scripts
245+
246+
database:
247+
container_name: eoapi.db
248+
image: ghcr.io/stac-utils/pgstac:v0.8.1
249+
environment:
250+
- POSTGRES_USER=username
251+
- POSTGRES_PASSWORD=password
252+
- POSTGRES_DB=postgis
253+
- PGUSER=username
254+
- PGPASSWORD=password
255+
- PGDATABASE=postgis
256+
ports:
257+
- "${MY_DOCKER_IP:-127.0.0.1}:5439:5432"
258+
command: postgres -N 500
259+
volumes:
260+
- ./.pgdata:/var/lib/postgresql/data
261+
262+
networks:
263+
default:
264+
name: eoapi-network

0 commit comments

Comments
 (0)