Skip to content

Commit 6fd6851

Browse files
committed
Add basic ingestion docs
1 parent 44a7d1b commit 6fd6851

File tree

1 file changed

+60
-0
lines changed

1 file changed

+60
-0
lines changed

docs/ingestion.md

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
2+
# Instructions for Loading Data into pgSTAC using Kubernetes and Helm
3+
4+
This guide outlines the steps necessary to configure and load data into a PostgreSQL STAC (pgSTAC) database within a Kubernetes environment using Helm. This guide is provided as a reference in the absence of a complete ingestion pipeline, which is best pr
5+
6+
## Prerequisites
7+
8+
- Ensure your Kubernetes cluster is running, and you have the necessary access to modify ConfigMaps and deploy Helm charts.
9+
- The `eoAPI` Helm chart is installed and properly configured.
10+
- STAC records for the collection and items you wish to load (e.g., `collection.json` and `items.json`).
11+
12+
## Step 1: Modify the `initdb-json-config-eoapi` ConfigMap
13+
14+
1. Edit the existing `initdb-json-config-eoapi` ConfigMap.
15+
2. Add the necessary collection and item files (these will be treated as separate entries by Kubernetes).
16+
17+
Here’s an example of how you could structure the ConfigMap:
18+
19+
```yaml
20+
apiVersion: v1
21+
kind: ConfigMap
22+
metadata:
23+
name: initdb-json-config-eoapi
24+
data:
25+
collection.json: |
26+
{ ... } # Your collection JSON goes here.
27+
items.json: |
28+
{ ... } # Your items JSON goes here.
29+
```
30+
31+
Make sure that your collection and items data are correctly formatted according to STAC specifications.
32+
33+
## Step 2: Configure the `pgstacBootstrap` Command
34+
35+
In your values file, ensure that `.Values.pgstacBootstrap.command` follows this format:
36+
37+
```bash
38+
#!/bin/bash
39+
bash /opt/initdb/apt-and-pip-install.sh
40+
pypgstac pgready --dsn $PGADMIN_URI
41+
pypgstac load collections /opt/initdb/json-data/collection.json --dsn $PGADMIN_URI --method insert_ignore
42+
pypgstac load items /opt/initdb/json-data/items.json --dsn $PGADMIN_URI --method insert_ignore
43+
exit 0
44+
```
45+
46+
## Step 3: Deploy the Changes
47+
48+
Once the ConfigMap and the `pgstacBootstrap` command are correctly set up, run the following Helm command to upgrade your deployment:
49+
50+
```bash
51+
helm upgrade <release-name> <chart-name> --values <values-file>
52+
```
53+
54+
Replace `<release-name>`, `<chart-name>`, and `<values-file>` with your actual values.
55+
56+
After running this command, the `pgbootstrap` pod should start, execute the command script, and then exit gracefully once the data has been successfully loaded.
57+
58+
## Conclusion
59+
60+
Following these steps, the provided collections and items should be available through eoAPI. Ensure you monitor the logs of the `pgbootstrap` pod for any errors or confirmations of successful completion.

0 commit comments

Comments
 (0)