-
Notifications
You must be signed in to change notification settings - Fork 12
Add ingest.sh script. #164
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
I've explored this topic further, and it seems that the most effective way to implement an API is by using the In the meantime, the ingest.sh script and command will simplify the ingestion process. |
j08lue
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will leave running the script to others.
It looks nicely laid out and documented and is providing at least a basic mechanism for data ingestion, which is a battery we have to include, IMO.
a64a9d6 to
bfb84a4
Compare
ciaransweet
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall looks great to me, a real step in to making this easier
Just a few comments that I think will waterproof this
bfb84a4 to
fc47b05
Compare
Co-authored-by: Jonas <[email protected]>
fc47b05 to
217fdd4
Compare
ciaransweet
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
|
Many thanks for your reviews! ❤️ |
Data Management (data ingestion in a first step) is a common requirement for several of our
eoapi-k8sdeployments and projects.From my perspective we should think about data on eoAPI as "content" and not so much as "configuration". While configuration should be in code and part of the reproducible deployment, content should me managable by an admin user. At some point we probably want to think about building some auth-supported API to allow management, eventually even connecting it to some user interface.
This PR consists in a first simple step to add very basic ingestion as a bash script
ingest.shand the relatedmake ingestcommand, it:rasterpod in the namespaceseoapianddefaultpypgstacon itcollection.jsonanditems.jsonfrom the current directory (can be overridden with the environment variablesEOAPI_COLLECTIONS_FILEandEOAPI_ITEMS_FILE.pypgstacloads the collection and items into the pySTAC postgresql database.Related to EOEPCA/data-access#107
fyi, @j08lue, @emmanuelmathot, @zacharyDez