Skip to content

Commit a2a95dc

Browse files
Add docs for GitHub curation
1 parent 1cfcee8 commit a2a95dc

File tree

1 file changed

+32
-11
lines changed

1 file changed

+32
-11
lines changed

README.md

Lines changed: 32 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# docker_dev_env_experiment
22

3-
Hello everyone! The purpose of this repo is to give you an easy way to run a full
3+
Hello everyone! The purpose of this repo is to give you an easy way to run a full
44
development environment for Clearly Defined including:
55
* [website](https://github.com/clearlydefined/website)
66
* [service](https://github.com/clearlydefined/service)
@@ -47,7 +47,7 @@ services:
4747
build:
4848
context: <path-to-website-repo-on-your-system>
4949
dockerfile: DevDockerfile
50-
ports:
50+
ports:
5151
- "3000:3000"
5252
stdin_open: true
5353
service:
@@ -115,8 +115,8 @@ DEFINITION_MONGO_COLLECTION_NAME="definitions-paged"
115115
# Harvest Store Info
116116
HARVEST_STORE_PROVIDER="file"
117117
118-
# Note - this is mounted as a volume
119-
# into the container for the
118+
# Note - this is mounted as a volume
119+
# into the container for the
120120
# clearly defined service
121121
# see docker-compose.yml for more details
122122
FILE_STORE_LOCATION="/tmp/harvested_data"
@@ -145,11 +145,32 @@ You can also query the service API with:
145145
curl http://localhost:4000
146146
```
147147

148+
### Additional Setup for GitHub curation(Optional)
149+
150+
If you want to work with curation on GitHub, you could follow these steps
151+
1. Fork [curated-data-dev](https://github.com/clearlydefined/curated-data-dev) into your own GitHub account and modify the .env file.
152+
```
153+
CURATION_GITHUB_OWNER="<your own GitHub account>"
154+
CURATION_GITHUB_REPO="curated-data-dev"
155+
```
156+
2. In order to get GitHub webhook events, a http forwarding proxy is needed. Here [ngork](https://ngrok.com/download) has been used. Run `ngork http http://localhost:4000`. You will see something similar to this
157+
```
158+
Session Status online
159+
Session Expires 1 hour, 59 minutes
160+
Version 2.3.35
161+
Region United States (us)
162+
Web Interface http://127.0.0.1:4040
163+
Forwarding http://83f8ddfb177b.ngrok.io -> http://localhost:4000
164+
Forwarding https://83f8ddfb177b.ngrok.io -> http://localhost:4000
165+
```
166+
3. Then you could create a webhook in your forked curate-data repository. Use ngork forwarding url(prefer https one) as the webhook payload URL. And put 'secret' as the webhook secret.
167+
4. You could verify Github webhook events with `ngork` management UI, http://localhost:4040
168+
148169
## What You're Running
149170

150171
Now, let's go through what your are running, container by container.
151172

152-
* Clearly Defined Website
173+
* Clearly Defined Website
153174
* Clearly Defined Service
154175
* Clearly Defined Crawler
155176
* Clearly Defined Mongo DB
@@ -161,7 +182,7 @@ This is the Clearly Defined React UI. It's what you see when you open your brows
161182

162183
### Clearly Defined Service
163184

164-
This is the backend of Clearly Defined, you can use it through the Website UI or through
185+
This is the backend of Clearly Defined, you can use it through the Website UI or through
165186
querying it directly through the command line.
166187

167188
Any Clearly Defined environment needs a place to store raw harvest information. In the case of this development environment, we use the file store for storing harvest information (our production setup uses Azure blob storage).
@@ -265,18 +286,18 @@ $ docker-compose up --detach --build crawler
265286

266287
### Limitations
267288

268-
When you look at a definition in the UI and create a curation (this uses the API call PATCH /curations), the curation WILL be opened
289+
When you look at a definition in the UI and create a curation (this uses the API call PATCH /curations), the curation WILL be opened
269290
as a pull request on the [curated-data-dev](https://github.com/clearlydefined/curated-data-dev), but you will not see it
270291
under the "Curations" section when you refresh the definition's page.
271292

272293
In the Azure dev and production environment, creating a curation will open a PR on the appropriate github curated-data repo,
273294
and then, once the pull request is open, GitHub will then use a webhook.
274295

275-
The webhook will POST to an Azure logic app. That app will then put the curation on the Azure storage queue, which is how it will
296+
The webhook will POST to an Azure logic app. That app will then put the curation on the Azure storage queue, which is how it will
276297
end up in the curation store (in this case, mongo).
277298

278-
I haven't yet figured out a way to do this without an Azure logic app (but will continue looking into this). I did try
279-
creating a GitHub webhook to POST to http://localhost:4000, but GitHub requires that the webhooks it POSTs to be
299+
I haven't yet figured out a way to do this without an Azure logic app (but will continue looking into this). I did try
300+
creating a GitHub webhook to POST to http://localhost:4000, but GitHub requires that the webhooks it POSTs to be
280301
accessible over the public internet.
281302

282303
When I figure out a solution, I will update this README.
@@ -305,7 +326,7 @@ services:
305326
# build:
306327
# context: <path-to-website-repo-on-your-system>
307328
# dockerfile: DevDockerfile
308-
# ports:
329+
# ports:
309330
# - "3000:3000"
310331
# stdin_open: true
311332
service:

0 commit comments

Comments
 (0)