You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/deploy/main.md
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -12,17 +12,17 @@ IMPORTANT NOTE: Do a `git-lfs pull` so that you have everything downloaded from
12
12
:::
13
13
14
14
# Deployment
15
-
[./deploy](https://github.com/CogStack/CogStack-NiFi/tree/master/deploy/) contains an example deployment of the customised NiFi image with related services for document processing, NLP and text analytics.
15
+
[./deploy](https://github.com/CogStack/CogStack-NiFi/tree/main/deploy/) contains an example deployment of the customised NiFi image with related services for document processing, NLP and text analytics.
16
16
17
17
The key files are:
18
18
-`services.yml` - defines all the available services in docker-compose format. K8s (i.e. multi container service deployments is coming soon...)
19
19
-`Makefile` - scripts for running docker-compose commands,
20
20
-`.env` - local environment variables definitions, deployment `.env` files are located in the `/deploy` folder, security `.env` files are located in the `/security` folder, containing users and certificate generation settings.
21
21
The above mentioned files should be the files that you will most likely need to change during a deployment.
22
22
23
-
Individual service configurations are provided in [`./services`](https://github.com/CogStack/CogStack-NiFi/tree/master/services/).
23
+
Individual service configurations are provided in [`./services`](https://github.com/CogStack/CogStack-NiFi/tree/main/services/).
24
24
25
-
Apache NiFi-related files are provided in [`./nifi`](https://github.com/CogStack/CogStack-NiFi/tree/master/nifi/) directory.
25
+
Apache NiFi-related files are provided in [`./nifi`](https://github.com/CogStack/CogStack-NiFi/tree/main/nifi/) directory.
Copy file name to clipboardExpand all lines: docs/deploy/services.md
+17-17Lines changed: 17 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -33,7 +33,7 @@ All the services are defined in `services.yml` file and these are:
33
33
-`git-ea` - Github-like web service, you can host your own repositories here if your organisation is strict security-wise
34
34
35
35
**IMPORTANT**
36
-
Please note that some of the necessary configuration parameters, variables and paths are also defined in the [`services.yml`](https://github.com/CogStack/CogStack-NiFi/tree/master/deploy/services.yml) file.
36
+
Please note that some of the necessary configuration parameters, variables and paths are also defined in the [`services.yml`](https://github.com/CogStack/CogStack-NiFi/tree/main/deploy/services.yml) file.
37
37
38
38
## Optional NLP services
39
39
In addition, there are defined such NLP services:
@@ -92,9 +92,9 @@ These paths can be defined in `.env` file in the deployment directory.
92
92
### Bio-YODIE
93
93
[Bio-YODIE](https://github.com/GateNLP/Bio-YODIE) is a named entity linking application build using [GATE NLP](https://gate.ac.uk/) suite ([publication](https://arxiv.org/abs/1811.04860)).
94
94
95
-
The application files are stored in [`nlp-services/applications/bio-yodie/`](https://github.com/CogStack/CogStack-NiFi/tree/master/services/nlp-services/applications/bio-yodie) directory.
95
+
The application files are stored in [`nlp-services/applications/bio-yodie/`](https://github.com/CogStack/CogStack-NiFi/tree/main/services/nlp-services/applications/bio-yodie) directory.
96
96
97
-
The Bio-Yodie service configuration is stored in [`nlp-services/applications/bio-yodie/config/`](https://github.com/CogStack/CogStack-NiFi/tree/master/services/nlp-services/applications/bio-yodie/config) directory - the key service configuration properties are defined in `application.properties` file.
97
+
The Bio-Yodie service configuration is stored in [`nlp-services/applications/bio-yodie/config/`](https://github.com/CogStack/CogStack-NiFi/tree/main/services/nlp-services/applications/bio-yodie/config) directory - the key service configuration properties are defined in `application.properties` file.
98
98
99
99
100
100
### GATE
@@ -140,9 +140,9 @@ MedCAT is deployed as a service exposing RESTful API using the implementation fr
140
140
### MedCAT Service
141
141
142
142
143
-
MedCAT Service resources are stored in [`./services/nlp-services/applications/medcat/`](https://github.com/CogStack/CogStack-NiFi/tree/master/services/nlp-services/applications/medcat) directory.
144
-
The key configuration properties stored as environment variables are defined in [`./services/nlp-services/applications/medcat/config/`](https://github.com/CogStack/CogStack-NiFi/tree/master/services/nlp-services/applications/medcat/config) sub-directory.
145
-
The models used by MedCAT are stored in [`./servies/nlp-services/applications/cat/models/`](https://github.com/CogStack/CogStack-NiFi/tree/master/services/nlp-services/applications/medcat/models).
143
+
MedCAT Service resources are stored in [`./services/nlp-services/applications/medcat/`](https://github.com/CogStack/CogStack-NiFi/tree/main/services/nlp-services/applications/medcat) directory.
144
+
The key configuration properties stored as environment variables are defined in [`./services/nlp-services/applications/medcat/config/`](https://github.com/CogStack/CogStack-NiFi/tree/main/services/nlp-services/applications/medcat/config) sub-directory.
145
+
The models used by MedCAT are stored in [`./servies/nlp-services/applications/cat/models/`](https://github.com/CogStack/CogStack-NiFi/tree/main/services/nlp-services/applications/medcat/models).
146
146
A default model to play with is provided, called `MedMen` and there is a script `./services/nlp-services/applications/medcat/models/download_medmen.sh` to download it, please make sure you are in the `./services/nlp-services/applications/medcat/models/` before executing the download script.
147
147
148
148
For more information on the MedCAT Service configuration and use please refer to [the official documentation](https://github.com/CogStack/MedCATservice).
@@ -194,7 +194,7 @@ During start-up the data is loaded from a previously generated DB dump.
194
194
195
195
All the necessary resources, data and scripts are stored in `pgsamples/` directory.
196
196
During the service initialization, the script `init_db.sh` will populate the database with sample data read from a database dump stored in `db_dump` directory.
197
-
The directory [`./services/pgsamples/scripts`](https://github.com/CogStack/CogStack-NiFi/tree/master/services/pgsamples/scripts) contains SQL schemas with scripts that will generate the database dump using sample data.
197
+
The directory [`./services/pgsamples/scripts`](https://github.com/CogStack/CogStack-NiFi/tree/main/services/pgsamples/scripts) contains SQL schemas with scripts that will generate the database dump using sample data.
198
198
199
199
When deployed the PostgreSQL database is exposed at port `5432` of the `samples-db` container.
200
200
The port is also bound from container to the host machine `5555` port.
@@ -262,7 +262,7 @@ Other `.env` files are mounted but those are only useful for custom scripts wher
262
262
`tika-service` provides document text extraction functionality of [Apache Tika](https://tika.apache.org/).
263
263
[Tika Service](https://github.com/CogStack/tika-service) implements the actual Apache Tika functionality behind a RESTful API.
264
264
265
-
The application data, alongside configuration file, is stored in [`./services/tika-service`](https://github.com/CogStack/CogStack-NiFi/tree/master/services/tika-service) directory.
265
+
The application data, alongside configuration file, is stored in [`./services/tika-service`](https://github.com/CogStack/CogStack-NiFi/tree/main/services/tika-service) directory.
266
266
267
267
When deployed Tika Service exposes port `8090` at `tika-service` container being available to all services within `cognet` Docker network, most importantly by `nifi` data processing engine.
268
268
The Tika service REST API endpoint for processing documents is available at `http://tika-service:8090/api/process`.
@@ -290,8 +290,8 @@ All settings are decribed [here](https://github.com/CogStack/ocr-service/blob/ma
290
290
## NLP Services
291
291
292
292
In the example deployment we use NLP applications running as a service exposing REST API.
293
-
The current version of API specs is specified in [`./services/nlp-services/api-specs/`](https://github.com/CogStack/CogStack-NiFi/tree/master/services/nlp-services/api-specs) directory (both [Swagger](https://swagger.io/) and [OpenAPI](https://www.openapis.org/) specs).
294
-
The applications are stored in [`./services/nlp-services/applications`](https://github.com/CogStack/CogStack-NiFi/tree/master/services/nlp-services/applications).
293
+
The current version of API specs is specified in [`./services/nlp-services/api-specs/`](https://github.com/CogStack/CogStack-NiFi/tree/main/services/nlp-services/api-specs) directory (both [Swagger](https://swagger.io/) and [OpenAPI](https://www.openapis.org/) specs).
294
+
The applications are stored in [`./services/nlp-services/applications`](https://github.com/CogStack/CogStack-NiFi/tree/main/services/nlp-services/applications).
295
295
296
296
297
297
### NLP API
@@ -359,14 +359,14 @@ As a companion service, `medcat-trainer-nginx` serves as a NGINX reverse-proxy f
359
359
360
360
When deployed, `medcat-trainer-ui` exposes port `8000` on the container.
361
361
`medcat-trainer-nginx` exposes port `8000` on the container and binds it to port `8001` on the host machine - it proxies all the requests to the MedCAT Trainer web service.
362
-
The NGINX configuration is stored in [`./services/medcat-trainer/nginx`](https://github.com/CogStack/CogStack-NiFi/tree/master/services/medcat-trainer/nginx) directory.
362
+
The NGINX configuration is stored in [`./services/medcat-trainer/nginx`](https://github.com/CogStack/CogStack-NiFi/tree/main/services/medcat-trainer/nginx) directory.
363
363
364
364
To access the MedCAT Trainer user interface and admin panel, one can use the default built-in credentials: user `admin` with password `admin`.
365
365
366
366
For more information on the MedCAT Trainer configuration and use please refer to [the official documentation](https://github.com/CogStack/MedCATtrainer).
367
367
368
-
MedCAT Trainer resources are stored in [`./services/medcat-trainer`](https://github.com/CogStack/CogStack-NiFi/tree/master/services/nlp-services//medcat-trainer) directory.
369
-
The key configuration is stored in [`./services/medcat-trainer/env`](https://github.com/CogStack/CogStack-NiFi/tree/master/services/medcat-trainer/envs/env) file.
368
+
MedCAT Trainer resources are stored in [`./services/medcat-trainer`](https://github.com/CogStack/CogStack-NiFi/tree/main/services/nlp-services//medcat-trainer) directory.
369
+
The key configuration is stored in [`./services/medcat-trainer/env`](https://github.com/CogStack/CogStack-NiFi/tree/main/services/medcat-trainer/envs/env) file.
370
370
371
371
372
372
@@ -438,7 +438,7 @@ You should not really need to ever modify these files, only the `.env` files sho
The used configuration files for ElasticSearch and Kibana are provided in [`./services/elasticsearch/config/`](https://github.com/CogStack/CogStack-NiFi/tree/master/services/elasticsearch/config) and [`./services/kibana/config/`](https://github.com/CogStack/CogStack-NiFi/tree/master/services/kibana/config) directories respectively for [`OpenSearch`](https://opensearch.org/docs/latest/install-and-configure/configuration/) and [`OpenSearch Dashboard`](https://opensearch.org/docs/latest/dashboards/index/).
441
+
The used configuration files for ElasticSearch and Kibana are provided in [`./services/elasticsearch/config/`](https://github.com/CogStack/CogStack-NiFi/tree/main/services/elasticsearch/config) and [`./services/kibana/config/`](https://github.com/CogStack/CogStack-NiFi/tree/main/services/kibana/config) directories respectively for [`OpenSearch`](https://opensearch.org/docs/latest/install-and-configure/configuration/) and [`OpenSearch Dashboard`](https://opensearch.org/docs/latest/dashboards/index/).
442
442
443
443
444
444
### Security
@@ -458,7 +458,7 @@ Also note that in some scenarios a manual creation of index mapping may be a goo
458
458
<spanstyle="color: red"><strong> IMPORTANT: not creating the mapping of an index will result in ElasticSearch/OpenSearch automatically map all field datatypes as string, making fields such as date/timestamps not incredibly !</strong></span>
459
459
460
460
461
-
A script `es_index_initializer.py` has been provided in [`./services/elasticsearch/scripts/`](https://github.com/CogStack/CogStack-NiFi/tree/master/services/elasticsearch/scripts) directory to help with that.
461
+
A script `es_index_initializer.py` has been provided in [`./services/elasticsearch/scripts/`](https://github.com/CogStack/CogStack-NiFi/tree/main/services/elasticsearch/scripts) directory to help with that.
462
462
463
463
### Installing and maintaining Elasticsearch/Opensearch
464
464
@@ -592,7 +592,7 @@ Since `jupyter-hub` is running in the `cognet` Docker network it has access to a
592
592
593
593
For more information on the use and configuration of Jupyter Hub please refer to [the official Jupyter Hub documentation](https://jupyter.org/hub).
594
594
595
-
The JupyterHub comes with an example Jupyter notebook that is stored in [`./services/jupyter-hub/notebooks`](https://github.com/CogStack/CogStack-NiFi/tree/master/services/jupyter-hub/notebooks) directory.
595
+
The JupyterHub comes with an example Jupyter notebook that is stored in [`./services/jupyter-hub/notebooks`](https://github.com/CogStack/CogStack-NiFi/tree/main/services/jupyter-hub/notebooks) directory.
596
596
597
597
### Access and account control
598
598
To access Jupyter Hub on the host machine (e.g.localhost), one can type in the browser `http://localhost:8888`.
@@ -703,7 +703,7 @@ This service users NiFi's `../../security/root-ca.p12` and `../../security/root
703
703
Although by default not used in the deployment example, NGINX is primarily used as a reverse proxy, limiting the access to the used services that normally expose endpoint for the end-user.
704
704
For a simple scenario, it can used only for securing access to Apache NiFi webservice endpoint.
705
705
706
-
All the necessary configuration files and scripts are located in [`./services/nginx/config/`](https://github.com/CogStack/CogStack-NiFi/tree/master/services/nginx/config) directory where the user and password generation script `setup_passwd.sh`.
706
+
All the necessary configuration files and scripts are located in [`./services/nginx/config/`](https://github.com/CogStack/CogStack-NiFi/tree/main/services/nginx/config) directory where the user and password generation script `setup_passwd.sh`.
0 commit comments