|
| 1 | +# Interfaces Overview (Indexing-Service) |
| 2 | + |
| 3 | +This document aims to answer questions on how to configure external dependencies and which public interfaces are offered by Indexing-Service in a comprehensive way. It is meant to be used for getting an overview and guidance in addition to the official documentation, which is available at the official [Indexing-Service Web page (not yet available)](https://kit-data-manager.github.io/webpage/indexing-service/).How can this software be connected with other software? This is the question this document aims to answer. |
| 4 | + |
| 5 | +> ℹ️ **Note:** |
| 6 | +> This document applies to the Indexing-Service version it is shipped with. If you have a specific version running, please refer to `INTERFACE.md` of this particular release. |
| 7 | +
|
| 8 | +## TOC |
| 9 | + |
| 10 | +- [Interfaces Overview for Indexing-Service](#interfaces-overview-for-indexing-service) |
| 11 | + * [External Dependencies](#external-dependencies-) 📤 |
| 12 | + + [Relational Database (mandatory)](#relational-database-mandatory-) ⛁ |
| 13 | + + [Local Filesystem (mandatory)](#local-filesystem-mandatory-) 📂 |
| 14 | + + [Messaging (mandatory)](#messaging-mandatory-) 💬 |
| 15 | + + [Search (mandatory)](#search-mandatory-) 🔍 |
| 16 | + + [Access Control (optional)](#access-control-optional-) 🔐 |
| 17 | + * [Public Interfaces](#public-interfaces-) 📥 |
| 18 | + + [HTTP / REST](#http--rest) |
| 19 | + + [Monitoring Health/Info](#monitoring-healthinfo) 🔍 |
| 20 | + |
| 21 | +## Requirements |
| 22 | +- [Messaging](#messaging-optional-) |
| 23 | +- [Elasticsearch](https://kit-data-manager.github.io/webpage/metastore/documentation/installation/framework/setup-elasticsearch.html) |
| 24 | +Messaging is used to inform indexing service about any updates regarding metadata documents. Indexing service will fetch the document, transform it (if mapping available) and send it to elasticsearch for indexing. |
| 25 | + |
| 26 | +## External Dependencies 📤 |
| 27 | + |
| 28 | +External dependencies are third-party services that are required for Indexing-Service to work properly or that can be added optionally to provide additional functionality. Typically, external dependencies require additional software to be installed and configured, before they can be included in the Indexing-Service configuration, which is typically done via the main configuration file `application.properties`. If you do not want to lose your default settings, we recommend that you make a copy of "application.properties" and move it to the "config" subfolder. Remove all properties you want to keep from the new file. **All properties in "config/application.properties" override the settings in "application.properties".** |
| 29 | + |
| 30 | + |
| 31 | +## External Dependencies 📤 |
| 32 | + |
| 33 | +External dependencies are third-party services that are required for Indexing-Service to work properly or that can be added optionally to provide additional functionality. Typically, external dependencies require |
| 34 | +additional software to be installed and configured, before they can be included in the Indexing-Service configuration, which is should be done via the configuration file `config/application.properties`. |
| 35 | + |
| 36 | +### Relational Database (mandatory) ⛁ |
| 37 | +A relational database is required by Indexing-Service to store administrative metadata for mapping documents. If not configured properly, Indexing-Service will fail to start. |
| 38 | + |
| 39 | +#### Configuration ⚙️ |
| 40 | + - H2 In-Memory (driver included, used for **testing only**, **not** recommended for **production**) [Example](https://github.com/kit-data-manager/indexing-service/blob/v0.9.0/src/test/resources/test-config/application-test.properties#L14-L18) |
| 41 | + - H2 File-Based (driver included, used for basic Docker setup or for 'quick and dirty' tests, **not** recommended for **production**) [Example](https://github.com/kit-data-manager/indexing-service/blob/v0.9.0/settings/application-docker.properties#L26-L33) |
| 42 | + - Postgres (driver included, **requires a running PostgreSQL server**, **recommended for production**) [PostgreSQL](https://www.postgresql.org/), [Example](https://github.com/kit-data-manager/indexing-service/blob/v0.9.0/settings/application-postgres.properties#L34-L47) |
| 43 | + |
| 44 | +> ℹ️ **Note:** |
| 45 | +> Other relational databases, such as MariaDB, SQLite, or Oracle, may also work, but require additional steps. To allow Indexing-Service to connect, the source code repository must be cloned, an appropriate JDBC driver has to be added to `build.gradle` and Indexing-Service has to be compiled. Proper JDBC drivers are typically provided on the database's web page. Afterwards, the database can be configured in `config/application.properties` similar to PostgreSQL but with database-specific property naming. Please refer to the driver documentation for details. |
| 46 | +
|
| 47 | +### Local Filesystem (mandatory) 📂 |
| 48 | +Indexing-Service requires access to the local file system in order to store and manage uploaded mapping documents. Indexing-Service requires access to one folder, which can be located on the local hard drive or mounted via NFS. It's recommended to create a subfolder named 'mapping' in the installation directory. |
| 49 | + |
| 50 | + |
| 51 | +#### Configuration ⚙️ |
| 52 | + - see `application.properties` |
| 53 | + - [Configure folders](https://github.com/kit-data-manager/indexing-service/blob/v0.9.0/settings/application-default.properties#L71-L74) |
| 54 | + |
| 55 | +> ℹ️ **Note:** |
| 56 | +> The file path to the folder has to start with three '/'. If you overwrite the default setting we recommend to create or edit 'config/application.properties'. e.g.: |
| 57 | +> ``` title= config/application.properties |
| 58 | +> metastore.indexer.mappingsLocation:file:///data/indexing/mapping |
| 59 | +>``` |
| 60 | +> **If the folder do not exist, it will be created.** |
| 61 | +
|
| 62 | +### Messaging (mandatory) 💬 |
| 63 | +AMQP-based messaging is a mandatory feature of Indexing-Service, which allows Indexing-Service to **receive** messages about creation, modification, and deletion events inside MetaStore related to **metadata documents** and process them in an asynchronous way. |
| 64 | +#### Installation |
| 65 | +- [Installing RabbitMQ](https://kit-data-manager.github.io/webpage/metastore/documentation/installation/framework/setup-rabbitMq.html) |
| 66 | +#### Configuration ⚙️ |
| 67 | + - [RabbitMQ](https://www.rabbitmq.com/) (dependencies included, serves as messaging distributor, requires a running RabbitMQ server) |
| 68 | + - [Introduction Messaging for MetaStore](https://kit-data-manager.github.io/webpage/metastore/documentation/installation/messaging/messaging-introduction.html) |
| 69 | + - [Configuration Messaging for Indexing-Service](https://kit-data-manager.github.io/webpage/metastore/documentation/installation/messaging/messaging-configuration.html) |
| 70 | + - [Example](https://kit-data-manager.github.io/webpage/metastore/documentation/installation/setup-metastore.html#rabbitmq) |
| 71 | +
|
| 72 | +### Enhanced Search (mandatory) 🔍 |
| 73 | +Indexing-Service requires an [elasticsearch instance](https://www.elastic.co/de/elasticsearch/) for indexing received metadata documents. (JSON metadata documents are provided by default. XML documents should be transformed to JSON using GEMMA (currently the only available mapping)) (In future versions [mapping-service](https://github.com/kit-data-manager/mapping-service) should be integrated.) |
| 74 | +#### Configuration ⚙️ |
| 75 | +- [Configuration Receiver](https://kit-data-manager.github.io/webpage/metastore/documentation/installation/setup-metastore.html#elasticsearch) |
| 76 | + |
| 77 | +### Access Control (optional) 🔐 |
| 78 | +By default, Indexing-Service itself is open for all kinds of operations, i.e., read and write, where write access should be restricted on the user interface level, e.g., by a password-protected area for critical operations. Optionally, authentication and authorization via JSON Web Tokens (JWT) issued by a Keycloak instance, can be configured, which allows a fine-granulated access management on document level. |
| 79 | +> ℹ️ **Note:** |
| 80 | +> As Indexing-Service is not planned to be used by humans, no access control is used by default. Therefor the service should not be accessible from outside KIT. Managing mappings should be done by an administrator. |
| 81 | +
|
| 82 | +#### Requirements |
| 83 | +- [Keycloak](https://www.keycloak.org/) |
| 84 | +#### Configuration ⚙️ |
| 85 | +- [Setup Indexing-Service](https://kit-data-manager.github.io/webpage/metastore/documentation/installation/setup-metastore.html#keycloak) |
| 86 | +Configuration is similar to MetaStore. |
| 87 | +
|
| 88 | +## Public Interfaces 📥 |
| 89 | +Public Interfaces are used to access Indexing-Service in order to obtain its contents, typically this happens via HTTP/REST. Depending on the interface, special clients or protocols must be used to access a specific public interface. |
| 90 | +
|
| 91 | +
|
| 92 | +Interfaces which trigger functionality or return information **on request**. |
| 93 | +
|
| 94 | +### HTTP / REST |
| 95 | +A REST interface which allows to access the service functionality, like creating/registering, updating and validating metadata documents/schemas on request. |
| 96 | +#### Documentation 📖 |
| 97 | + - [OpenAPI](http://localhost:8050/swagger-ui.html) via running instance. |
| 98 | + - [Usage with Examples](https://github.com/kit-data-manager/indexing-service/blob/main/restDocu.md) |
| 99 | + |
| 100 | +#### Services known to connect to this interface: |
| 101 | +-/- |
| 102 | +
|
| 103 | +### Monitoring Health/Info |
| 104 | +A REST interface which allows to access the status of the service: |
| 105 | +1. *hostname:port/contextpath*/actuator/info (information about version) |
| 106 | +2. *hostname:port/contextpath*/actuator/health (information about (connected) services) |
| 107 | +
|
0 commit comments