Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .run/S3ExampleApplication.run.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="S3ExampleApplication" type="SpringBootApplicationConfigurationType" factoryName="Spring Boot">
<option name="ACTIVE_PROFILES" value="local" />
<module name="refarch-s3-integration-example" />
<option name="SPRING_BOOT_MAIN_CLASS" value="de.muenchen.refarch.integration.s3.example.S3ExampleApplication" />
<extension name="net.ashald.envfile">
<option name="IS_ENABLED" value="false" />
<option name="IS_SUBST" value="false" />
<option name="IS_PATH_MACRO_SUPPORTED" value="false" />
<option name="IS_IGNORE_MISSING_FILES" value="false" />
<option name="IS_ENABLE_EXPERIMENTAL_INTEGRATIONS" value="false" />
<ENTRIES>
<ENTRY IS_ENABLED="true" PARSER="runconfig" IS_EXECUTABLE="false" />
</ENTRIES>
</extension>
<method v="2">
<option name="Make" enabled="true" />
</method>
</configuration>
</component>
21 changes: 0 additions & 21 deletions .run/S3IntegrationApplication.run.xml

This file was deleted.

113 changes: 10 additions & 103 deletions docs/integrations/s3.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,120 +6,27 @@ Integration for CRUD operations on a S3 storage.

The S3 integration follows the [default naming conventions](./index.md#naming-conventions).

Beside the default integration it contains different client libraries for accessing the integration.

- REST: The REST client uses the REST endpoints of the `s3-rest-service` module to manage data in S3.
- Java: The Java client directly uses the in ports of the `s3-core` module.

### Dependency graph

The following graph shows the relationships between the various modules and how they interact and rely on each other.

```mermaid
flowchart RL
rest-service --> rest-starter --> rest-core --> core
rest-starter --> starter --> core

java-client-starter --> java-client --> client-core
java-client --> core
java-client-starter --> starter
rest-client-starter --> rest-client --> client-core
rest-client -.REST.-> rest-service
```

## Usage

```xml
<dependencies>
<!-- REST -->
<!-- requires running instance of s3-rest-service -->
<dependency>
<groupId>de.muenchen.refarch</groupId>
<artifactId>refarch-s3-integration-rest-starter</artifactId>
<version>...</version>
</dependency>
<!-- or Java -->
<dependency>
<groupId>de.muenchen.refarch</groupId>
<artifactId>refarch-s3-integration-java-starter</artifactId>
<artifactId>refarch-s3-integration-starter</artifactId>
<version>...</version>
</dependency>
</dependencies>
```

## Configuration

Following are the properties to configure the different modules. Some of them are custom defined and others are synonyms
for spring package properties.
Whether a property is an alias can be checked in the corresponding `application.yml` of each module.

### s3-integration-rest-service
### s3-integration-starter

| Property | Description | Example |
| ----------------------------------------------- | --------------------------------- | ---------------- |
| `refarch.s3.url` | URL of S3 endpoint to connect to. | `s3.example.com` |
| `refarch.s3.bucket-name` | Name of the bucket to connect to. | `refarch-bucket` |
| `refarch.s3.access-key` | Access key to use for connection. | |
| `refarch.s3.secret-key` | Secret key to use for connection. | |
| `refarch.s3.initial-connection-test` (optional) | Test connection to S3 at startup. | `true` (default) |

For authenticating the different endpoints OAuth 2.0 authentication needs to be configured.
See below example or the [according Spring documentation](https://docs.spring.io/spring-security/reference/servlet/oauth2/index.html#oauth2-resource-server).

```yml
spring:
security:
oauth2:
resourceserver:
jwt:
issuer-uri: https://sso.example.com/auth/realms/refarch
security:
oauth2:
resource:
user-info-uri: ${spring.security.oauth2.resourceserver.jwt.issuer-uri}/protocol/openid-connect/userinfo
```

### s3-integration-java-client-starter

| Property | Description | Example |
| ---------------------------------------------------------- | ----------------------------------------------------- | ------------------------ |
| `refarch.s3.client.max-file-size` (optional) | Single file limit for up- or downloading in byte. | `10MB` |
| `refarch.s3.client.max-batch-size` (optional) | Limit for up- or downloading a list of files in byte. | `100MB` |
| `refarch.s3.client.supported-file-extensions.*` (optional) | Map of allowed file extensions for up- and download. | `pdf: "application/pdf"` |

### s3-integration-rest-client-starter

All properties of [s3-integration-java-client-starter](#s3-integration-java-client-starter) and following:

| Property | Description | Example |
| ---------------------------------------- | ------------------------------------------------------------------------ | ------------------------------------ |
| `refarch.s3.client.document-storage-url` | URL to the RefArch S3 integration service. | `http://s3-integration-service:8080` |
| `refarch.s3.client.enable-security` | Switch to enable or disable OAuth 2.0 authentication against S3 service. | `true` |

For authentication against the `s3-service` a OAuth 2.0 registration with the name `s3` needs to be provided.
See following example or the [according Spring documentation](https://docs.spring.io/spring-security/reference/servlet/oauth2/index.html#oauth2-client).

```yml
spring:
security:
oauth2:
client:
provider:
sso:
issuer-uri: https://sso.example.com/auth/realms/refarch
user-info-uri: ${spring.security.oauth2.client.provider.sso.issuer-uri}/protocol/openid-connect/userinfo
jwk-set-uri: ${spring.security.oauth2.client.provider.sso.issuer-uri}/protocol/openid-connect/certs
# used for RequestResponseLoggingFilter in s3-rest-service
# only required if filter is explicitly enabled
user-name-attribute: user_name
registration:
s3:
provider: sso
authorization-grant-type: client_credentials
client-id: refarch_client
client-secret: client_secret_123
# profile required for username used in s3-rest-service RequestResponseLoggingFilter
# openid required for user info endpoint used in s3-rest-service JwtUserInfoAuthenticationConverter
# both scopes are only required if the according functions are explicitly used
scope: profile, openid
```
| Property | Description | Example |
| ------------------------------------------------- | ----------------------------------------------------- | --------------------- |
| `refarch.s3.url` | URL of S3 endpoint to connect to. | `s3.example.com` |
| `refarch.s3.region` (optional) | Region to use for the connection. | `us-east-1` (default) |
| `refarch.s3.access-key` | Access key to use for connection. | |
| `refarch.s3.secret-key` | Secret key to use for connection. | |
| `refarch.s3.path-style-access-enabled` (optional) | If bucket is specified via path (subdomain if false). | `true` (default) |
| `refarch.s3.initial-connection-test` (optional) | Test connection to S3 at startup. | `true` (default) |
15 changes: 15 additions & 0 deletions docs/support/migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,18 @@ variables inside the Docker compose file.
For deploying the central Gateway use the [according container image](../gateway.md#usage) and compare your working config with the `application.yml`,
`application-local.yml` (inside [gateway code `resources` folder](https://github.com/it-at-m/refarch/tree/main/refarch-gateway/src/main/resources))
and [configuration section](../gateway.md#configuration) to find properties which need to be migrated.

## S3-Integration v3

### Removed without replacement

- separate s3 service (image: `refarch/s3-integration-rest-service`)
- file size and type validation
- delete folder method

### Migrate

- Replace `DocumentStorageFileRepository` with `FileOperationsInPort`
- Some methods were merged
- Replace `DocumentStorageFolderRepository` with `FolderOperationsInPort`
- Some methods were merged
5 changes: 1 addition & 4 deletions refarch-integrations/refarch-s3-integration/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,6 @@
<modules>
<module>refarch-s3-integration-core</module>
<module>refarch-s3-integration-starter</module>
<module>refarch-s3-integration-client-core</module>

<module>refarch-s3-integration-java</module>
<module>refarch-s3-integration-rest</module>
<module>refarch-s3-integration-example</module>
</modules>
</project>

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

Loading
Loading