Merge branch 'main' into feature/graphInsightsDocu-CMEM-7120

Author: rpietzsch

.gitignore

@@ -8,3 +8,4 @@ tmp
 .DS_Store
 .vscode/
 .python-version
+

docs/build/mapping-creator/index.md

@@ -7,60 +7,66 @@ tags:
 ---
 # Mapping Creator
 
-## Configuration
+## Configuration Info
+
+A specific configuration is require to activate this feature.
+Furthermore, for best experience an LLM provider configuration is also needed.
+See the link in the info box for details.
 
 !!! info "Configuration"
 
     See [Mapping Creator and LLM Configuration](../../deploy-and-configure/configuration/dataintegration/index.md#mapping-creator-and-llm-configuration) to learn how to enable and configure this feature.
 
-The Mapping Creator need to configured in order to be activated and used.
-Furthermore, for best experience it requires an LLM provider configuration, see the link in the info box for the details.
-
 ## Usage
 
+The Mapping Creator is  a feature of BUILD transformation tasks (_BUILD_> _Project_> (_Workflow_>) _Transformation_> _Mapping Creator_).
+
 !!! question "Help"
 
     ![Help](mapping-creator-help.png){ class="bordered" width="20%" align=right padding=40 }
 
     Access help and a feature tour via the help menu :eccenca-item-question:.
 
-**TODO:** describe how to get there in the first place!!! BUILD > Project > [ Workflow > ] Transformation > Mapping Creator
-
 Use the Mapping Creator to connect your data to semantic models.
-Using visual tools, drag-and-drop, and suggestions, you’ll create mappings between your source data and knowledge graph classes as well as their properties.
+Using visual tools, drag-and-drop, and suggestions, you are able to create mappings between your source data and knowledge graph classes as well as their properties.
 
 ![Mapping Creator](mapping-creator.png){ class="bordered" }
 
-The Mapping Creator is split into three parts:
-
--   The left sidebar shows the source schema
--   The right sidebar shows the target schema
--   Mappings between elements in the source schema and elements in the target schema.
+The Mapping Creator consists of three parts:
 
-**TODO**: rename _sidebar_ and describe the actual _sidebars_ properly (left: schema information, right: see below (Edit saved mapping rule))
+-   Source schema shown on the left side
+-   Target Schema shown on the right side
+-   Mappings between elements in the source schema and in the target schema
 
 You can move, connect or disconnect, and inspect each element visually.
 
+<!--
+**TODO**: Clicking on an element in the source schema opens a detail view showing more schema information and describe the actual _sidebars_ properly (left: schema information, right: see below (Edit saved mapping rule)) **TODO**:@rpietzsch please add Details...
+-->
+
 ### Color Legend
 
+Each color and visual element used in the Mapping Creator has a specific meaning as shown in the screenshot below.
+
 **TODO:** show the color legend here, too
 
-<figure markdown="span">
-    ![Color Legend](mapping-creator-color-legend.png){ class="bordered" }
-    <figcaption>Color Legend Image Caption</figcaption>
-</figure>
+![Color Legend](mapping-creator-color-legend.png){ class="bordered" }
 
 ### Manual creation
 
-Every mapping begins by selecting or creating a target class.
-This defines where your data will be be mapped in the knowledge graph.
+To start a mapping you need to select or define a target class first.
+The target class defines where your data will be be mapped in the knowledge graph.
 
 #### Add properties
 
 ![Resource edit actions](mapping-creator-edit-actions.png){ class="bordered" width="40%" }
 
 To complete a mapping properties need to be added to complete your desired target schema (i.e. the graph fragment that your transformation shall yield).
-The following two options exist to add properties.
+
+There are two options to add properties:
+
+-   during class selection
+-   from vocabularies
 
 ##### During class selection
 
@@ -74,7 +80,7 @@ In the _add target class_ dialog you may select different kind of properties:
 
 The property preview helps to confirm your choice.
 
-##### By adding a property from your vocabularies
+##### From vocabularies
 
 ![Adding a property from your vocabulary](mapping-creator-property-selection.png){ class="bordered" width="80%" }
 
@@ -96,15 +102,15 @@ Click on an already saved target element to show more details.
 With the :eccenca-item-edit: Pencil icon you can edit the mapping rule and rename elements, or adjust configurations.
 Advanced users can refine transformation logic here.
 
-### Smart suggestions
+### Smart suggestions with AI support
 
 !!! info
     Note that this feature is only available if a Large Language Model is configured.
 
 Click the :eccenca-application-ai-suggestion: Magic Wand to automatically generate mapping suggestions based on your data and the selected class.
 Accept (:octicons-thumbsup-16:) or reject (:octicons-thumbsdown-16:) each suggestion as needed.
 
-#### Class Suggestions
+#### Class suggestions
 
 ![Smart Class Suggestions](mapping-creator-class-suggestion.png){ class="bordered" }
 
@@ -113,7 +119,7 @@ Alternatively, you can search for classes manually by typing text into the searc
 
 Hover over the wand icon for each suggestion to understand why this class has been recommended.
 
-#### Property Suggestions
+#### Property suggestions
 
 Property suggestions are generated that map your source data to elements of the selected target vocabularies.
 You can accept (:octicons-thumbsup-16:) or reject (:octicons-thumbsdown-16:) each suggestion.

docs/deploy-and-configure/configuration/dataintegration/index.md

@@ -1033,7 +1033,7 @@ Enable (or disable) the new visual [Mapping Create UI](../../../build/mapping-cr
 com.eccenca.di.mappingCreatorEnabled = true
 ```
 
-The [Mapping Creator can optionally use LLM](../../../build/mapping-creator/index.md#smart-suggestions) to automatically generate class and property mappings.
+The [Mapping Creator can optionally use LLM](../../../build/mapping-creator/index.md#smart-suggestions-with-ai-support) to automatically generate class and property mappings.
 Use the following configuration section as a blueprint to set up your OpenAI-compatible endpoint, providing:
 
 -   the API key,

docs/deploy-and-configure/installation/.pages

@@ -2,5 +2,6 @@ nav:
     - Installation: index.md
     - "Scenario: Single Node Cloud Installation": scenario-single-node-cloud-installation
     - "Scenario: Local Installation": scenario-local-installation
+    - "Scenario: Kubernetes Deployment": scenario-k8s-deployment
     - Migrating Stores: migrating-stores
 

docs/deploy-and-configure/installation/scenario-k8s-deployment/files/cmem-values.yaml

@@ -0,0 +1,77 @@
+# Default values for cmem.
+nameOverride: ""
+
+ingress:
+  enabled: true
+  className: nginx
+  annotations:
+    #kubernetes.io/tls-acme: "true"
+    cert-manager.io/cluster-issuer: OVERRIDDEN_BY_TASKFILE
+
+  hosts:
+    - host: "OVERRIDDEN_BY_TASKFILE"
+      paths:
+        - path: /
+          pathType: Prefix
+          serviceName: explore
+          servicePort: 8080
+        - path: /dataintegration
+          pathType: Prefix
+          serviceName: dataintegration
+          servicePort: 8080
+  tls:
+    - hosts:
+         - "OVERRIDDEN_BY_TASKFILE"
+      secretName:  cmem-ingress-cert
+
+global:
+  # The OAuth client that will be used to load the workspace initially and run the schedulers.
+  # In order to enable that feature, you have to enable the service account feature in the Keycloak
+  # OIDC client "cmem-service-client". Furthermore, an AccessCondition that maps with `eccauth:requiresClient :cmem-service-account`
+  # is required.
+
+  protocol: "https"
+  cmemClientId: cmem-service-account
+  cmemClientSecret: c8c12828-000c-467b-9b6d-2d6b5e16df4a
+  hostname: "OVERRIDDEN_BY_TASKFILE"
+  # This is the base keycloak realm url, e.g. https://cmem.example.com/auth/realms/cmem
+  # license: cmem-license
+  disablePythonInstallFromOutside: false
+
+bootstrap:
+  enabled: true
+
+explore:
+  config:
+    # disabeld by default
+    # create Objects of kind Pod- or SeviceMonitor see 
+    # https://github.com/prometheus-operator/prometheus-operator/blob/main/Documentation/developer/getting-started.md#deploying-a-sample-application
+    serviceMonitorEnabaled: false
+    podMonitorEnabled: true
+  store:
+    graphdb:
+      enabled: true
+      repository: cmem
+      user: provisioner
+      password: "iHaveSuperpowers"
+      # With Dataplatform 22.2, we have a "host" property
+      host: "graphdb.graphdb"
+      sslEnabled: false
+      createRepoOnStartup: true
+      changeTrackingActive: true
+      changeTrackingMaxQuadsMem: 1000
+
+dataintegration:
+  config:
+    # pwgen 16 1
+    passwordCryptKey: "Peek1uh1Ui8viequ"
+    # pwgen 40 1
+    playHTTPSecret: "waejai5aiMoShuvahcai3iaC3eefeichahYaemac"
+  persistence:
+    storageClassName: local-path
+  tmp:
+    storageClassName: local-path
+  initContainer:
+    pythonPluginImageInstall:
+      enabled: true
+  automountServiceAccountToken: true

docs/deploy-and-configure/installation/scenario-k8s-deployment/files/graphdb-values.yaml

@@ -0,0 +1,37 @@
+---
+
+# We use a separate, simpler ingress definition
+ingress:
+  enabled: false
+
+license:
+  # Reference to a secret containing 'graphdb.license' file that will be mounted in the GraphDB pod.
+  # The value is processed as a Helm template.
+  existingSecret: "graphdb-license"
+  # File name of the GraphDB license file in the existing license secret.
+  # The default is graphdb.license, but it can be changed to map to a different secret key.
+  licenseFilename: graphdb.license
+
+resources:
+  limits:
+    memory: 4Gi
+    cpu: 2000m
+  requests:
+    memory: 4Gi
+    cpu: 500m
+
+security:
+  enabled: true
+  provisioningUsername: provisioner
+  # bcrypt encrypted password. default: iHaveSuperpowers
+  provisioningPassword: iHaveSuperpowers
+
+persistence:
+  enabled: true
+  volumeClaimTemplate:
+    name: "storage"
+    spec:
+      accessModes: [ "ReadWriteOnce" ]
+      resources:
+        requests:
+          storage: 5Gi

docs/deploy-and-configure/installation/scenario-k8s-deployment/files/keycloak-values.yaml

@@ -0,0 +1,29 @@
+---
+postgres:
+  internal: true
+  provisioning:
+    enabled: true
+    # If true, this will drop the public schema and re-provision the database on every start.
+    force: true
+
+ingress:
+  enabled: true
+  className: nginx
+  annotations:
+    cert-manager.io/common-name: "kc.docker.localhost"
+  hosts:
+    - host: "kc.docker.localhost"
+      paths:
+        - path: /auth
+          pathType: Prefix
+          serviceName: keycloak
+          servicePort: 8080
+  tls:
+    - hosts:
+         - "kc.docker.localhost"
+      secretName:  keycloak-ingress-cert
+
+tests:
+  # Change this to your ingress controller IP in case you are using
+  # kind/docker-desktop/k3d with host.docker.internal or docker.localhost
+  ingressControllerIP: "10.43.82.133"