Merge branch 'main' into add-language-dropdown-for-3.13-and-later

Author: josh-wong

docs/configurations.mdx

@@ -101,7 +101,7 @@ Select a database to see the configurations available for each storage.
 If you're using SQLite3 as a JDBC database, you must set `scalar.db.contact_points` as follows:
 
 ```properties
-scalar.db.contact_points=jdbc:sqlite:<YOUR_DB>.sqlite3?busy_timeout=10000
+scalar.db.contact_points=jdbc:sqlite:<SQLITE_DB_FILE_PATH>?busy_timeout=10000
 ```
 
 Unlike other JDBC databases, [SQLite3 doesn't fully support concurrent access](https://www.sqlite.org/lang_transaction.html). To avoid frequent errors caused internally by [`SQLITE_BUSY`](https://www.sqlite.org/rescode.html#busy), setting a [`busy_timeout`](https://www.sqlite.org/c3ref/busy_timeout.html) parameter is recommended.

docs/design.mdx

@@ -5,9 +5,35 @@ tags:
   - Enterprise Premium
 ---
 
-# ScalarDB Design Document
+# ScalarDB Design
 
-For details about the design and implementation of ScalarDB, please see the following documents, which we presented at the VLDB 2023 conference:
+This document briefly explains the design and implementation of ScalarDB. For what ScalarDB is and its use cases, see [ScalarDB Overview](./overview.mdx).
+
+## Overall architecture
+
+ScalarDB is hybrid transaction/analytical processing (HTAP) middleware that sits in between applications and databases. As shown in the following figure, ScalarDB consists of three components: Core, Cluster, and Analytics. ScalarDB basically employs a layered architecture, so the Cluster and Analytics components use the Core component to interact with underlying databases but sometimes bypass the Core component for performance optimization without sacrificing correctness. Likewise, each component also consists of several layers. 
+
+![ScalarDB architecture](images/scalardb-architecture.png)
+
+## Components
+
+The following subsections explain each component one by one.
+
+### Core
+
+ScalarDB Core, which is provided as open-source software under the Apache 2 License, is an integral part of ScalarDB. Core provides a database manager that has an abstraction layer that abstracts underlying databases and adapters (or shims) that implement the abstraction for each database. In addition, it provides a transaction manager on top of the database abstraction that achieves database-agnostic transaction management based on Scalar's novel distributed transaction protocol called Consensus Commit. Core is provided as a library that offers a simple CRUD interface.
+
+### Cluster
+
+ScalarDB Cluster, which is licensed under a commercial license, is a component that provides a clustering solution for the Core component to work as a clustered server. Cluster is mainly designed for OLTP workloads, which have many small, transactional and non-transactional reads and writes. In addition, it provides several enterprise features such as authentication, authorization, encryption at rest, and fine-grained access control (still under development). Not only does Cluster offer the same CRUD interface as the Core component, but it also offers SQL and GraphQL interfaces. Since Cluster is provided as a container in a Kubernetes Pod, you can increase performance and availability by having more containers.
+
+### Analytics
+
+ScalarDB Analytics, which is licensed under a commercial license, is a component that provides scalable analytical processing for the data managed by the Core component or managed by applications that don’t use ScalarDB. Analytics is mainly designed for OLAP workloads, which have a small number of large, analytical read queries. In addition, it offers a SQL and DataSet API through Spark. Since the Analytics component is provided as a Java package that can be installed on Apache Spark engines, you can increase performance by having more Spark worker nodes.
+
+## See also
+
+For details about the design and implementation of ScalarDB, see the following documents, which were presented at the VLDB 2023 conference:
 
 - **Speakerdeck presentation:** [ScalarDB: Universal Transaction Manager for Polystores](https://speakerdeck.com/scalar/scalardb-universal-transaction-manager-for-polystores-vldb23)
 - **Detailed paper:** [ScalarDB: Universal Transaction Manager for Polystores](https://www.vldb.org/pvldb/vol16/p3768-yamada.pdf)

docs/getting-started-with-scalardb-by-using-kotlin.mdx

@@ -20,19 +20,15 @@ The electronic money application is simplified for this tutorial and isn't suita
 
 ## Prerequisites for this sample application
 
-Because ScalarDB is written in Java, you must have one of the following Java Development Kits (JDKs) installed in your environment:
-
-- [Oracle JDK](https://www.oracle.com/java/technologies/downloads/) 8
-- OpenJDK 8 from [Eclipse Temurin](https://adoptium.net/temurin/releases/), [Amazon Corretto](https://aws.amazon.com/corretto/), or [Microsoft](https://learn.microsoft.com/en-us/java/openjdk/download)
+- OpenJDK LTS version (8, 11, 17, or 21) from [Eclipse Temurin](https://adoptium.net/temurin/releases/)
+- [Docker](https://www.docker.com/get-started/) 20.10 or later with [Docker Compose](https://docs.docker.com/compose/install/) V2 or later
 
 :::note
 
-This sample application only works with Java 8. However, ScalarDB itself works with Java LTS versions, which means that you can use Java LTS versions for your application that uses ScalarDB. For details on the requirements of ScalarDB, such as which Java versions can be used, see [Requirements](./requirements.mdx).
+This sample application has been tested with OpenJDK from Eclipse Temurin. ScalarDB itself, however, has been tested with JDK distributions from various vendors. For details about the requirements for ScalarDB, including compatible JDK distributions, please see [Requirements](./requirements.mdx).
 
 :::
 
-In addition, since you'll be using Docker Compose to run the databases, you must have [Docker](https://www.docker.com/get-started/) 20.10 or later with [Docker Compose](https://docs.docker.com/compose/install/) V2 or later installed.
-
 ## Clone the ScalarDB samples repository
 
 Open **Terminal**, then clone the ScalarDB samples repository by running the following command:

docs/getting-started-with-scalardb.mdx

@@ -20,19 +20,15 @@ Since the focus of the sample application is to demonstrate using ScalarDB, appl
 
 ## Prerequisites for this sample application
 
-Because ScalarDB is written in Java, you must have one of the following Java Development Kits (JDKs) installed in your environment:
-
-- [Oracle JDK](https://www.oracle.com/java/technologies/downloads/) 8
-- OpenJDK 8 from [Eclipse Temurin](https://adoptium.net/temurin/releases/), [Amazon Corretto](https://aws.amazon.com/corretto/), or [Microsoft](https://learn.microsoft.com/en-us/java/openjdk/download)
+- OpenJDK LTS version (8, 11, 17, or 21) from [Eclipse Temurin](https://adoptium.net/temurin/releases/)
+- [Docker](https://www.docker.com/get-started/) 20.10 or later with [Docker Compose](https://docs.docker.com/compose/install/) V2 or later
 
 :::note
 
-This sample application only works with Java 8. However, ScalarDB itself works with Java LTS versions, which means that you can use Java LTS versions for your application that uses ScalarDB. For details on the requirements of ScalarDB, such as which Java versions can be used, see [Requirements](./requirements.mdx).
+This sample application has been tested with OpenJDK from Eclipse Temurin. ScalarDB itself, however, has been tested with JDK distributions from various vendors. For details about the requirements for ScalarDB, including compatible JDK distributions, please see [Requirements](./requirements.mdx).
 
 :::
 
-In addition, since you'll be using Docker Compose to run the databases, you must have [Docker](https://www.docker.com/get-started/) 20.10 or later with [Docker Compose](https://docs.docker.com/compose/install/) V2 or later installed.
-
 ## Clone the ScalarDB samples repository
 
 Open **Terminal**, then clone the ScalarDB samples repository by running the following command:

docs/helm-charts/configure-custom-values-scalardb-cluster.mdx

@@ -373,3 +373,46 @@ scalardbCluster:
       operator: Equal
       value: scalardb-cluster
 ```
+
+### Encryption configurations (optional based on your environment)
+
+You can enable [encryption at rest](https://scalardb.scalar-labs.com/docs/latest/scalardb-cluster/encrypt-data-at-rest/) to protect the data in the backend databases. When you use the encryption feature, you have the following two deployment options:
+
+1. Use HashiCorp Vault (HashiCorp Cloud Platform (HCP) Vault Dedicated) to manage and store the DEKs.
+1. Use ScalarDB Cluster to manage the DEK, and store it in Kubernetes Secrets.
+
+#### Use HashiCorp Vault
+
+You can use HashiCorp Vault (HCP Vault Dedicated) to encrypt data as follows, replacing the contents in the angle brackets as described:
+
+```yaml
+scalardbCluster:
+  scalardbClusterNodeProperties: |
+    ...(omit)...
+    scalar.db.cluster.encryption.enabled=true
+    scalar.db.cluster.encryption.type=vault
+    scalar.db.cluster.encryption.vault.address=https://<FQDN_OR_IP_OF_VAULT_SERVER>:<PORT_OF_VAULT_SERVER>
+    scalar.db.cluster.encryption.vault.token=<TOKEN_OF_VAULT>
+    scalar.db.cluster.encryption.vault.transit_secrets_engine_path=<PATH_TO_TRANSIT_OF_VAULT>
+  encryption:
+    enabled: true
+    type: "vault"
+```
+
+#### Use ScalarDB Cluster and Kubernetes Secrets
+
+You can use ScalarDB Cluster and Kubernetes Secrets to encrypt data as follows, replacing the contents in the angle brackets as described:
+
+```yaml
+scalardbCluster:
+  scalardbClusterNodeProperties: |
+    ...(omit)...
+    scalar.db.cluster.encryption.enabled=true
+    scalar.db.cluster.encryption.type=self
+    scalar.db.cluster.encryption.self.kubernetes.secret.namespace_name=${env:SCALAR_DB_CLUSTER_ENCRYPTION_SELF_KUBERNETES_SECRET_NAMESPACE_NAME}
+  encryption:
+    enabled: true
+    type: "self"
+```
+
+In this case, you don't need to replace `${env:SCALAR_DB_CLUSTER_ENCRYPTION_SELF_KUBERNETES_SECRET_NAMESPACE_NAME}` since the Helm Chart for ScalarDB Cluster automatically sets the namespace information as an environment variable. Because of this, you can keep the value `${env:SCALAR_DB_CLUSTER_ENCRYPTION_SELF_KUBERNETES_SECRET_NAMESPACE_NAME}` as is.

docs/helm-charts/getting-started-scalar-manager.mdx

@@ -3,30 +3,30 @@ tags:
   - Enterprise Option
 ---
 
-# Getting Started with Helm Charts (Scalar Manager)
+# Deploy Scalar Manager
 
-Scalar Manager is a centralized management and monitoring solution for ScalarDB and ScalarDL within Kubernetes cluster environments that allows you to:
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
 
-* Check the availability of ScalarDB or ScalarDL.
-* Schedule or execute pausing jobs that create transactionally consistent periods in the databases used by ScalarDB or ScalarDL.
-* Check the time-series metrics and logs of ScalarDB or ScalarDL through Grafana dashboards.
+[Scalar Manager](../scalar-manager/overview.mdx) is a centralized management and monitoring solution for ScalarDB and ScalarDL within Kubernetes cluster environments that allows you to:
 
-For more details, refer to [Scalar Manager Overview](../scalar-manager/overview.mdx).
+- Check the availability of ScalarDB or ScalarDL.
+- Schedule or execute pausing jobs that create transactionally consistent periods in the databases used by ScalarDB or ScalarDL.
+- Check the time-series metrics and logs of ScalarDB or ScalarDL through Grafana dashboards.
 
-This guide will show you how to deploy and access Scalar Manager on a Kubernetes cluster.
+This guide explains how to deploy and access Scalar Manager on a Kubernetes cluster by using Scalar Helm Charts.
 
-## Assumption
+## Prerequisites
 
-This guide assumes that you are aware of how to deploy ScalarDB or ScalarDL with the [monitoring](getting-started-monitoring.mdx) and [logging](getting-started-logging.mdx) tools to a Kubernetes cluster.
+Before you deploy Scalar Manager, you must do the following:
 
-## Requirement
+- Install the tools mentioned in [Getting Started with Scalar Helm Charts](getting-started-scalar-helm-charts.mdx).
+- Deploy `kube-prometheus-stack` according to the instructions in [Getting Started with Helm Charts (Monitoring using Prometheus Operator)](getting-started-monitoring.mdx).
+- Deploy `loki-stack` according to the instructions in [Getting Started with Helm Charts (Logging using Loki Stack)](getting-started-logging.mdx).
 
-* You must deploy `kube-prometheus-stack` according to the instructions in [Getting Started with Helm Charts (Monitoring using Prometheus Operator)](getting-started-monitoring.mdx).
-* You must deploy `loki-stack` according to the instructions in [Getting Started with Helm Charts (Logging using Loki Stack)](getting-started-logging.mdx).
+## Deployment architecture diagram
 
-## What we create
-
-We will deploy the following components on a Kubernetes cluster as follows.
+The following is an architecture diagram for the components deployed in a Kubernetes cluster.
 
 ```
 +--------------------------------------------------------------------------------------------------+
@@ -60,94 +60,129 @@ We will deploy the following components on a Kubernetes cluster as follows.
 |    |                       |         Kubernetes                                                  |
 +----+-----------------------+---------------------------------------------------------------------+
      |                       |
-  expose to localhost (127.0.0.1) or use load balancer etc to access
+  Expose the environment to localhost (127.0.0.1) or use a load balancer to access it
      |                       |
-  (Access Dashboard through HTTP)
+  (Access the dashboard through HTTP)
      |                       |
 +----+----+             +----+----+
 | Browser | <-(Embed)-- + Browser |
 +---------+             +---------+
 ```
 
-## Step 1. Upgrade the `kube-prometheus-stack` to allow Grafana to be embedded
+## Step 1. Start minikube
+
+Open **Terminal**, and start minikube by running the following command:
+
+```console
+minikube start
+```
+
+## Step 2. Upgrade the `kube-prometheus-stack` to allow Grafana to be embedded
+
+In your custom values file for `kube-prometheus-stack` (for example, `scalar-prometheus-custom-values.yaml`), add the following configurations or revise them if they already exist:
+
+```yaml
+kubeStateMetrics:
+  enabled: true
+
+nodeExporter:
+  enabled: true
+
+kubelet:
+  enabled: true
+
+grafana:
+  grafana.ini:
+    users:
+      default_theme: light
+    security:
+      allow_embedding: true
+    auth.anonymous:
+      enabled: true
+      org_name: "Main Org."
+      org_role: Editor
+```
+
+Then, upgrade the Helm installation by running the following command:
+
+```console
+helm upgrade scalar-monitoring prometheus-community/kube-prometheus-stack -n monitoring -f scalar-prometheus-custom-values.yaml
+```
+
+## Step 3. Set environment variables
 
-1. Add or revise this value to the custom values file (e.g. scalar-prometheus-custom-values.yaml) of the `kube-prometheus-stack`
-   ```yaml
-   kubeStateMetrics:
-     enabled: true
-   nodeExporter:
-     enabled: true
-   kubelet:
-     enabled: true
-   grafana:
-     grafana.ini:
-       users:
-         default_theme: light
-       security:
-         allow_embedding: true
-       auth.anonymous:
-         enabled: true
-         org_name: "Main Org."
-         org_role: Editor
-   ```
+Set environment variables for Scalar Manager by running the following commands, replacing the contents in angle brackets as described:
 
-1. Upgrade the Helm installation
-   ```console
-   helm upgrade scalar-monitoring prometheus-community/kube-prometheus-stack -n monitoring -f scalar-prometheus-custom-values.yaml
-   ```
+```console
+SCALAR_MANAGER_RELEASE_NAME=<ADD_RELEASE_NAME>
+SCALAR_MANAGER_NAMESPACE=<ADD_NAMESPACE>
+SCALAR_MANAGER_CUSTOM_VALUES_FILE=<ADD_PATH_TO_CUSTOM_VALUES_FILE>
+SCALAR_MANAGER_CHART_VERSION=<ADD_CHART_VERSION>
+```
 
-## Step 2. Prepare a custom values file for Scalar Manager
+## Step 4. Prepare a custom values file for Scalar Manager
 
-1. Create an empty .yaml file named `scalar-manager-custom-values.yaml` for `scalar-manager`.
+Prepare a custom values file for Scalar Manager by doing the following:
 
-1. Set the service type to access Scalar Manager. The default value is `ClusterIP`, but if we access using the `minikube tunnel` command or some load balancer, we can set it as `LoadBalancer`.
-   ```yaml
-   service:
-     type: LoadBalancer
-     port: 8000
-   ```
+1. Create an empty file named `scalar-manager-custom-values.yaml`.
+1. Follow the instructions in [Configure a custom values file for Scalar Manager](configure-custom-values-scalar-manager.mdx).
 
-## Step 3. Deploy `scalar-manager`
+## Step 5. Install and deploy `scalar-manager`
 
-1. Deploy the `scalar-manager` Helm Chart.
-   ```console
-   helm install scalar-manager scalar-labs/scalar-manager -f scalar-manager-custom-values.yaml
-   ```
+Install and deploy the `scalar-manager` Helm Chart by running the following command:
 
-## Step 4. Access Scalar Manager
+```console
+helm install ${SCALAR_MANAGER_RELEASE_NAME} scalar-labs/scalar-manager -n ${SCALAR_MANAGER_NAMESPACE} -f ${SCALAR_MANAGER_CUSTOM_VALUES_FILE} --version ${SCALAR_MANAGER_CHART_VERSION}
+```
 
-### If you use minikube
+## Step 6. Access Scalar Manager
 
-1. To expose Scalar Manager's service resource as your `localhost (127.0.0.1)`, open another terminal, and run the `minikube tunnel` command.
-   ```console
-   minikube tunnel
-   ```
+How you access Scalar Manager depends on the tool that you're using for Kubernetes clusters.
 
-1. Open the browser with URL `http://localhost:8000`
+<Tabs groupId="kubernetes-tools" queryString>
+  <TabItem value="minikube" label="minikube" default>
+    To expose Scalar Manager's service resource as your localhost (127.0.0.1), open another terminal, and run the `minikube tunnel` command.
 
-### If you use other Kubernetes than minikube
+    ```console
+    minikube tunnel
+    ```
 
-If you're using a Kubernetes cluster other than minikube, you'll need to access the `LoadBalancer` service according to the manner of each Kubernetes cluster. For example, you'll need to use a load balancer provided by your cloud services provider or use the `kubectl port-forward` command.
+    Then, access Scalar Manager by going to http://localhost:8000.
+  </TabItem>
+  <TabItem value="other-kubernetes-clustering-tools" label="Other Kubernetes clustering tools">
+    If you're using a Kubernetes cluster other than minikube, you'll need to access the `LoadBalancer` service according to the manner of each Kubernetes cluster. For example, you'll need to use a load balancer provided by your cloud services provider or use the `kubectl port-forward` command.
 
 :::note
 
-Scalar Manager will try to detect the external IP of Grafana and then embed Grafana based on the IP. Therefore, you must configure the Grafana service type as `LoadBalancer`, and the external IP must be accessible from your browser.
+Scalar Manager will try to detect the external IP address for Grafana and then embed Grafana based on that IP address. Therefore, you must configure the Grafana service type as `LoadBalancer`, and the external IP address must be accessible from your browser.
 
 :::
-
-## Step 5. Delete Scalar Manager
-1. Uninstall `scalar-manager`
-   ```console
-   helm uninstall scalar-manager
-   ```
+  </TabItem>
+</Tabs>
 
 ## Additional details
 
 This section provides additional details related to configurations and resource discovery.
 
-### Configurations
+### Upgrade the Scalar Manager deployment
+
+To upgrade the deployment of Scalar Manager, run the following command:
+
+```console
+helm upgrade ${SCALAR_MANAGER_RELEASE_NAME} scalar-labs/scalar-manager -n ${SCALAR_MANAGER_NAMESPACE} -f ${SCALAR_MANAGER_CUSTOM_VALUES_FILE} --version ${SCALAR_MANAGER_CHART_VERSION}
+```
+
+### Uninstall Scalar Manager
+
+To uninstall Scalar Manager, run the following command:
+
+```console
+helm uninstall ${SCALAR_MANAGER_RELEASE_NAME} -n ${SCALAR_MANAGER_NAMESPACE}
+```
+
+### Optional Scalar Manager configurations
 
-You can see configurations for Scalar Manager in [Configure a custom values file for Scalar Manager](./configure-custom-values-scalar-manager.mdx)
+For optional configurations that you can set for Scalar Manager, see [Optional configurations](./configure-custom-values-scalar-manager.mdx#optional-configurations)
 
 ### Resource discovery