Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/api-guide.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -580,7 +580,7 @@ And if you need to check if a value of a column is null, you can use the `isNull
boolean isNull = result.isNull("<COLUMN_NAME>");
```

For more details, see the `Result` page in the [Javadoc](https://javadoc.io/doc/com.scalar-labs/scalardb/3.13.1/index.html) of the version of ScalarDB that you're using.
For more details, see the `Result` page in the [Javadoc](https://javadoc.io/doc/com.scalar-labs/scalardb/3.14.0/index.html) of the version of ScalarDB that you're using.

###### Execute `Get` by using a secondary index

Expand Down
10 changes: 5 additions & 5 deletions docs/configurations.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -203,11 +203,11 @@ For details about client configurations, see the [ScalarDB Cluster client config

The following are additional configurations available for ScalarDB:

| Name | Description | Default |
|------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------|
| `scalar.db.metadata.cache_expiration_time_secs` | ScalarDB has a metadata cache to reduce the number of requests to the database. This setting specifies the expiration time of the cache in seconds. | `-1` (no expiration) |
| `scalar.db.active_transaction_management.expiration_time_millis` | ScalarDB maintains ongoing transactions, which can be resumed by using a transaction ID. This setting specifies the expiration time of this transaction management feature in milliseconds. | `-1` (no expiration) |
| `scalar.db.default_namespace_name` | The given namespace name will be used by operations that do not already specify a namespace. | |
| Name | Description | Default |
|------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------|
| `scalar.db.metadata.cache_expiration_time_secs` | ScalarDB has a metadata cache to reduce the number of requests to the database. This setting specifies the expiration time of the cache in seconds. If you specify `-1`, the cache will never expire. | `60` |
| `scalar.db.active_transaction_management.expiration_time_millis` | ScalarDB maintains ongoing transactions, which can be resumed by using a transaction ID. This setting specifies the expiration time of this transaction management feature in milliseconds. | `-1` (no expiration) |
| `scalar.db.default_namespace_name` | The given namespace name will be used by operations that do not already specify a namespace. | |

## Placeholder usage

Expand Down
30 changes: 28 additions & 2 deletions docs/design.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,35 @@ tags:
- Enterprise Premium
---

# ScalarDB Design Document
# ScalarDB Design

For details about the design and implementation of ScalarDB, please see the following documents, which we presented at the VLDB 2023 conference:
This document briefly explains the design and implementation of ScalarDB. For what ScalarDB is and its use cases, see [ScalarDB Overview](./overview.mdx).

## Overall architecture

ScalarDB is hybrid transaction/analytical processing (HTAP) middleware that sits in between applications and databases. As shown in the following figure, ScalarDB consists of three components: Core, Cluster, and Analytics. ScalarDB basically employs a layered architecture, so the Cluster and Analytics components use the Core component to interact with underlying databases but sometimes bypass the Core component for performance optimization without sacrificing correctness. Likewise, each component also consists of several layers.

![ScalarDB architecture](images/scalardb-architecture.png)

## Components

The following subsections explain each component one by one.

### Core

ScalarDB Core, which is provided as open-source software under the Apache 2 License, is an integral part of ScalarDB. Core provides a database manager that has an abstraction layer that abstracts underlying databases and adapters (or shims) that implement the abstraction for each database. In addition, it provides a transaction manager on top of the database abstraction that achieves database-agnostic transaction management based on Scalar's novel distributed transaction protocol called Consensus Commit. Core is provided as a library that offers a simple CRUD interface.

### Cluster

ScalarDB Cluster, which is licensed under a commercial license, is a component that provides a clustering solution for the Core component to work as a clustered server. Cluster is mainly designed for OLTP workloads, which have many small, transactional and non-transactional reads and writes. In addition, it provides several enterprise features such as authentication, authorization, encryption at rest, and fine-grained access control (still under development). Not only does Cluster offer the same CRUD interface as the Core component, but it also offers SQL and GraphQL interfaces. Since Cluster is provided as a container in a Kubernetes Pod, you can increase performance and availability by having more containers.

### Analytics

ScalarDB Analytics, which is licensed under a commercial license, is a component that provides scalable analytical processing for the data managed by the Core component or managed by applications that don’t use ScalarDB. Analytics is mainly designed for OLAP workloads, which have a small number of large, analytical read queries. In addition, it offers a SQL and DataSet API through Spark. Since the Analytics component is provided as a Java package that can be installed on Apache Spark engines, you can increase performance by having more Spark worker nodes.

## See also

For details about the design and implementation of ScalarDB, see the following documents, which were presented at the VLDB 2023 conference:

- **Speakerdeck presentation:** [ScalarDB: Universal Transaction Manager for Polystores](https://speakerdeck.com/scalar/scalardb-universal-transaction-manager-for-polystores-vldb23)
- **Detailed paper:** [ScalarDB: Universal Transaction Manager for Polystores](https://www.vldb.org/pvldb/vol16/p3768-yamada.pdf)
2 changes: 1 addition & 1 deletion docs/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ tags:

# ScalarDB

import { CardRowAbout, CardRowGettingStarted, CardRowSamples, CardRowDevelop, CardRowDeploy, CardRowManage, CardRowReference } from '/src/components/Cards/3.13';
import { CardRowAbout, CardRowGettingStarted, CardRowSamples, CardRowDevelop, CardRowDeploy, CardRowManage, CardRowReference } from '/src/components/Cards/3.14';

ScalarDB is a cross-database HTAP engine. It achieves ACID transactions and real-time analytics across diverse databases to simplify the complexity of managing multiple databases.

Expand Down
7 changes: 3 additions & 4 deletions docs/quick-start-overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,11 @@ ScalarDB Cluster is available only in the Enterprise edition.

## Try running analytical queries through ScalarDB Analytics

In this sub-category, you can see tutorials on how to run analytical queries over the databases that you write through ScalarDB by using a component called ScalarDB Analytics.
ScalarDB Analytics currently targets only ScalarDB-managed databases, updated through ScalarDB transactions, but will target non-ScalarDB-managed databases in the future.
In this sub-category, you can see tutorials on how to run analytical queries over the databases that you write to by using a component called ScalarDB Analytics. ScalarDB Analytics targets both ScalarDB-managed databases, which are updated through ScalarDB transactions, and non-ScalarDB-managed databases.

:::note

- ScalarDB Analytics with PostgreSQL is available only under the Apache 2 License and doesn't require a commercial license.
- ScalarDB Analytics with Spark is in private preview.
- ScalarDB Analytics with Spark is in public preview.

:::
:::
Loading