Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
641 changes: 641 additions & 0 deletions docs/data-loader.mdx

Large diffs are not rendered by default.

20 changes: 10 additions & 10 deletions docs/scalardb-data-loader/getting-started-export.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,37 +8,37 @@ displayed_sidebar: docsEnglish

# Getting started with Export

This document explains how you can get started with the ScalarDB Data Loader Export function.
This document explains how you can get started with ScalarDB Data Loader Export function.

## Features

The ScalarDB Data Loader allows you to export data in the following formats:
ScalarDB Data Loader allows you to export data in the following formats:

- JSON
- JSONLines
- JSON Lines
- CSV

Each export will run a ScalarDB scan operation based on the provided CLI arguments when running data loader.
Each export will run a ScalarDB scan operation based on the provided CLI arguments when running Data Loader.

## Usage

The data loader export function can be started with the following minimal configuration:
Data Loader export function can be started with the following minimal configuration:

```console
./scalardb-data-loader export --config scalardb.properties --namespace namespace --table tableName
```



- --config: the path to the scalardb connection properties file
- --config: the path to the ScalarDB connection properties file
- --namespace: the namespace of the table that contains the data
- --table: name of the table that contains the data

By default, the data loader will create the output file in the working directory if the `--output-file` argument is omitted as well.
By default, Data Loader will create the output file in the working directory if the `--output-file` argument is omitted as well.

### Command-line flags

Here is a list of flags (options) that can be used with the scalardb data loader.
Here is a list of flags (options) that can be used with ScalarDB Data Loader.

| Flag | Description | Usage |
| ----------------- | ------------------------------------------------------------ | ------------------------------------------------------ |
Expand All @@ -52,11 +52,11 @@ Here is a list of flags (options) that can be used with the scalardb data loader
| --start-exclusive | Is the scan start exclusive or not. If omitted, the default value is `false`. This flag is only applicable to `--key` | `scalardb-data-loader --start-exclusive` |
| --end | Clustering key to mark scan end. This flag is only applicable to `--key`. | `scalardb-data-loader --end columnName=value` |
| --end-exclusive | Is the scan start exclusive or not. If omitted, the default value is `false`. This flag is only applicable to `--key` | `scalardb-data-loader --end-exclusive` |
| --limit | Limit the results of the scan. If omitted, the default value is `0` which means their is no limit. | `scalardb-data-loader --limit 1000` |
| --limit | Limit the results of the scan. If omitted, the default value is `0` which means there is no limit. | `scalardb-data-loader --limit 1000` |
| --output-file | The name and path of the output file. If omitted, the tool will save the file in the current folder with the following name format:<br />`export_namespace.tableName_timestamp.json` or `export_namespace.tableName_timestamp.csv`<br /><br />The ouput folder needs to exists. The dataloader does not create the output folder for you. | `scalardb-data-loader --output-file ./out/output.json` |
| --format | The output format. By default `json` is selected. | `scalardb-data-loader --format json` |
| --metadata | When set to true the transaction metadata is included in the export. By default this is set to `false` | `scalardb-data-loader --metadata` |
| --delimiter | The delimiter used in CSV files. Default value is `;` | `scalardb-data-loader --delimiter ;` |
| --no-headers | Exclude header row in CSV file. Default is `false` | `scalardb-data-loader --no-headers` |
| --threads | Thread count for concurrent processing | `scalardb-data-loader --threads 500` |
| --threads | Thread count for concurrent processing. The default value is the number of available processors. | `scalardb-data-loader --threads 500` |

Loading
Loading