Skip to content

Commit 990ee56

Browse files
authored
Merge pull request #32 from keboola/proofreading16
Proofreading16
2 parents 4970b69 + 748c60a commit 990ee56

File tree

2 files changed

+55
-55
lines changed

2 files changed

+55
-55
lines changed

CONFIG.md

Lines changed: 38 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
# Configuring in Keboola Connection UI
1+
# Configuring in Keboola UI
22

3-
## Sample scan mode
3+
## Sample Scan Mode
44

55

66
```json
@@ -41,7 +41,7 @@
4141
```
4242

4343

44-
## Sample query mode
44+
## Sample Query Mode
4545

4646

4747
```json
@@ -84,7 +84,7 @@
8484
```
8585

8686

87-
## Sample query mode (with secondary index)
87+
## Sample Query Mode (with Secondary Index)
8888

8989

9090
```json
@@ -127,47 +127,47 @@
127127

128128
- `db`: DynamoDB instance connection options
129129
- `endpoint`: `https://dynamodb.REGION.amazonaws.com`
130-
- `accessKeyId`: Access key id
130+
- `accessKeyId`: Access key ID
131131
- `#secretAccessKey`: Secret access key (will be encrypted)
132132
- `regionName`: Region
133133

134-
- `exports`: array of exports
135-
- `id`: unique numeric identifier of export
136-
- `name`: unique string identifier of export (base table will be named after it)
137-
- `table`: name of the table to export from
138-
- `index`: (optional) name of the index to export from
139-
- `enabled` (optional, default: `true`): if export is enabled or not (there has to be at least one enabled export)
140-
- `incremental`: if load of tables to storage will be incremental
141-
- `primaryKey`: primary key to set on imported table, defined as array
142-
- `mode`: (optional): enum(scan|query) reading mode from dynamoDb - default is scan
143-
- `keyConditionExpression`: (required): provide a specific value for the partition key
144-
- `expressionAttributeValues`: (required): values that can be substituted in an expression
145-
- `expressionAttributeNames`: (optional): substitution tokens for attribute names in an expression. You can use placeholder `$` instead of `#` see sample
146-
- `dateFilter` (optional): how to filter scanned documents (only for scan mode)
147-
- `field`: field name in document by which you want to filter
148-
- `format`: date format (e.g. `Y-m-d` for date or `Y` for year)
149-
- `value`: date string from which date value will be created (e.g. `-2 days`)
150-
- `limit` (optional): how many documents you want to export
151-
- `mapping`: how to map fields in document to CSV columns
134+
- `exports`: An array of export configurations.
135+
- `id`: A unique numeric identifier for the export.
136+
- `name`: A unique string identifier for the export (the base table will be named after it).
137+
- `table`: The name of the table to export from.
138+
- `index` (optional): The name of the index to export from.
139+
- `enabled` (optional, default: `true`): Specifies whether the export is enabled. (At least one export must be enabled.)
140+
- `incremental`: Determines whether tables loads to Storage will be incremental.
141+
- `primaryKey`: The primary key to set on the imported table, defined as an array.
142+
- `mode` (optional): enum(scan|query): The reading mode from DynamoDB. Default: `scan`.
143+
- `keyConditionExpression` (required): Specifies a specific value for the partition key.
144+
- `expressionAttributeValues` (required): Defines values that can be substituted in an expression.
145+
- `expressionAttributeNames` (optional): Substitution tokens for attribute names in an expression. You can use the placeholder `$` instead of `#` (see sample).
146+
- `dateFilter` (optional): Defines how to filter scanned documents. *(Applicable only for scan mode.)*
147+
- `field`: The document field name used for filtering.
148+
- `format`: The date format (e.g., `Y-m-d` for a full date or `Y` for a year).
149+
- `value`: The relative date string from which date value will be calculated (e.g., `-2 days`).
150+
- `limit` (optional): Specifies how many documents to export.
151+
- `mapping`: Defines how document fields should be mapped to CSV columns.
152152

153153

154154
### `dateFilter`
155155

156-
*Note: To be able to use `dateFilter` and incremental loads, make sure your database (or index)
157-
contains field by which documents can be filtered. E.g. add `creationDate` to every document you create*
156+
***Note:** To use `dateFilter` and incremental loads, make sure that your database (or index)
157+
contains a field that can be used for filtering documents. For example, add a `creationDate` field to every document you create.*
158158

159-
Extractor uses `Scan` operation to selecting documents from DynamoDB.
159+
The extractor uses the `Scan` operation to select documents from DynamoDB.
160160

161-
You can specify `dateFilter` parameter to filter documents you want export. Filter condition is
162-
composed from 3 fields: `field`, `format` and `value`.
161+
You can specify the `dateFilter` parameter to filter the documents you want export. The filter condition is
162+
composed of three fields: `field`, `format`, and `value`.
163163

164-
The `value` field is passed to [**strtotime**](https://secure.php.net/strtotime) function. Then the
165-
`format` and `value` fields are passed to [**date**](https://secure.php.net/date) function to create
166-
final value which will be used to filter documents. Something like `date($format, strtotime($value))`.
164+
The `value` field is passed to the [**strtotime**](https://secure.php.net/strtotime) function. Then, the
165+
`format` and `value` fields are passed to the [**date**](https://secure.php.net/date) function to generate the
166+
final value, which will be used to filter documents. The equivalent function call is: `date($format, strtotime($value))`.
167167

168168
#### Example
169169

170-
(for date `2018-03-13 18:00:00`)
170+
(For date `2018-03-13 18:00:00`)
171171

172172
|field|format|value|composed condition|
173173
|---|---|---|---|
@@ -178,13 +178,13 @@ final value which will be used to filter documents. Something like `date($format
178178

179179
### `mapping`
180180

181-
- [php-csvmap](https://github.com/keboola/php-csvmap) library is used to parse exported documents
182-
- in most cases you'll be happy with simple `"some.path.key": "destination"` mapping
183-
- for advanced usages please check documentation of mapping sections of in
184-
[MongoDB Extractor](https://help.keboola.com/extractors/database/mongodb/mapping/) or
185-
[Generic Extractor](https://developers.keboola.com/extend/generic-extractor/configuration/config/mappings/)
181+
- The [php-csvmap](https://github.com/keboola/php-csvmap) library is used to parse exported documents.
182+
- In most cases, a simple mapping like `"some.path.key": "destination"` will suffice.
183+
- For advanced use cases, please refer mapping sections of in:
184+
- [MongoDB Extractor](https://help.keboola.com/extractors/database/mongodb/mapping/)
185+
- [Generic Extractor](https://developers.keboola.com/extend/generic-extractor/configuration/config/mappings/)
186186

187187
## Links
188188

189189
- [Scanning a Table](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/SQLtoNoSQL.ReadData.Scan.html)
190-
- [DynamoDB Scan operation](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Scan.html)
190+
- [DynamoDB Scan Operation](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Scan.html)

README.md

Lines changed: 17 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,18 @@
11
# DynamoDB Extractor
22

3-
Docker application for exporting data from Amazon DynamoDB.
3+
This Docker application exports data from Amazon DynamoDB to Keboola.
44

55
## Configuration
66

7-
Sample configuration and its description can be found [here](/CONFIG.md).
7+
A sample configuration and its description can be found [here](/CONFIG.md).
88

99
## Output
1010

11-
After successful extraction there are several CSV files which contains exported data. First output
12-
file is named after `name` parameter in export configuration. Other files are named after destination
13-
parameter in mapping section.
11+
After a successful extraction, several CSV files containing exported data will be generated.
12+
- The first output file is named after the `name` parameter in the export configuration.
13+
- Additional files are named according to the destination parameter in the mapping section.
1414

15-
Also, there is manifest file for each of the export.
15+
A manifest file is also created for each export.
1616

1717
## Development
1818

@@ -21,26 +21,26 @@ Requirements:
2121
- Docker Engine: `~1.12`
2222
- Docker Compose: `~1.8`
2323

24-
Application is prepared for run in container, you can start development same way:
24+
This application is designed to run in a Docker container. To start development, follow these steps:
2525

2626
1. Clone this repository: `git clone git@github.com:keboola/dynamodb-extractor.git`
27-
2. Change directory: `cd dynamodb-extractor`
27+
2. Navigate to the project directory: `cd dynamodb-extractor`
2828
3. Build services: `docker compose build`
29-
4. Run tests `docker compose run --rm app composer ci`
29+
4. Run tests: `docker compose run --rm app composer ci`
3030

31-
After seeing all tests green, continue:
31+
Once all tests pass successfully, continue with:
3232

33-
1. Run service: `docker compose run --rm app bash`
33+
1. Run the service: `docker compose run --rm app bash`
3434
2. Create tables/indexes and load sample data: `php tests/fixtures/init.php`
35-
3. Write tests and code
35+
3. Write tests and develop the required code.
3636
4. Run tests: `composer tests`
3737

38-
To simulate real run:
38+
To simulate a real run:
3939

40-
1. Create data dir: `mkdir -p data`
41-
2. Follow configuration sample and create `config.json` file and place it to your data directory (`data/config.json`)
42-
3. Simulate real run (with entrypoint command): `php ./src/app.php run ./data`
40+
1. Create a data directory: `mkdir -p data`
41+
2. Follow the configuration sample and create a `config.json` file in the data directory (`data/config.json`).
42+
3. Simulate a real run using the entrypoint command: `php ./src/app.php run ./data`
4343

4444
## License
4545

46-
MIT licensed, see [LICENSE](./LICENSE) file.
46+
This project is MIT licensed. See the [LICENSE](./LICENSE) file for details.

0 commit comments

Comments
 (0)