Skip to content
This repository was archived by the owner on Jul 16, 2024. It is now read-only.

Commit ff6aa37

Browse files
ijemmyvgkowski
andauthored
feat: Add batch replayer (#227)
* Add BatchReplayer and PartitionedDataset classes BatchReplayer has the same function as the current data-generator. However, it adds the missing file header and use Step Function + Lambda instead of other Athena. This is more cost-effective, but it requires data to be pre-partitioned with a manifest file. * Restore projen tasks * feat: Split lambda functions into 2 steps and write output files in paraellel * fix: Fix bug that Map index isn't passed to Lambda correctly * feat: write dataframe into multiple files that doesn't exceed given max size * fix: remove custom workspace setting * fix: main merge Co-authored-by: Vincent Gromakowski <gromav@amazon.com>
1 parent 38dd20b commit ff6aa37

File tree

17 files changed

+3382
-2164
lines changed

17 files changed

+3382
-2164
lines changed

core/.projen/deps.json

Lines changed: 53 additions & 53 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

core/.projenrc.js

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ const project = new AwsCdkConstructLibrary({
2222
'analytics',
2323
],
2424

25-
cdkVersion: '1.130',
25+
cdkVersion: '1.134',
2626
defaultReleaseBranch: 'main',
2727
license: 'MIT-0',
2828
name: 'aws-analytics-reference-architecture',

0 commit comments

Comments
 (0)