Feat/kinesis e2e infra by ekjotmultani · Pull Request #6693 · aws-amplify/amplify-flutter

ekjotmultani · 2026-02-27T20:27:29Z

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

cadivus

This should be integrated into infra_gen2.

cadivus · 2026-03-03T12:44:56Z

.github/workflows/kinesis_e2e.yaml

+name: Kinesis E2E Tests
+on:
+  pull_request:
+    paths:
+      - "packages/kinesis/**"
+      - ".github/workflows/kinesis_e2e.yaml"
+  workflow_dispatch:


How does this differ so much from all the other e2e tests that we need to implement everything custom?

this custom code is outdated sorry, but still kinesis E2E is fundamentally different, it needs OIDC + Secrets Manager for credentials rather than fetch_backends + amplify pull

it needs OIDC + Secrets Manager for credentials rather than fetch_backends + amplify pull

But fetch_backends already uses OIDC + Secrets Manager. The only thing it does on top is run tool/pull_test_backend.sh to download amplify_outputs.dart from S3.

For the credentials, the Android PR shows the better approach:
Include auth as a category and grant Kinesis permissions to the authenticated IAM role:

// Android: aws-kinesis/infra/amplify/backend.ts const backend = defineBackend({ auth }); backend.auth.resources.authenticatedUserIamRole.addToPrincipalPolicy( new PolicyStatement({ actions: ['kinesis:PutRecord', 'kinesis:PutRecords', 'kinesis:DescribeStream'], resources: [stream.streamArn], }) );

(link)

This is exactly the same pattern the Flutter repo already uses for Analytics/Pinpoint, which also isn't a "native" Amplify category but piggybacks on auth + custom CDK resources + backend.addOutput().

Also, using some tokens in the app is not a real e2e test. This is not what customers most likely do. Using Cognito instead of some "injected" credentials is way closer to what a customer would to.
And this way, we need the amplify_outputs.dart.

I would say that these dart only next version packages could be decoupled from the cognito pattern, but for consistency I have updated them to match the other platforms and our own packages

cadivus · 2026-03-03T12:45:18Z

.github/workflows/kinesis_e2e.yaml

+      - name: Install native sqlite3
+        run: sudo apt-get update && sudo apt-get install -y libsqlite3-dev


Why is this needed? For building on Android?

cadivus · 2026-03-03T12:45:37Z

.github/workflows/kinesis_e2e.yaml

+            smithy_aws:
+              path: ../../smithy/smithy_aws
+          EOF
+


Why don't we use the aft bootstrapping? Why this custom solution?

cadivus · 2026-03-03T12:47:01Z

infra/lib/stack.ts

+    AuthIntegrationTestStack,
+    AuthIntegrationTestStackEnvironmentProps,


Why these infra changes? Isn't infra unrelated to this PR?

cadivus · 2026-03-03T12:49:21Z

infra-gen2/backends/kinesis/main/amplify/backend.ts

+new secretsmanager.Secret(kinesisStack, "KinesisE2ESecret", {
+  secretName: "kinesis-e2e",
+  description: "Kinesis E2E test credentials and resource names",
+  secretObjectValue: {
+    ACCESS_KEY_ID: cdk.SecretValue.unsafePlainText(accessKey.accessKeyId),
+    SECRET_ACCESS_KEY: accessKey.secretAccessKey,
+    STREAM_NAME: cdk.SecretValue.unsafePlainText(stream.streamName),
+    DELIVERY_STREAM_NAME: cdk.SecretValue.unsafePlainText(
+      deliveryStream.deliveryStreamName!,
+    ),
+  },
+});


What are these secrets needed for? If they are for fetching the e2e backends, I don't think this is the right place.

not for fetching but for creating

but I am pushing out the updated branch, this whole commit was outdated and mistakenly pushed

Adds the foundational package with minimal constructs needed for the Kinesis client libraries: - AmplifyException base class - AWSCredentialsProvider<T> and AWSCredentials sealed hierarchy - Logger interface with AmplifyLogger implementation - LogLevel enum - Result<T, E> sealed type

Provides V2CredentialsProviderBridge to bridge aws_common (V2) credentials to amplify_foundation_dart (V3) credentials. Shared by kinesis client packages to avoid duplicating bridge logic.

…ncies

…plify backend Provisions Kinesis Data Stream, Firehose delivery stream, IAM user, and Secrets Manager secret via an Amplify Gen 2 backend using backend.createStack() to access CDK directly — no auth or other Amplify categories needed. Removes the standalone KinesisTestStack and KinesisTestResources construct from infra/, along with references in the main integ stack and GitHub stack.

…rt-only E2E

- Revert unrelated whitespace changes to infra/lib/stack.ts and infra/lib/github/github.ts - Replace manual pubspec_overrides.yaml with aft bootstrap - Update workflow to match repo conventions (action versions, submodule init, log_cw_metric_wrapper, timeout, defaults) - Remove IAM user, access key, and Secrets Manager secret from backend.ts (credentials managed outside stack)

…lid category list in log_cw_metric so that\nthe kinesis package working directory is recognized by the CI\nmetric logging step."

…e same pattern as Analytics/Pinpoint and the Android KDS\nlibrary: include auth (Cognito) in the backend and grant the\nauthenticated IAM role permissions to Kinesis/Firehose resources.\n\nThis replaces the custom Secrets Manager credential injection with\nthe standard fetch_backends workflow pattern, making E2E tests use\nCognito-authenticated credentials like real customers would.\n\n- Add auth resource with pre-sign-up auto-confirm trigger\n- Grant authenticatedUserIamRole Kinesis and Firehose permissions\n- Rewrite E2E workflow to use fetch_backends composite action\n- Remove custom KINESIS_E2E secret fetching from workflow"

…ory.kinesis so deploy_gen2.dart can deploy the kinesis\nbackend and upload amplify_outputs.dart to S3. The kinesis backend\nshares its config with both aws_kinesis_datastreams and\naws_amazon_firehose packages via sharedOutputs."

jvh-aws · 2026-03-06T12:57:51Z

infra-gen2/backends/kinesis/main/amplify/backend.ts

+  enforceSSL: true,
+});
+
+// --- IAM role for Firehose → S3 ---


I think this is unrelated to Kinesis. Can we handle this separately and remove from this PR?

ekjotmultani changed the base branch from main to feat/amplify-foundation-dart-minimal February 27, 2026 20:28

ekjotmultani marked this pull request as ready for review February 27, 2026 20:32

ekjotmultani requested a review from a team as a code owner February 27, 2026 20:32

cadivus requested changes Feb 27, 2026

View reviewed changes

cadivus reviewed Mar 3, 2026

View reviewed changes

cadivus requested changes Mar 3, 2026

View reviewed changes

cadivus force-pushed the feat/amplify-foundation-dart-minimal branch from 000928f to 9b582eb Compare March 3, 2026 15:57

ekjotmultani force-pushed the feat/kinesis-e2e-infra branch from 0a451d4 to a04df03 Compare March 4, 2026 05:56

cadivus force-pushed the feat/amplify-foundation-dart-minimal branch 2 times, most recently from 1f9c2b1 to 93c5331 Compare March 5, 2026 16:39

Base automatically changed from feat/amplify-foundation-dart-minimal to main March 5, 2026 16:48

ekjotmultani and others added 19 commits March 5, 2026 23:36

feat: add amplify_foundation_dart_bridge package

b339e0f

Provides V2CredentialsProviderBridge to bridge aws_common (V2) credentials to amplify_foundation_dart (V3) credentials. Shared by kinesis client packages to avoid duplicating bridge logic.

Fix LICENSE files

af3c758

Review fixes

54f0c90

Fix dependencies

2ce15c8

Satisfy the linter

d8d40e0

Move packages

304a2dd

Refactor logging

4d7d190

Fix repository URLs

8fb3ad6

Improve tests

e82ea81

Make LogMessage final

7be49a6

Give StaticCredentials a const constructor

1b31ab2

Make constructors const

3560da8

Credentials: handling of nullable fields

a3b4aa1

feat(infra): add Kinesis E2E test resources CDK stack and CI workflow

aa4d047

fix(ci): generate pubspec_overrides.yaml for unpublished path depende…

cc2c187

…ncies

chore(infra-gen2): update package-lock.json for kinesis workspace

697aa06

fix(ci): install native sqlite3 and strip sqlite3_flutter_libs for Da…

fbfa966

…rt-only E2E

ekjotmultani added 5 commits March 5, 2026 23:37

chore(ci): remove unnecessary sed hack for sqlite3_flutter_libs

d5ce6c8

chore: add kinesis to CW metric categories\n\nAdd 'kinesis' to the va…

de55ab5

…lid category list in log_cw_metric so that\nthe kinesis package working directory is recognized by the CI\nmetric logging step."

chore: remove orphaned foundation files from rebase"

f8687db

ekjotmultani force-pushed the feat/kinesis-e2e-infra branch from a04df03 to 18f1e9f Compare March 6, 2026 08:01

ekjotmultani added 3 commits March 6, 2026 00:18

fix: add kinesis case to Category switch in AmplifyClassImpl

d9bc849

chore: add kinesis category to CloudWatch metric logger

bceff3b

jvh-aws reviewed Mar 6, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat/kinesis e2e infra#6693

Feat/kinesis e2e infra#6693
ekjotmultani wants to merge 27 commits intomainfrom
feat/kinesis-e2e-infra

ekjotmultani commented Feb 27, 2026

Uh oh!

cadivus left a comment

Uh oh!

cadivus Mar 3, 2026

Uh oh!

ekjotmultani Mar 4, 2026

Uh oh!

cadivus Mar 4, 2026

Uh oh!

ekjotmultani Mar 6, 2026

Uh oh!

cadivus Mar 3, 2026

Uh oh!

cadivus Mar 3, 2026

Uh oh!

cadivus Mar 3, 2026

Uh oh!

cadivus Mar 3, 2026

Uh oh!

ekjotmultani Mar 4, 2026

Uh oh!

ekjotmultani Mar 4, 2026

Uh oh!

jvh-aws Mar 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		- name: Install native sqlite3
		run: sudo apt-get update && sudo apt-get install -y libsqlite3-dev

		AuthIntegrationTestStack,
		AuthIntegrationTestStackEnvironmentProps,

Conversation

ekjotmultani commented Feb 27, 2026

Uh oh!

cadivus left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants