Skip to content

Conversation

typotter
Copy link
Collaborator

@typotter typotter commented Aug 29, 2024

🎟️ Fixes FF-3096 towards FF-3085

πŸ‘―β€β™‚οΈ Related PRs

Motivation

Changes are often made to the sdk-test-data repository to capture new behaviours, bugs and edge cases. When these changes are pushed to main, the SDKs are cloned locally (locally to the github action running) and their respective tests are run. These tests are set up and run by copies of the SDK test workflows - see sdk-test-data workflow. There are a number of limitations to this setup:

  • Test steps are copied from the SDK’s respective workflows; changes to the SDK test workflows need to be replicated in sdk-test-data and this is not obvious to devs
  • The SDK tests are not able to run against in-flight changes to sdk-test-data
  • When new test-data is committed that breaks an SDK, that breakage is not surfaced in the SDK’s repository until some action triggers its testing workflow (on demand, on pull-request, etc.).

Description of Changes

This change
πŸš€ - Each SDK's testing workflow is enhanced into a reusable workflow, exposing parameters for SDK branch and the sdk-test-data branch to use in testing.

  • The test workflow runs using the main branch of sdk-test-data on all main pushes and Pull Requests using the PR's branch

External to this Change
sdk-test-data get two testing workflows.

  1. ♻️ "Local Testing"- For all pull request changes, the "Local Testing" workflow calls the reusable SDK workflows, test results are recorded only in the sdk-test-data action. This is run using the main SDK branch and the "current" branch of workflow, i.e. the pull request branch. (SDK repo does not see/is not notified of test failures during PR lifecycle, only on push to main)
  2. ♻️ πŸ§‘β€πŸ’» "Remote Testing" - On all pushes to the main branch of sdk-test-data, the SDK testing workflows are triggered to run within their respective repositories, alerting all subscribers of failed runs (repo is red/green).

Copy link

@aarsilv aarsilv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving, but give the on: section of the workflow a look

Comment on lines 8 to 9
push:

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be restricted to main branch?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a change in syntax to accommodate multiple triggers. That being said, there's not a lot of consistency across the SDKs here on this, so let's set the bar here.
We want tests to run when a) push to main, b) PR opened, c) PR updated
So, you are right that we can restrict push to branches: main as pull_request includes the synchronize event which is triggered by pushing commits to a branch under a PR.

on:
push:

pull_request:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to include '**/*' as the paths for this?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe so. paths provides a way to filter the event trigger and **/* matches everything which is the same as unfiltered.

@typotter typotter requested a review from aarsilv September 5, 2024 17:32
@typotter typotter assigned aarsilv and unassigned typotter Sep 5, 2024
@typotter typotter assigned typotter and unassigned aarsilv Sep 6, 2024
@typotter typotter merged commit 5afce90 into main Sep 6, 2024
1 check passed
@typotter typotter deleted the tp/workflows/remote branch September 6, 2024 04:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants