[Proposal] Update E2E tests to build and use dapr/dapr from source code

[mukundansundar]

Describe the proposal

The Dapr CLI E2E tests are intended to tests the execution of dapr CLI commands and their effects.
For example dapr init installs a particular version of dapr and that is what the E2E tests verify.

An interesting scenario that is also there is when a new flag is added to daprd, that needs to be added to the CLI dapr run also.

In this case the E2E tests for dapr run will fail since it uses a daprd binary which is the previously released stable version of Dapr. In that case dapr run tries to pass the flag onto to daprd in the E2E test and it fails with the error flag provided but not defined

Example: flag provided but not defined: -api-log-level

See error

But for testing dapr init flow, we still need to test only if the stable version of daprd and dashboard are downloaded and installed.

But for the command like dapr run, argument can be made that it needs to be tested with the latest built code from dapr/dapr master. Similar arguments can be made for dapr dashboard and such. Otherwise contributors would have to wait for the stable release of the enhancement made in dapr/dapr to be in stable release phase for it to be echoed in CLI. This is with respect to only commands and arguments that are proxied over to daprd or dashboard.

A solution would be to have the current E2E tests for dapr init as such and once that is done for any further commands, get the source code for dapr/dapr and build and use the binary from that. (Similar to how a DAPR_REF is specified for Java_SDK as an override to build and use the specified commit). In the CLI E2E tests, we would need to have the binary built. Complete dapr init test and verify; once that is done replace the installed daprd binary within ~/.dapr/bin with the built from source edge binary and run the tests for the other commands on that. This is for the self hosted scenarios.

@yaron2 @artursouza Any thoughts on this?

Release Note

RELEASE NOTE:

[pravinpushkar]

@mukundansundar Have two separate flows for testing with stable version as well as with built from master. Thoughts?

[mukundansundar]

@mukundansundar Have two separate flows for testing with stable version as well as with built from master. Thoughts?

That can be tested for certain commands like dapr run, dapr publish, dapr invoke etc. But certain commands like dapr init, dapr upgrade only need to be tested with the latest stable release/ RC releases during release endgame.

[mukundansundar]

@artursouza @yaron2 thoughts?

[mukundansundar]

@artursouza @yaron2 thoughts on this?

[dapr-bot]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.