@W-19704630: [MSDK Android] Login Activity Tests Failing Only In CI - Ignore For 13.1

[JohnsonEricAtSalesforce]

🎸 Ready For Review 🥁

This really simple non-functional update ignores a test that used to pass both for developer workstations and our CI. That test now consistently fails in CI and seems to be causing the CI run to take much longer than before. Also, it is blocking the generation of code coverage reports. The previous is unfortunate since this test was created primarily for code coverage, alas.

I'll create a work item for the new release to diagnose the issue and restore this test.

[JohnsonEricAtSalesforce]

Though I've marked this ready for review, we do need to observe how the CI run performs. If it doesn't succeed for some as yet unknown reason, perhaps we'll learn more about the original failure.

[JohnsonEricAtSalesforce]

For historical context, we don't take ignoring a test lightly. This test does seem to be failing for non-functional reasons in the runner and without a failure that would indicate any potential customer impact. The detailed log passed to Github Actions is simply FAIL with line number 1 (which is the copyright line, so probably a default). The detailed log is:

 libs/test/SalesforceSDKTest/src/com/salesforce/androidsdk/ui/LoginActivityTest.kt:1 | FAILED508.107s
  ℹ️ Posting with conclusion 'failure' to https://github.com/forcedotcom/SalesforceMobileSDK-Android/pull/2766 (sha: 3776e1290d1c1a010b142cdd6278390741a2acc2)
🚀 Publish results
  ℹ️ - SalesforceSDK Test Results | api_35_test_result.xml - 350 tests run, 349 passed, 0 skipped, 1 failed.
     🧪 - libs/test/SalesforceSDKTest/src/com/salesforce/androidsdk/ui/LoginActivityTest.kt | FAILED
  ℹ️ - SalesforceSDK Test Results | api_35_test_result.xml - Creating check (Annotations: 1)
  Error: ❌ Tests reported 1 failures

However, that's in the "Test Report" section of the detailed log. There's no failure reported anywhere else in the Github Actions test execution logs.

The Google Test Lab report does show the test failure, but again with no detail. Only the text FAILED. There is a message that reads 'The test timed out. The test ran longer than its maximum allowed duration, and was stopped.', which could indicate that this Activity test format simply isn't compatible with Test Lab.

I'll do an additional measure of due diligence to see if this could be resolved for 13.1, but it seems it may be less valuable than the release given this test does pass under other conditions.

[JohnsonEricAtSalesforce]

I also confirmed that ActivityScenario is supported by Test Lab and no active reports of it timing out seem to be applicable to our scenario or the specific device being tested. We may be able to pick this up later and use some possibly time-consuming process of elimination test development to identify the root cause and a happy path for this test. This test might also be able to move our of an activity scenario as well with a little re-work.

[JohnsonEricAtSalesforce]

I was wondering if this would be the case: Ignoring the test only resulted in another downstream test failing due to seemingly Test Lab specific reasons. If Test Lab is undergoing some regression or outage, which itself would be delightfully ironic, we may have to rely on our local test runs for 13.1 if that's acceptable.

[JohnsonEricAtSalesforce]

Since the issue was broader than just this once test suite, a more immediate fix is being pursued. We can close this one.