Skip to content

Proposal: Allow Android real-world tests to pass CI even when expiredΒ #183

@complexspaces

Description

@complexspaces

We run into the case time and time again where CI starts failing after a period of inactivity in the repository. The root cause of this is our real-world test cases: they include certificates from real CAs, which therefore have real (usually ~90 day) expiry periods.

Most of our testing infrastructure for the various platforms is robust enough to handle this by faking all of the associated timestamps used in the trust evaluation so that little-to-nothing bitrots. Android is the odd one out here, for reasons documented in #59.

I would like to propose, until time can be made to redo/improve the Android verifier implementation, that we allow real-world tests that fail to be marked as passing and instead add a warning to the GitHub action output:

  1. The code is compiled for #[cfg(target_os = "android")]
  2. The expected test case result is not ::Expired.
  3. The actual test case result is ::Expired.

To implement this all I believe we would need is some Android-specific error handling logic in the real world result handling. It would simply println! a warning string in the format GHA expects.

No changes to the Android verifier Kotlin should be required, as we already check the timestamp for obviously revoked certs before running any of the platform-provided X.509 chain verification (which is where the spurious errors come from). This means that when we explicitly pass in a timestamp that's too old for a certificate, we will still get the expected failure result.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions