chore(ci): refactor e2e tests to use Foliage MONGOSH-1907 #2290

gagik · 2024-12-10T12:43:27Z

Much of the work was done by parsing the YAML of the original typed out variants in .evergreen.yaml.in and then applying the following transformations:

// This is a JSON-converted representation of these lines in .evergreen.yml:
// https://github.com/mongodb-js/mongosh/pull/2290/files#diff-a2d1c28777c30d9782121327f86cf2b70f35cf62fe0f1879e59e915929615118L1657-L1948
let a: any[] = [ { name: 'linux_x64_build',
    display_name: 'RHEL 7.0 x64 (build)',
    run_on: 'rhel70-build',
    expansions: { executable_os_id: 'linux-x64' },
    tags: [ 'nightly-driver' ],
    tasks: [ { name: 'compile_artifact' } ] },
  { name: 'linux_x64_build_rhel8',
    display_name: 'RHEL 8.0 x64 (build)',
    run_on: 'rhel80-build',
    expansions: { executable_os_id: 'linux-x64' },
    tags: [ 'nightly-driver' ],
    tasks: [ { name: 'compile_artifact' } ] },
  { name: 'linux_x64_build_openssl11',
    display_name: 'RHEL 7.0 x64 (build, shared OpenSSL 1.1)',
    run_on: 'rhel70-build',
    expansions: 
     { executable_os_id: 'linux-x64-openssl11',
       mongosh_shared_openssl: 'openssl11' },
    tags: [ 'nightly-driver' ],
    tasks: [ { name: 'compile_artifact' } ] },
  { name: 'linux_x64_build_openssl11_rhel8',
    display_name: 'RHEL 8.0 x64 (build, shared OpenSSL 1.1)',
    run_on: 'rhel80-build',
    expansions: 
     { executable_os_id: 'linux-x64-openssl11',
       mongosh_shared_openssl: 'openssl11' },
    tags: [ 'nightly-driver' ],
    tasks: [ { name: 'compile_artifact' } ] },
  { name: 'linux_x64_build_openssl3',
    display_name: 'RHEL 7.0 x64 (build, shared OpenSSL 3)',
    run_on: 'rhel70-build',
    expansions: 
     { executable_os_id: 'linux-x64-openssl3',
       mongosh_shared_openssl: 'openssl3' },
    tags: [ 'nightly-driver' ],
    tasks: [ { name: 'compile_artifact' } ] },
  { name: 'linux_x64_build_openssl3_rhel8',
    display_name: 'RHEL 8.0 x64 (build, shared OpenSSL 3)',
    run_on: 'rhel80-build',
    expansions: 
     { executable_os_id: 'linux-x64-openssl3',
       mongosh_shared_openssl: 'openssl3' },
    tags: [ 'nightly-driver' ],
    tasks: [ { name: 'compile_artifact' } ] },
  { name: 'linux_arm64_build',
    display_name: 'Amazon 2 arm64 (build)',
    run_on: 'amazon2-arm64-large',
    expansions: { executable_os_id: 'linux-arm64' },
    tasks: [ { name: 'compile_artifact' } ] },
  { name: 'linux_arm64_build_openssl11',
    display_name: 'Amazon 2 arm64 (build, shared OpenSSL 1.1)',
    run_on: 'amazon2-arm64-large',
    expansions: 
     { executable_os_id: 'linux-arm64-openssl11',
       mongosh_shared_openssl: 'openssl11' },
    tasks: [ { name: 'compile_artifact' } ] },
  { name: 'linux_arm64_build_openssl3',
    display_name: 'Amazon 2 arm64 (build, shared OpenSSL 3)',
    run_on: 'amazon2-arm64-large',
    expansions: 
     { executable_os_id: 'linux-arm64-openssl3',
       mongosh_shared_openssl: 'openssl3' },
    tasks: [ { name: 'compile_artifact' } ] },
  { name: 'linux_ppc64le_build',
    display_name: 'RHEL 8 PPC (build)',
    run_on: 'rhel8-power-small',
    expansions: { executable_os_id: 'linux-ppc64le' },
    tasks: [ { name: 'compile_artifact' } ] },
  { name: 'linux_s390x_build',
    display_name: 'RHEL 7 s390x (build)',
    run_on: 'rhel7-zseries-large',
    expansions: { executable_os_id: 'linux-s390x' },
    tasks: [ { name: 'compile_artifact' } ] },
  { name: 'e2e_rhel70_x64',
    display_name: 'RHEL 7.0 x64 (E2E Tests)',
    run_on: 'rhel70-large',
    tags: [ 'nightly-driver' ],
    tasks: [ { name: 'e2e_tests_linux_x64_70x' } ] },
  { name: 'e2e_rhel76_x64',
    display_name: 'RHEL 7.6 x64 (E2E Tests)',
    run_on: 'rhel76-large',
    tags: [ 'nightly-driver' ],
    tasks: [ { name: 'e2e_tests_linux_x64_70x' } ] },
  { name: 'e2e_rhel80_x64',
    display_name: 'RHEL 8.0 x64 (E2E Tests)',
    run_on: 'rhel80-small',
    tags: [ 'nightly-driver' ],
    tasks: [ { name: 'e2e_tests_linux_x64' } ] },
  { name: 'e2e_rhel90_x64',
    display_name: 'RHEL 9.0 x64 (E2E Tests)',
    run_on: 'rhel90-small',
    expansions: { disable_openssl_shared_config_for_bundled_openssl: true },
    tags: [ 'nightly-driver' ],
    tasks: [ { name: 'e2e_tests_linux_x64_70x' } ] },
  { name: 'e2e_rhel93_x64',
    display_name: 'RHEL 9.3 x64 (E2E Tests)',
    run_on: 'rhel93-small',
    expansions: { disable_openssl_shared_config_for_bundled_openssl: true },
    tags: [ 'nightly-driver' ],
    tasks: [ { name: 'e2e_tests_linux_x64' } ] },
  { name: 'e2e_rhel83_x64',
    display_name: 'RHEL 8.3 x64 (E2E Tests, FIPS-available OS)',
    run_on: 'rhel83-fips',
    tags: [ 'nightly-driver' ],
    tasks: 
     [ { name: 'e2e_tests_linux_x64' },
       { name: 'e2e_tests_linux_x64_openssl11' },
       { name: 'e2e_tests_linux_x64_openssl11_fips' } ] },
  { name: 'e2e_rhel93_fips_x64',
    display_name: 'RHEL 9.3 x64 (E2E Tests, FIPS-available OS)',
    run_on: 'rhel93-fips',
    tags: [ 'nightly-driver' ],
    tasks: 
     [ { name: 'e2e_tests_linux_x64' },
       { name: 'e2e_tests_linux_x64_openssl3' },
       { name: 'e2e_tests_linux_x64_openssl3_fips' } ] },
  { name: 'e2e_ubuntu1804_x64',
    display_name: 'Ubuntu 18.04 x64 (E2E Tests)',
    run_on: 'ubuntu1804-large',
    tags: [ 'nightly-driver' ],
    tasks: [ { name: 'e2e_tests_linux_x64_60x' } ] },
  { name: 'e2e_ubuntu2004_x64',
    display_name: 'Ubuntu 20.04 x64 (E2E Tests)',
    run_on: 'ubuntu2004-small',
    tags: [ 'nightly-driver' ],
    tasks: 
     [ { name: 'e2e_tests_linux_x64' },
       { name: 'e2e_tests_linux_x64_openssl11' } ] },
  { name: 'e2e_ubuntu2204_x64',
    display_name: 'Ubuntu 22.04 x64 (E2E Tests)',
    run_on: 'ubuntu2204-small',
    tags: [ 'nightly-driver' ],
    tasks: 
     [ { name: 'e2e_tests_linux_x64' },
       { name: 'e2e_tests_linux_x64_openssl3' } ] },
  { name: 'e2e_ubuntu2404_x64',
    display_name: 'Ubuntu 24.04 x64 (E2E Tests)',
    run_on: 'ubuntu2404-small',
    tags: [ 'nightly-driver' ],
    tasks: 
     [ { name: 'e2e_tests_linux_x64' },
       { name: 'e2e_tests_linux_x64_openssl3' } ] },
  { name: 'e2e_debian10_x64',
    display_name: 'Debian 10 x64 (E2E Tests)',
    run_on: 'debian10-small',
    tags: [ 'nightly-driver' ],
    tasks: 
     [ { name: 'e2e_tests_linux_x64_60x' },
       { name: 'e2e_tests_linux_x64_openssl11_60x' } ] },
  { name: 'e2e_debian11_x64',
    display_name: 'Debian 11 x64 (E2E Tests)',
    run_on: 'debian11-small',
    tags: [ 'nightly-driver' ],
    tasks: 
     [ { name: 'e2e_tests_linux_x64_70x' },
       { name: 'e2e_tests_linux_x64_openssl11_70x' } ] },
  { name: 'e2e_amazon2_x64',
    display_name: 'Amazon Linux 2 x64 (E2E Tests)',
    run_on: 'amazon2-large',
    tags: [ 'nightly-driver' ],
    tasks: [ { name: 'e2e_tests_linux_x64_70x' } ] },
  { name: 'e2e_amazon2023_x64',
    display_name: 'Amazon Linux 2023 x64 (E2E Tests)',
    run_on: 'amazon2023.0-small',
    expansions: { disable_openssl_shared_config_for_bundled_openssl: true },
    tags: [ 'nightly-driver' ],
    tasks: [ { name: 'e2e_tests_linux_x64' } ] },
  { name: 'e2e_suse12_x64',
    display_name: 'SLES 12 x64 (E2E Tests)',
    run_on: 'suse12-sp5-large',
    tags: [ 'nightly-driver' ],
    tasks: [ { name: 'e2e_tests_linux_x64_70x' } ] },
  { name: 'e2e_suse15_x64',
    display_name: 'SLES 15 x64 (E2E Tests)',
    run_on: 'suse15sp4-small',
    tags: [ 'nightly-driver' ],
    tasks: [ { name: 'e2e_tests_linux_x64' } ] },
  { name: 'e2e_ubuntu1804_arm64',
    display_name: 'Ubuntu 18.04 arm64 (E2E Tests)',
    run_on: 'ubuntu1804-arm64-large',
    tasks: [ { name: 'e2e_tests_linux_arm64_60x' } ] },
  { name: 'e2e_ubuntu2004_arm64',
    display_name: 'Ubuntu 20.04 arm64 (E2E Tests)',
    run_on: 'ubuntu2004-arm64-small',
    tasks: 
     [ { name: 'e2e_tests_linux_arm64' },
       { name: 'e2e_tests_linux_arm64_openssl11' } ] },
  { name: 'e2e_ubuntu2204_arm64',
    display_name: 'Ubuntu 22.04 arm64 (E2E Tests)',
    run_on: 'ubuntu2204-arm64-small',
    tasks: 
     [ { name: 'e2e_tests_linux_arm64' },
       { name: 'e2e_tests_linux_arm64_openssl3' } ] },
  { name: 'e2e_ubuntu2404_arm64',
    display_name: 'Ubuntu 24.04 arm64 (E2E Tests)',
    run_on: 'ubuntu2404-arm64-small',
    tasks: 
     [ { name: 'e2e_tests_linux_arm64' },
       { name: 'e2e_tests_linux_arm64_openssl3' } ] },
  { name: 'e2e_amazon2_arm64',
    display_name: 'Amazon Linux 2 arm64 (E2E Tests)',
    run_on: 'amazon2-arm64-large',
    tasks: [ { name: 'e2e_tests_linux_arm64_70x' } ] },
  { name: 'e2e_amazon2023_arm64',
    display_name: 'Amazon Linux 2023 arm64 (E2E Tests)',
    run_on: 'amazon2023.0-arm64-small',
    expansions: { disable_openssl_shared_config_for_bundled_openssl: true },
    tasks: [ { name: 'e2e_tests_linux_arm64' } ] },
  { name: 'e2e_rhel82_arm64',
    display_name: 'RHEL 8.2 arm64 (E2E Tests)',
    run_on: 'rhel82-arm64-small',
    tasks: [ { name: 'e2e_tests_linux_arm64' } ] },
  { name: 'e2e_rhel90_arm64',
    display_name: 'RHEL 9.0 arm64 (E2E Tests)',
    run_on: 'rhel90-arm64-small',
    expansions: { disable_openssl_shared_config_for_bundled_openssl: true },
    tasks: 
     [ { name: 'e2e_tests_linux_arm64_70x' },
       { name: 'e2e_tests_linux_arm64_openssl3_70x' } ] },
  { name: 'e2e_rhel93_arm64',
    display_name: 'RHEL 9.3 arm64 (E2E Tests)',
    run_on: 'rhel93-arm64-small',
    expansions: { disable_openssl_shared_config_for_bundled_openssl: true },
    tasks: 
     [ { name: 'e2e_tests_linux_arm64' },
       { name: 'e2e_tests_linux_arm64_openssl3' } ] },
  { name: 'e2e_rhel8_ppc64le',
    display_name: 'RHEL 8 PPC (E2E Tests)',
    run_on: 'rhel8-power-small',
    tasks: [ { name: 'e2e_tests_linux_ppc64le' } ] },
  { name: 'e2e_rhel7_s390x',
    display_name: 'RHEL 7 s390x (E2E Tests)',
    run_on: 'rhel7-zseries-large',
    tasks: [ { name: 'e2e_tests_linux_s390x_60x' } ] },
  { name: 'e2e_rhel8_s390x',
    display_name: 'RHEL 8 s390x (E2E Tests)',
    run_on: 'rhel8-zseries-small',
    tasks: [ { name: 'e2e_tests_linux_s390x' } ] },
  { name: 'e2e_macos14_x64',
    display_name: 'MacOS 14 x64 (E2E Tests)',
    run_on: 'macos-14',
    expansions: { executable_os_id: 'darwin-x64' },
    tasks: [ { name: 'e2e_tests_darwin_x64' } ] },
  { name: 'e2e_macos14_arm64',
    display_name: 'MacOS 14 arm64 (E2E Tests)',
    run_on: 'macos-14-arm64',
    expansions: { executable_os_id: 'darwin-arm64' },
    tasks: [ { name: 'e2e_tests_darwin_arm64' } ] } ];


const reformedA = []

for (var i = 0 ; i < a.length; i++) {
    const variant = a[i];
    const {run_on: runOn, name: variantName, display_name: displayName, tags} = variant;
    const { executable_os_id: executableOsId, disable_openssl_shared_config_for_bundled_openssl: disabledOpenSslSharedConfig } = variant.expansions ?? {};
    let tasks = a[i].tasks as {name: string}[];
    for (var task of tasks) {
        const {name: taskName} = task;
        let options: {
            displayName: string;
            name?: string;
            sharedOpenSsl?: string;
            mVersion?: '7.0.x' | '6.0.x' | 'stable';
            runOn: string;
            isBuild?: boolean;
            executableOsId?: string;
            fips?: boolean;
            disabledOpenSslSharedConfig?: boolean;
            tags?: string[];
        } = {
            displayName: displayName.split(' (')[0],
            runOn,
        };

        if (tags) options.tags = tags;

        if (variantName.includes('build')) options.isBuild = true;

        if (variantName.includes('_build')) options.name = variantName.replace(/linux_(.*)_build/, 'linux_$1');

        if (taskName.includes('fips')) options.fips = true;

        if (executableOsId) options.executableOsId = executableOsId;

        if (disabledOpenSslSharedConfig) options.disabledOpenSslSharedConfig = disabledOpenSslSharedConfig;

        if (taskName.includes('openssl3') || variantName.includes('openssl3')) options.sharedOpenSsl = 'openssl3';

        if (taskName.includes('openssl11') || variantName.includes('openssl11')) options.sharedOpenSsl = 'openssl11';

        if (taskName.startsWith('e2e_tests_linux_x64')) options.executableOsId = 'linux-x64';
        if (!options.executableOsId && taskName.search(/linux|amazon|rhel|/) != -1) options.executableOsId = taskName.includes('arm64') ? 'linux-arm64' : 'linux-x64';
        
        if (taskName.includes('70x')) {
            options.mVersion = '7.0.x';
        } else if (taskName.includes('60x')) {
            options.mVersion = '6.0.x';
        } else {
            options.mVersion = 'stable';
        }
        if (options.executableOsId == undefined) throw new Error("what");

        reformedA.push(options);
    }
}

console.log(reformedA);

…oliage

.evergreen/evergreen.yml.in

config/release-package-matrix.js

.evergreen/evergreen.yml.in

…oliage

gagik added 9 commits December 13, 2024 01:05

WIP: refactor e2e tests for foliage

98441b4

WIP

78fc9d1

use an array structure for everything

b264735

WIP

4c3da64

use explicit false

3328ab2

update to use same compile_artifact

2b91023

Share all build variants

b168637

delete redundant perf tests

9a878c4

remove build from e2e variant names

aa0e4d4

gagik force-pushed the gagik/e2e-foliage branch from acb7d05 to aa0e4d4 Compare December 13, 2024 00:05

gagik added 5 commits December 13, 2024 01:15

add tags

0deb771

Merge branch 'main' of github.com:mongodb-js/mongosh into gagik/e2e-f…

4cd6692

…oliage

fix openssl versioning

e394c30

remove from evergreen build variant

947c47a

remove double openssl

2108539

gagik changed the title ~~WIP: refactor e2e tests for foliage~~ chore(ci): refactor e2e tests to use foliage Dec 16, 2024

gagik changed the title ~~chore(ci): refactor e2e tests to use foliage~~ chore(ci): refactor e2e tests to use Foliage MONGOSH-1907 Dec 16, 2024

gagik added 4 commits December 16, 2024 16:12

add default values for manually typed variants

0fa58f9

move everything to the array

c6548c7

Merge branch 'main' of github.com:mongodb-js/mongosh into gagik/e2e-f…

208c92c

…oliage

add missing big sur test

3334933

gagik commented Dec 16, 2024

View reviewed changes

.evergreen/evergreen.yml.in Outdated Show resolved Hide resolved

config/release-package-matrix.js Outdated Show resolved Hide resolved

gagik marked this pull request as ready for review December 16, 2024 16:39

gagik added 5 commits December 17, 2024 10:40

add ssl expansions

169d7d2

pass as variable

eb4aedb

use compile build variant and fix macos build

56f4d97

derive executable OS ID from name

3cc584e

fix darwin typo

e540069

gagik requested a review from addaleax December 17, 2024 11:25

addaleax reviewed Dec 17, 2024

View reviewed changes

.evergreen/evergreen.yml.in Outdated Show resolved Hide resolved

.evergreen/evergreen.yml.in Outdated Show resolved Hide resolved

.evergreen/evergreen.yml.in Show resolved Hide resolved

.evergreen/evergreen.yml.in Outdated Show resolved Hide resolved

addaleax approved these changes Dec 17, 2024

View reviewed changes

.evergreen/evergreen.yml.in Outdated Show resolved Hide resolved

gagik added 4 commits December 17, 2024 18:04

separate build and move files

c650f8e

Merge branch 'main' of github.com:mongodb-js/mongosh into gagik/e2e-f…

10baa36

…oliage

rename to build variants

26e52fe

fix broken tests

3b6e787

gagik merged commit b0f8e40 into main Dec 18, 2024
130 of 134 checks passed

gagik deleted the gagik/e2e-foliage branch December 18, 2024 10:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

chore(ci): refactor e2e tests to use Foliage MONGOSH-1907 #2290

chore(ci): refactor e2e tests to use Foliage MONGOSH-1907 #2290

Uh oh!

gagik commented Dec 10, 2024 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

chore(ci): refactor e2e tests to use Foliage MONGOSH-1907 #2290

chore(ci): refactor e2e tests to use Foliage MONGOSH-1907 #2290

Uh oh!

Conversation

gagik commented Dec 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gagik commented Dec 10, 2024 •

edited

Loading