Consider running E2E tests on 8.x and 9.0/main

[mashhurs]

Description

With recent branch separation (8.x) and CI re-arrangement, E2E tests stopped running CI related jobs. The fun fact is CI defines to run from always main and in 8.x there is no run_e2e_test.sh script which executes python E2E scripts (following error is from run_e2e_test.sh).

ERROR: Could not open requirements file: [Errno 2] No such file or directory: '.buildkite/scripts/e2e/requirements.txt'

FYI: we have same setup with build CI and it is running as expected: https://buildkite.com/elastic/logstash-filter-elastic-integration-build/builds/438#_

Acceptance criteria

The situations should consider on PR merge and weekly run.
Based on the fact we have now 8.x and main branches for 8.x and 9.0 compatibility, consider running E2E tests for 8.x and 9.0/main with a greed status.

[mashhurs]

Completed with https://github.com/elastic/ingest-dev/issues/4686