Add Pipeline to deploy custom agent image for FIPS testing

[michel-laterman]

What does this PR do?

Add a new buildkite pipeline to build a custom agent image and use it in an ECH deployment for testing.

Why is it important?

FIPS integration tests will require a custom agent running in the CFT region.

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

Disruptive User Impact

N/A

[michel-laterman]

buildkite test this

[michel-laterman]

@v1v @pazone, can you take a look at this, we need permissions in order to push a custom image to use in the CFT region

denied: requested access to the resource is denied
--
  | Error: Failed pushing docker image: running "docker image push docker.elastic.co/observability-ci/elastic-agent-fips:git-b84b80343415" failed with exit code 1

[v1v]

| Error: Failed pushing docker image: running "docker image push docker.elastic.co/observability-ci/elastic-agent-fips:git-b84b80343415" failed with exit code 1

Can you share the URL link to the error?

I'm not familiar with the current user and namespace, but as far as I see, those details are stored at https://github.com/elastic/elastic-agent/blob/aa224536eadf49f8b9b962df240c0caa4861970e/.buildkite/hooks/pre-command#l17.

However, I think you need to configure the pre-command hook to run for the new BK pipelines:

• elastic-agent/.buildkite/hooks/pre-command

  Lines 39 to 47 in aa22453

  	if [[ "$BUILDKITE_PIPELINE_SLUG" == "elastic-agent-package" ]]; then
  	if [[ "$BUILDKITE_STEP_KEY" == "package_elastic-agent" ]]; then
  	docker_login
  	fi
  	if [[ "$BUILDKITE_STEP_KEY" == "dra-publish" || "$BUILDKITE_STEP_KEY" == "bk-api-publish-independent-agent" ]]; then
  	release_manager_login
  	fi
  	fi

  is the settings for the elastic-agent-package BK pipeline.

I see you have enabled the pre-command for the new step

[v1v]

Updatecli tests

$ gh pr checkout 8035
$ git checkout -b test/fips-ech-updatecli
$ git push origin test/fips-ech-updatecli
$ # modify .ci/updatecli/values.d/scm.yml to use my fork
$ GITHUB_TOKEN=$(gh auth token) \                      
  GITHUB_ACTOR=v1v \
  BRANCH_NAME=test/fips-ech-updatecli updatecli apply \ 
    --config .ci/updatecli/updatecli-bump-vm-images.yml \
    --values .ci/updatecli/values.d/scm.yml

diff --git a/.ci/updatecli/values.d/scm.yml b/.ci/updatecli/values.d/scm.yml
index 2708262728..c01cb89d50 100644
--- a/.ci/updatecli/values.d/scm.yml
+++ b/.ci/updatecli/values.d/scm.yml
@@ -1,8 +1,8 @@
 scm:
   enabled: true
-  owner: elastic
+  owner: v1v

Produced v1v#9

[v1v]

LGTM, with some minor comments

[v1v]

LGTM, the questions I asked earlier are not blockers, but likely something to be done in follow-up

[v1v]

LGTM, the questions I asked earlier are not blockers, but likely something to be done in follow-up

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 4ef376f

Failed CI Steps

• Serverless integration test
• Serverless integration test
• x86_64:sudo: fleet

History

• 💛 Build #21990 was flaky aa44551
• 💔 Build #21979 failed 33647be
• 💔 Build #21971 failed 33647be
• 💔 Build #21922 failed 645c8b6
• 💛 Build #21907 was flaky afb5d73

cc @michel-laterman

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube