Skip to content

Commit cff925e

Browse files
sheylatrudojpayne3506Copilot
authored
ci: Update ACN Pipeline for Compliance (#3597)
* Edit Pipeline Trigger * ci: E2E work * Update Variables to Generated Values * fixup! Update Variables to Generated Values * Use New Pipeline Files Using new files to prevent existing pipelines from failing. * fixup! Use New Pipeline Files * fixup! Use New Pipeline Files * fixup! Use New Pipeline Files * fixup! Use New Pipeline Files * fixup! Use New Pipeline Files * fixup! Use New Pipeline Files * Update Dockerfiles * Update Dockerfiles * fixup! Update Dockerfiles * fixup! Use New Pipeline Files * fixup! Use New Pipeline Files * fixup! Use New Pipeline Files * Update .pipelines/build/utils/rename-dockerfile-references.steps.yaml Co-authored-by: Copilot <[email protected]> Signed-off-by: sheylatrudo <[email protected]> * Update .pipelines/build/utils/rename-dockerfile-references.steps.yaml Co-authored-by: Copilot <[email protected]> Signed-off-by: sheylatrudo <[email protected]> * Update .pipelines/singletenancy/dualstack-overlay/dualstackoverlay-e2e.steps.yaml Co-authored-by: John Payne <[email protected]> Signed-off-by: sheylatrudo <[email protected]> * Address PR Comments * fixup! Use New Pipeline Files * fixup! Use New Pipeline Files * fixup! Address PR Comments * fixup! Address PR Comments * fixup! Address PR Comments * Remove Dockerfile Changes for Testing * fixup! Remove Dockerfile Changes for Testing * Update .pipelines/singletenancy/cilium-dualstack-overlay/cilium-dualstackoverlay-e2e.steps.yaml Signed-off-by: John Payne <[email protected]> * Update .pipelines/singletenancy/cilium-nodesubnet/cilium-nodesubnet-e2e.steps.yaml Signed-off-by: John Payne <[email protected]> * Update .pipelines/singletenancy/cilium-overlay-withhubble/cilium-overlay-e2e.steps.yaml Signed-off-by: John Payne <[email protected]> * Update .pipelines/singletenancy/cilium-overlay/cilium-overlay-e2e.steps.yaml Signed-off-by: John Payne <[email protected]> * fixup! Address PR Comments * fixup! Address PR Comments * fixup! Address PR Comments * Add IPV6_IMAGE_REPO_PATH to Tests * Update .pipelines/cni/cilium/cilium-scale-test.yaml Co-authored-by: John Payne <[email protected]> Signed-off-by: sheylatrudo <[email protected]> --------- Signed-off-by: sheylatrudo <[email protected]> Signed-off-by: John Payne <[email protected]> Co-authored-by: jpayne3506 <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: John Payne <[email protected]>
1 parent c5b409c commit cff925e

File tree

59 files changed

+2553
-1688
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

59 files changed

+2553
-1688
lines changed

‎.config/.gdnsuppress‎

Lines changed: 0 additions & 26 deletions
This file was deleted.

‎.config/credScanSuppressions.json‎

Lines changed: 0 additions & 4 deletions
This file was deleted.

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ ipam-*.xml
1010

1111
# Environment
1212
.vscode/*
13+
**/*.sw?
1314

1415
# Coverage
1516
*.out

‎.pipelines/OneBranch.NonOfficial.CrossPlat.yaml‎

Lines changed: 0 additions & 28 deletions
This file was deleted.

‎.pipelines/OneBranch.Official.CrossPlat.yaml‎

Lines changed: 0 additions & 27 deletions
This file was deleted.

‎.pipelines/build/binaries.jobs.yaml‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
parameters:
2+
- name: binaries
3+
type: jobList
4+
5+
6+
jobs:
7+
- ${{ each job_data in parameters.binaries }}:
8+
- ${{ if eq(job_data.templateContext.action, 'build') }}:
9+
- job: binaries_${{ job_data.job }}
10+
displayName: "Build Binary - ${{ job_data.displayName }} -"
11+
strategy: ${{ job_data.strategy }}
12+
pool:
13+
type: linux
14+
${{ if eq(job_data.job, 'linux_arm64') }}:
15+
hostArchitecture: arm64
16+
17+
variables:
18+
ob_outputDirectory: $(Build.ArtifactStagingDirectory)/out
19+
ob_artifactSuffix: _$(artifact)
20+
ob_git_checkout: false
21+
REPO_ROOT: $(Build.SourcesDirectory)/${{ job_data.templateContext.repositoryArtifact }}
22+
${{ if eq(job_data.job, 'linux_amd64') }}:
23+
LinuxContainerImage: 'onebranch.azurecr.io/linux/ubuntu-2204:latest'
24+
ARCH: amd64
25+
OS: linux
26+
${{ elseif eq(job_data.job, 'windows_amd64') }}:
27+
LinuxContainerImage: 'onebranch.azurecr.io/linux/ubuntu-2204:latest'
28+
ARCH: amd64
29+
OS: windows
30+
${{ elseif eq(job_data.job, 'linux_arm64') }}:
31+
ob_enable_qemu: true
32+
ARCH: arm64
33+
OS: linux
34+
35+
steps:
36+
- task: DownloadPipelineArtifact@2
37+
inputs:
38+
targetPath: $(Build.SourcesDirectory)/${{ job_data.templateContext.repositoryArtifact }}
39+
artifact: '${{ job_data.templateContext.repositoryArtifact }}'
40+
41+
- template: binary.steps.yaml
42+
parameters:
43+
target: $(name)
44+
os: $(OS)
45+
arch: $(ARCH)

‎.pipelines/build/binary.steps.yaml‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
parameters:
2+
- name: target
3+
type: string
4+
5+
- name: os
6+
type: string
7+
8+
- name: arch
9+
type: string
10+
11+
12+
steps:
13+
- task: GoTool@0
14+
inputs:
15+
version: '$(GOVERSION)'
16+
17+
- bash: |
18+
# Ubuntu
19+
if [[ -f /etc/debian_version ]];then
20+
sudo apt-get update -y
21+
if [[ $GOARCH =~ amd64 ]]; then
22+
sudo apt-get install -y llvm clang linux-libc-dev linux-headers-generic libbpf-dev libc6-dev nftables iproute2 gcc-multilib tree
23+
for dir in /usr/include/x86_64-linux-gnu/*; do
24+
sudo ln -sfn "$dir" /usr/include/$(basename "$dir")
25+
done
26+
27+
elif [[ $GOARCH =~ arm64 ]]; then
28+
sudo apt-get install -y llvm clang linux-libc-dev linux-headers-generic libbpf-dev libc6-dev nftables iproute2 gcc-aarch64-linux-gnu tree
29+
for dir in /usr/include/aarch64-linux-gnu/*; do
30+
sudo ln -sfn "$dir" /usr/include/$(basename "$dir")
31+
done
32+
fi
33+
# Mariner
34+
else
35+
sudo tdnf install -y llvm clang libbpf-devel nftables tree
36+
for dir in /usr/include/aarch64-linux-gnu/*; do
37+
if [[ -d $dir ]]; then
38+
sudo ln -sfn "$dir" /usr/include/$(basename "$dir")
39+
elif [[ -f "$dir" ]]; then
40+
sudo ln -Tsfn "$dir" /usr/include/$(basename "$dir")
41+
fi
42+
done
43+
fi
44+
displayName: "Install Binary Pre-Reqs"
45+
workingDirectory: $(ACN_DIR)
46+
continueOnError: true
47+
env:
48+
GOOS: ${{ parameters.os }}
49+
GOARCH: ${{ parameters.arch }}
50+
51+
- bash: |
52+
make "$TARGET"
53+
displayName: "Build Binary - ${{ parameters.target }}"
54+
workingDirectory: $(ACN_DIR)
55+
env:
56+
REPO_ROOT: $(ACN_DIR)
57+
TARGET: ${{ parameters.target }}
58+
GOOS: ${{ parameters.os }}
59+
GOARCH: ${{ parameters.arch }}
60+
61+
- script: |
62+
SOURCE_DIR="./output"
63+
TARGET_DIR="$BUILD_ARTIFACTSTAGINGDIRECTORY"/out
64+
tree "$SOURCE_DIR"
65+
66+
mkdir -p "$TARGET_DIR"
67+
find "$SOURCE_DIR" -name '*.tgz*' -print -exec mv -t "$TARGET_DIR"/ {} +
68+
find "$SOURCE_DIR" -name '*.zip' -print -exec mv -t "$TARGET_DIR"/ {} +
69+
70+
tree "$TARGET_DIR"
71+
displayName: "Prepare Artifacts"
72+
workingDirectory: $(ACN_DIR)

‎.pipelines/build/generate-manifest.steps.yaml‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
parameters:
2+
- name: platforms
3+
type: object
4+
default: []
5+
6+
7+
steps:
8+
- bash: |
9+
set -e
10+
MANIFEST_DATA=$(echo "$IMAGE_PLATFORM_DATA" | \
11+
jq -r '.[] |
12+
.args = [ (.platform | split("/")[0]), (.platform | split("/")[1]) ] |
13+
.args = [ ("--os " + .args[0] ), ("--arch " + .args[1] ) ] |
14+
if .osVersion then .args += ["--os-version " + .osVersion] else . end |
15+
{ image: .imageReference, annotate: .args }' | \
16+
jq -rcs)
17+
echo >&2 "##vso[task.setvariable variable=MANIFEST_JSON;isOutput=true]$MANIFEST_DATA"
18+
echo "$MANIFEST_DATA" | jq -r .
19+
displayName: "Populate Image Manifest Data"
20+
name: data
21+
env:
22+
IMAGE_PLATFORM_DATA: '${{ convertToJson(parameters.platforms) }}'
23+

‎.pipelines/build/image.steps.yaml‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
parameters:
2+
- name: arch
3+
type: string
4+
default: ""
5+
6+
- name: name
7+
type: string
8+
default: ""
9+
10+
- name: os
11+
type: string
12+
default: ""
13+
14+
- name: build_tag
15+
type: string
16+
default: ""
17+
18+
- name: dockerfile_path
19+
type: string
20+
default: ""
21+
22+
- name: archive_file
23+
type: string
24+
default: '$(name)-$(os)-$(platform)-$(Tag)'
25+
26+
- name: source
27+
type: string
28+
default: drop_setup_env_source
29+
30+
- name: extra_args
31+
type: string
32+
default: ''
33+
34+
35+
- name: default_args
36+
type: object
37+
default:
38+
- "--target $(os) "
39+
- "--platform $(os)/$(arch) "
40+
41+
- name: common_build_args
42+
type: object
43+
default:
44+
- "PLATFORM=$(os)/$(arch) "
45+
- "ARCH=$(arch) "
46+
- "OS=$(os) "
47+
- "VERSION=$(Tag) "
48+
49+
50+
steps:
51+
- task: DownloadPipelineArtifact@2
52+
inputs:
53+
targetPath: $(Build.SourcesDirectory)/dst/${{ parameters.source }}
54+
artifact: '${{ parameters.source }}'
55+
56+
- task: onebranch.pipeline.containercontrol@1
57+
displayName: "Login to ACR"
58+
inputs:
59+
command: login
60+
endpoint: $(ACR_ARM_SERVICE_CONNECTION)
61+
62+
# Build and push the Docker image
63+
- task: onebranch.pipeline.imagebuildinfo@1
64+
displayName: Image Build
65+
retryCountOnTaskFailure: 3
66+
timeoutInMinutes: 30
67+
inputs:
68+
endpoint: $(ACR_ARM_SERVICE_CONNECTION)
69+
registry: $(ACR).azurecr.io
70+
repositoryName: $(os)-$(arch)/${{ parameters.name }}
71+
os: '${{ parameters.os }}'
72+
buildkit: 1
73+
dockerFileRelPath: ${{ parameters.dockerfile_path }}/Dockerfile
74+
dockerFileContextPath: ${{ parameters.source }}
75+
enable_network: true
76+
enable_pull: true
77+
build_tag: ${{ parameters.build_tag }}
78+
enable_acr_push: true
79+
80+
saveImageToPath: images/$(os)-$(arch)/${{ parameters.archive_file }}.tar.gz
81+
#compress: true
82+
#saveMetadataToPath: images/$(os)-$(arch)/metadata/${{ parameters.archive_file }}-metadata.json
83+
#enable_isolated_acr_push: true
84+
85+
# Docker Build Arguments
86+
${{ if parameters.common_build_args }}:
87+
arguments: --build-arg ${{ join('--build-arg ', parameters.common_build_args) }} ${{ parameters.extra_args }} ${{ join(' ', parameters.default_args) }}
88+
${{ else }}:
89+
arguments: ${{ parameters.extra_args }} ${{ join(' ', parameters.default_args) }}

‎.pipelines/build/manifests.jobs.yaml‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
parameters:
2+
- name: generate
3+
type: jobList
4+
5+
6+
jobs:
7+
- ${{ each job_data in parameters.generate }}:
8+
- job: ${{ job_data.job }}_generate_manifest
9+
displayName: "Generate Image Manifest - ${{ job_data.job }}"
10+
pool:
11+
type: linux
12+
variables:
13+
ob_outputDirectory: $(Build.SourcesDirectory)/out
14+
ob_git_checkout: false
15+
steps:
16+
- template: /.pipelines/build/generate-manifest.steps.yaml
17+
parameters:
18+
platforms: ${{ job_data.templateContext.platforms }}
19+
20+
- job: ${{ job_data.job }}_publish_manifest
21+
displayName: "Publish Image Manifest - ${{ job_data.job }}"
22+
dependsOn:
23+
- ${{ job_data.job }}_generate_manifest
24+
pool:
25+
type: docker
26+
os: linux
27+
variables:
28+
LinuxContainerImage: 'mcr.microsoft.com/onebranch/azurelinux/build:3.0'
29+
ob_outputDirectory: $(Build.SourcesDirectory)/out
30+
ob_git_checkout: false
31+
32+
MANIFEST_JSON: $[ dependencies.${{ job_data.job }}_generate_manifest.outputs['data.MANIFEST_JSON'] ]
33+
steps:
34+
- template: /.pipelines/build/publish-manifest.steps.yaml
35+
parameters:
36+
image_repository: ${{ job_data.templateContext.name }}
37+
image_tag: ${{ job_data.templateContext.image_tag }}
38+
manifest_data: $(MANIFEST_JSON)

0 commit comments

Comments
 (0)