Skip to content

Commit 35cb40a

Browse files
tlinhartcstyan
tlinhart
and
cstyan
authored
Add support for ARM64 to lambda-promtail drone build job (grafana#5354)
* Add support for ARM64 to lambda-promtail drone build job * update drone.yml based on tomas' changes Signed-off-by: Callum Styan <[email protected]> * Add entry to CHANGELOG.md * Change Docker image for building Promtail Lambda * reverse previous commit Signed-off-by: Callum Styan <[email protected]> * Add manifest for ECR Public images * Minor change to drone.yml for volume mounts + generate drone.jsonnet again. Signed-off-by: Callum Styan <[email protected]> * temp, test drone changes without having to push to main branch Signed-off-by: Callum Styan <[email protected]> * Fix linting issues Signed-off-by: Callum Styan <[email protected]> * temp, push the lambda-promtail images always so we can test the manifest task Signed-off-by: Callum Styan <[email protected]> * We don't build a base arm image, only arm64 Signed-off-by: Callum Styan <[email protected]> * Add volumes definition to ECR manifest pipeline * Regenerate drone.yml with Tomas' most recent commit Signed-off-by: Callum Styan <[email protected]> * Test some some drone changes. Signed-off-by: Callum Styan <[email protected]> * fix lint error in drone.jsonnet Signed-off-by: Callum Styan <[email protected]> * Drop out one of my more recent test changes. Signed-off-by: Callum Styan <[email protected]> * Regenerate drone config. Signed-off-by: Callum Styan <[email protected]> Co-authored-by: Callum Styan <[email protected]>
1 parent d6f50ca commit 35cb40a

File tree

4 files changed

+209
-15
lines changed

4 files changed

+209
-15
lines changed

‎.drone/docker-manifest-ecr.tmpl

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
image: public.ecr.aws/grafana/{{config.target}}:{{#if build.tag}}{{trimPrefix "v" build.tag}}{{else}}{{build.branch}}-{{substr 0 7 build.commit}}{{/if}}
2+
tags:
3+
- main
4+
{{#if build.tag}}
5+
- latest
6+
{{/if}}
7+
{{#if build.tags}}
8+
{{#each build.tags}}
9+
- {{this}}
10+
{{/each}}
11+
{{/if}}
12+
manifests:
13+
- image: public.ecr.aws/grafana/{{config.target}}:{{#if build.tag}}{{trimPrefix "v" build.tag}}{{else}}{{build.branch}}-{{substr 0 7 build.commit}}{{/if}}-amd64
14+
platform:
15+
architecture: amd64
16+
os: linux
17+
- image: public.ecr.aws/grafana/{{config.target}}:{{#if build.tag}}{{trimPrefix "v" build.tag}}{{else}}{{build.branch}}-{{substr 0 7 build.commit}}{{/if}}-arm64
18+
platform:
19+
architecture: arm64
20+
os: linux
21+
variant: v8

‎.drone/drone.jsonnet

Lines changed: 73 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -257,17 +257,9 @@ local promtail(arch) = pipeline('promtail-' + arch) + arch_image(arch) {
257257
depends_on: ['check'],
258258
};
259259

260-
local lambda_promtail(tags='') = pipeline('lambda-promtail') {
260+
local lambda_promtail(arch) = pipeline('lambda-promtail-' + arch) + arch_image(arch) {
261261
steps+: [
262-
{
263-
name: 'image-tag',
264-
image: 'alpine',
265-
commands: [
266-
'apk add --no-cache bash git',
267-
'git fetch origin --tags',
268-
'echo $(./tools/image-tag)-amd64 > .tags',
269-
] + if tags != '' then ['echo ",%s" >> .tags' % tags] else [],
270-
},
262+
// dry run for everything that is not tag or main
271263
lambda_promtail_ecr('lambda-promtail') {
272264
depends_on: ['image-tag'],
273265
when: condition('exclude').tagMain,
@@ -342,6 +334,58 @@ local manifest(apps) = pipeline('manifest') {
342334
],
343335
};
344336

337+
local manifest_ecr(apps, archs) = pipeline('manifest-ecr') {
338+
steps: std.foldl(
339+
function(acc, app) acc + [{
340+
name: 'manifest-' + app,
341+
image: 'plugins/manifest',
342+
volumes: [{
343+
name: 'dockerconf',
344+
path: '/.docker',
345+
}],
346+
settings: {
347+
// the target parameter is abused for the app's name,
348+
// as it is unused in spec mode. See docker-manifest-ecr.tmpl
349+
target: app,
350+
spec: '.drone/docker-manifest-ecr.tmpl',
351+
ignore_missing: true,
352+
},
353+
depends_on: ['clone'] + (
354+
// Depend on the previous app, if any.
355+
if std.length(acc) > 0
356+
then [acc[std.length(acc) - 1].name]
357+
else []
358+
),
359+
}],
360+
apps,
361+
[{
362+
name: 'ecr-login',
363+
image: 'docker:dind',
364+
volumes: [{
365+
name: 'dockerconf',
366+
path: '/root/.docker',
367+
}],
368+
environment: {
369+
AWS_ACCESS_KEY_ID: { from_secret: ecr_key.name },
370+
AWS_SECRET_ACCESS_KEY: { from_secret: ecr_secret_key.name },
371+
},
372+
commands: [
373+
'apk add --no-cache aws-cli',
374+
'docker login --username AWS --password $(aws ecr-public get-login-password --region us-east-1) public.ecr.aws',
375+
],
376+
depends_on: ['clone'],
377+
}],
378+
),
379+
volumes: [{
380+
name: 'dockerconf',
381+
temp: {},
382+
}],
383+
depends_on: [
384+
'lambda-promtail-%s' % arch
385+
for arch in archs
386+
],
387+
};
388+
345389
[
346390
pipeline('loki-build-image') {
347391
workspace: {
@@ -472,6 +516,7 @@ local manifest(apps) = pipeline('manifest') {
472516
commands: [
473517
'apk add --no-cache bash git',
474518
'git fetch origin --tags',
519+
'echo $(./tools/image-tag)',
475520
'echo $(./tools/image-tag) > .tag',
476521
],
477522
depends_on: ['clone'],
@@ -489,5 +534,21 @@ local manifest(apps) = pipeline('manifest') {
489534
],
490535
},
491536
] + [promtail_win()]
492-
+ [lambda_promtail('main')]
493-
+ [github_secret, pull_secret, docker_username_secret, docker_password_secret, ecr_key, ecr_secret_key, deploy_configuration]
537+
+ [
538+
lambda_promtail(arch)
539+
for arch in ['amd64', 'arm64']
540+
] + [
541+
manifest_ecr(['lambda-promtail'], ['amd64', 'arm64']) {
542+
trigger: condition('include').tagMain {
543+
event: ['push'],
544+
},
545+
},
546+
] + [
547+
github_secret,
548+
pull_secret,
549+
docker_username_secret,
550+
docker_password_secret,
551+
ecr_key,
552+
ecr_secret_key,
553+
deploy_configuration,
554+
]

‎.drone/drone.yml

Lines changed: 114 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1024,6 +1024,7 @@ steps:
10241024
- commands:
10251025
- apk add --no-cache bash git
10261026
- git fetch origin --tags
1027+
- echo $(./tools/image-tag)
10271028
- echo $(./tools/image-tag) > .tag
10281029
depends_on:
10291030
- clone
@@ -1074,13 +1075,15 @@ trigger:
10741075
depends_on:
10751076
- check
10761077
kind: pipeline
1077-
name: lambda-promtail
1078+
name: lambda-promtail-amd64
1079+
platform:
1080+
arch: amd64
1081+
os: linux
10781082
steps:
10791083
- commands:
10801084
- apk add --no-cache bash git
10811085
- git fetch origin --tags
10821086
- echo $(./tools/image-tag)-amd64 > .tags
1083-
- echo ",main" >> .tags
10841087
image: alpine
10851088
name: image-tag
10861089
- depends_on:
@@ -1131,6 +1134,114 @@ trigger:
11311134
- pull_request
11321135
- tag
11331136
---
1137+
depends_on:
1138+
- check
1139+
kind: pipeline
1140+
name: lambda-promtail-arm64
1141+
platform:
1142+
arch: arm64
1143+
os: linux
1144+
steps:
1145+
- commands:
1146+
- apk add --no-cache bash git
1147+
- git fetch origin --tags
1148+
- echo $(./tools/image-tag)-arm64 > .tags
1149+
image: alpine
1150+
name: image-tag
1151+
- depends_on:
1152+
- image-tag
1153+
image: cstyan/ecr
1154+
name: build-lambda-promtail-image
1155+
privileged: true
1156+
settings:
1157+
access_key:
1158+
from_secret: ecr_key
1159+
dockerfile: tools/lambda-promtail/Dockerfile
1160+
dry_run: true
1161+
region: us-east-1
1162+
registry: public.ecr.aws/grafana
1163+
repo: public.ecr.aws/grafana/lambda-promtail
1164+
secret_key:
1165+
from_secret: ecr_secret_key
1166+
when:
1167+
ref:
1168+
exclude:
1169+
- refs/heads/main
1170+
- refs/heads/k???
1171+
- refs/tags/v*
1172+
- depends_on:
1173+
- image-tag
1174+
image: cstyan/ecr
1175+
name: publish-lambda-promtail-image
1176+
privileged: true
1177+
settings:
1178+
access_key:
1179+
from_secret: ecr_key
1180+
dockerfile: tools/lambda-promtail/Dockerfile
1181+
dry_run: false
1182+
region: us-east-1
1183+
registry: public.ecr.aws/grafana
1184+
repo: public.ecr.aws/grafana/lambda-promtail
1185+
secret_key:
1186+
from_secret: ecr_secret_key
1187+
when:
1188+
ref:
1189+
include:
1190+
- refs/heads/main
1191+
- refs/heads/k???
1192+
- refs/tags/v*
1193+
trigger:
1194+
event:
1195+
- push
1196+
- pull_request
1197+
- tag
1198+
---
1199+
depends_on:
1200+
- lambda-promtail-amd64
1201+
- lambda-promtail-arm64
1202+
kind: pipeline
1203+
name: manifest-ecr
1204+
steps:
1205+
- commands:
1206+
- apk add --no-cache aws-cli
1207+
- docker login --username AWS --password $(aws ecr-public get-login-password --region
1208+
us-east-1) public.ecr.aws
1209+
depends_on:
1210+
- clone
1211+
environment:
1212+
AWS_ACCESS_KEY_ID:
1213+
from_secret: ecr_key
1214+
AWS_SECRET_ACCESS_KEY:
1215+
from_secret: ecr_secret_key
1216+
image: docker:dind
1217+
name: ecr-login
1218+
volumes:
1219+
- name: dockerconf
1220+
path: /root/.docker
1221+
- depends_on:
1222+
- clone
1223+
- ecr-login
1224+
image: plugins/manifest
1225+
name: manifest-lambda-promtail
1226+
settings:
1227+
ignore_missing: true
1228+
spec: .drone/docker-manifest-ecr.tmpl
1229+
target: lambda-promtail
1230+
volumes:
1231+
- name: dockerconf
1232+
path: /.docker
1233+
trigger:
1234+
event:
1235+
- push
1236+
ref:
1237+
include:
1238+
- refs/heads/main
1239+
- refs/heads/k???
1240+
- refs/tags/v*
1241+
volumes:
1242+
- name: dockerconf
1243+
temp: {}
1244+
---
11341245
get:
11351246
name: pat
11361247
path: infra/data/ci/github/grafanabot
@@ -1174,6 +1285,6 @@ kind: secret
11741285
name: deploy_config
11751286
---
11761287
kind: signature
1177-
hmac: 81be9fce60976703815abf1fd2f1c9255a142d615b90d4b5278ff4a64fa25e92
1288+
hmac: 8335495d285498f513bb265cf90079175c25f5b6c5717deab7abfae65341726a
11781289

11791290
...

‎CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -179,6 +179,7 @@ to include only the most relevant.
179179
* [5409](https://github.com/grafana/loki/pull/5409) **ldb**: Enable best effort parsing for Syslog messages
180180
* [5392](https://github.com/grafana/loki/pull/5392) **MichelHollands**: Etcd credentials are parsed as secrets instead of plain text now.
181181
* [5361](https://github.com/grafana/loki/pull/5361) **ctovena**: Add usage report to grafana.com.
182+
* [5354](https://github.com/grafana/loki/pull/5354) **tlinhart**: Add support for ARM64 to lambda-promtail drone build job.
182183
* [5289](https://github.com/grafana/loki/pull/5289) **ctovena**: Fix deduplication bug in queries when mutating labels.
183184
* [5302](https://github.com/grafana/loki/pull/5302) **MasslessParticle** Update azure blobstore client to use new sdk.
184185
* [5243](https://github.com/grafana/loki/pull/5290) **ssncferreira**: Update Promtail to support duration string formats.

0 commit comments

Comments
 (0)