V2.0 使用argo workflow替换Jenkins 的Proposal

[colynn]

您有什么需求，是否与某个功能或问题相关? 请描述

因为atomci 1.0版本的pipeline对于jenkins的依赖过多，且灵活性不够，并且Jenkins与代码主程序是是割裂，也经常性出现安全漏洞；

另外一个好的cicd平台肯定是需要大家一起共建才能完成，故v2.0的版本也会引入插件化的理念，让我们每个人均可以定义/贡献自己的流程，期望argo workflow的替换可以成功，之后主要的进程会更新在这个issue内。

你想要的解决方案是什么

如题

• 请你通过留下👍 表情为此issue投票，帮助社区和维护者优先考虑这个请求；

• 请不要留下 "+1 "或 "me too"的评论，它们会给问题订阅者带来额外的噪音，并且无助于优先处理请求。

• 如果你有兴趣在这个问题上工作，或者已经提交了pull request，请留下评论.

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request

• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request

• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

[colynn]

• argo-workflow dockerfile结构
• try to use argo-workflow's openAPI and the core concepts
• the mapping of atomci core concept and argo-workflow
• how about argo-workflow's artifacts
• argo-workflow UI auth

• argo-workflow Field Reference

[fanhousanbu]

argo-workflow是计划怎么部署？由atomci在引入构建机的时候自动部署吗？

[colynn]

argo-workflow是计划怎么部署？由atomci在引入构建机的时候自动部署吗？

计划是先走openAPI的方式先将argo-workflow 引入进来，通过多容器的方式来运行；

[colynn]

Try to use argo-workflow open API

Generate Token

# create role
kubectl create role atomci -n argo  --verb=get,list,update,create,delete --resource=workflows.argoproj.io

# create serviceaccount
kubectl create sa atomci -n argo

# create rolebinding
kubectl create rolebinding atomci-binding -n argo --role=atomci --serviceaccount=argo:atomci

### get token
kubectl -n argo describe sa atomci |grep secrets

# atomci-token-d4zgj get from the above's command
kubectl -n argo get secrets atomci-token-d4zgj -o=jsonpath='{.data.token}' | base64 --decode

Token Usage&Test

ARGO_TOKEN="Bearer $(kubectl get secret jenkins.service-account-token -o=jsonpath='{.data.token}' | base64 --decode)"
echo $ARGO_TOKEN 

curl https://localhost:2746/api/v1/workflows/argo -H "Authorization: $ARGO_TOKEN"
# 200 OK

[colynn]

the mapping of atomci core concept and argo-workflow

AtomCI	Argo-workflow	Notes
任务模板	Container 类型的template
项目流程	Workflow template
构建部署-创建流水线	submit --from workflowtemplate/ with parameters
流水线	workflow instance

Container-type's template sample

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: compile-go-project
spec:
  entrypoint: compile-go
  templates:
    - name: compile-go
      container:
        image: golang:latest
        command: ["/bin/bash", "-c"]
        args:
          - |
            set -e
            go mod download
            go build -o my-app
        volumeMounts:
          - name: source
            mountPath: /go/src/my-project
          - name: output
            mountPath: /artifacts
      outputs:
        artifacts:
          - name: my-app
            path: /artifacts/my-app
      volumes:
        - name: source
          configMap:
            name: my-project-source
        - name: output
          emptyDir: {}

WorkflowTemplate sample

# filename: ci-template-sample.yaml
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: ci-sample
  annotations:
    workflows.argoproj.io/description: |
      This workflows builds and tests Argo Workflows.
      
      It demonstrates:
      * Clone/ Build
spec:
  arguments:
    parameters:
      - name: branch
        value: master
  entrypoint: main

  volumes:
    - hostPath:
        path: /tmp/golang
        type: Directory
      name: work

  templates:
    - name: main
      steps:
      - - name: clone
          template: clone
      - - name: build
          template: build
    - name: clone
      container:
        image: golang:1.18
        workingDir: /go/src/github.com/golang/example
        command: [ sh, -euxc ]
        args:
          - |
            git clone -v -b "{{workflow.parameters.branch}}"  --single-branch --depth 1 https://github.com/golang/example.git .

        volumeMounts:
          - mountPath: /go/src/github.com/golang/example
            name: work
            subPath: src
    - name: build
      container:
        image: golang:1.18
        workingDir: /go/src/github.com/golang/example
        command: [ sh, -euxc ] 
        args:
          - |
            cd hello ; go build 
        volumeMounts:
          - mountPath: /go/src/github.com/golang/example
            name: work
            subPath: src 

# create workflowtemplate
argo template create [-n argo]  ci-template-sample.yaml

#  then submit a workflow using this template:
argo submit --from workflowtemplate/ci-sample -p branch="master"

[colynn]

artifacts的使用注意

如果我们想通过artifacts的属性来传递构建物，那么首先需要正确配置Artifact Repository Ref.

You can reduce duplication in your templates by configuring repositories that can be accessed by any workflow. This can also remove sensitive information from your templates.

When you want to use any keyword(eg artifact) , you should remember this link will give you a lot of help.

[colynn]

argo-workflowTemplate in real world

• clone
• build & save binary artifact
• docker-image & publish

requirements

# create docker-config secret
kubectl create secret generic docker-config -n argo --from-file=/root/.docker/config.json

# create  Minio secret
# TODO: accesskey/secretkey you need change to real.
apiVersion: v1
data:
  accesskey: x
  secretkey: x
kind: Secret
metadata:
  name: s3-credentials
  namespace: argo
type: Opaque

workflow template

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: ci-sample
  annotations:
    workflows.argoproj.io/description: |
      This workflows builds and tests Argo Workflows.
      
      It demonstrates:
      * Clone/ Build
spec:
  serviceAccountName: argo
  arguments:
    parameters:
      - name: branch
        value: master
  entrypoint: main

  volumes:
    - hostPath:
        path: /tmp/codespace/
        type: Directory
      name: work
    - hostPath:
        path: /var/run/docker.sock
        type: Socket
      name: dockersock
    # Mount the configuration so we can push the image.
    # This should create the /.docker/config.json file.
    - name: docker-config
      secret:
        secretName: docker-config

  templates:
    - name: main
      steps:
      - - name: clone
          template: clone
      - - name: build
          template: build
      - - name: docker-image
          template: image

    - name: clone
      container:
        image: golang:1.18
        workingDir: /go/codespace/src
        command: [ sh, -euxc ]
        args:
          - |
            rm -rf golang-app-demo;  git clone -v -b "{{workflow.parameters.branch}}"  --single-branch https://github.com/go-atomci/golang-app-demo.git 
        volumeMounts:
          - mountPath: /go/codespace/src
            name: work
            subPath: src
    - name: build
      container:
        image: golang:1.18
        workingDir: /go/codespace/src
        command: [ sh, -euxc ] 
        args:
          - |
            cd golang-app-demo; go build -o bin/sample  cmd/sample/main.go 
        volumeMounts:
          - mountPath: /go/codespace/src
            name: work
            subPath: src 
      outputs:
        artifacts:
          - name: sample-binary
            path: golang-app-demo/bin/sample
            s3:
              endpoint: minio-default.component:9000
              bucket: argo-artifacts
              insecure: true
              key: sample.tgz
              accessKeySecret:
                name: s3-credentials
                key: accesskey
              secretKeySecret:
                name: s3-credentials
                key: secretkey

    - name: image
      container:
        image: alpine:3.13
        workingDir: /go/codespace/src
        command: [ sh, -euxc ]
        args:
          - |
            [ -f docker-19.03.15.tgz ] ||  wget http://pkg.infra.sensetime.com/artifactory/depend/gitlab/docker-19.03.15.tgz ;
            tar --extract --file docker-19.03.15.tgz --strip-components 1 --directory /usr/local/bin/ ;
            docker version ;
            cd golang-app-demo ; 
            docker build -t 10.151.3.75/library/golang-app-demo:latest  -f Dockerfile . ;
            docker push 10.151.3.75/library/golang-app-demo:latest
        volumeMounts:
          - mountPath: /go/codespace/src
            name: work
            subPath: src
          - mountPath: /var/run/docker.sock
            name: dockersock
          - name: docker-config
            mountPath: /.docker
        env:
          - name: DOCKER_CONFIG
            value: /.docker

then run it

# create workflowtemplate
argo template create [-n argo]  ci-template-sample.yaml

#  then submit a workflow using this template:
argo submit --from workflowtemplate/ci-sample -p branch="main"

[colynn]

• Argo-workflow use API to get the workflow step's detail log

Noice

• ci-sample-s7c8z was the name of workflow
• argo was the name of namespace.

# List workflows
GET /api/v1/workflows/argo?listOptions.limit=50

# list workflow-template
GET /api/v1/workflow-templates/argo?listOptions.limit=50

# List cron-workflows
GET /api/v1/cron-workflows/argo


### workflow action
# Resubmit workflow
PUT /api/v1/workflows/argo/ci-sample-s7c8z/resubmit

# suspend workflow
PUT /api/v1/workflows/argo/ci-sample-s7c8z/suspend

# resume workflow
PUT /api/v1/workflows/argo/ci-sample-s7c8z/resume

# delete workflow
DELETE /api/v1/workflows/argo/ci-sample-s7c8z

# get workflow
GET /api/v1/workflows/argo/ci-sample-s7c8z

approve

  - name: approve
    suspend: {}

Reference to: https://github.com/argoproj/argo-workflows/blob/master/examples/suspend-template.yaml

[colynn]

We know that if we delete the workflow's pod, then you will not see the log of this workflow, So we need to archive the workflow's pod logs.

Argo-workflow Configuring Archive Logs

Archive logs follow priorities:
workflow-controller config (on) > workflow spec (on/off) > template (on/off)

We do not recommend you rely on Argo Workflows to archive logs. Instead, use a conventional Kubernetes logging facility.

https://argoproj.github.io/argo-workflows/configure-archive-logs/

enable archive logs

1. You need config the Artifact Repository first.

The actual repository used by a workflow is chosen by the following rules:

• Anything explicitly configured using Artifact Repository Ref. This is the most flexible, safe, and secure option.
• From a config map named artifact-repositories if it has the workflows.argoproj.io/default-artifact-repository annotation in the workflow's namespace.
• From a workflow controller config-map.

https://argoproj.github.io/argo-workflows/workflow-controller-configmap/

For AtomCI 's TODO

• 提供artifact repository 的配置，在『服务集成』模块新增类型， 但是用户的入参应该为具体的 s3配置信息；
• 将『服务集成』模块内 artifact repository 类型的配置信息， 通过 Atomci -controller 服务转化为 artifact-repositories的configMap， 并将相应的验证信息转存为secrets.

疑问

• 是否可以配置多个artifact repository, 如果可以那会是哪个生效？ Reference To

%Reply: 当然可以配置多个artifact repository, 如果 workflowTemplate/ workflow/ cluster workflowTemplate 定义了artifact repository 通过声明artifactRepositoryRef则使用其定义，

若均没有定义则使用默认的 artifact repository定义， 若没有默认定义将会失败并返回错误。

Argo key-only artifact

When these are omitted, the bucket/secrets from the configured artifact repository is used. 仅仅定义一个key及path, 其他的信息从artifact repository ref定义中获取。
This should probably be your default if you're using argo-workflow v3.0

[colynn]

服务结构

• atomci - frontend (vue前端)
• atomci - backend (原后端及应用业务逻辑)
• atomci - controller (以atomci-backend为基础数据，创建argo-workflow依赖的配置， 比如artifact repository的配置转为 configMap)
• argo-server ( origin v3.4.5)
• workflow-controller ( origin v3.4.5)