验证FluxCD在host cluster控制remote cluster

[chengleqi]

1. host cluster获得remote cluster的kubeconfig

2. 创建访问remote cluster用到的secret

kubectl -n devops-project create secret generic remote-cluster \
    --from-file=value.yaml=/root/.kube/remote-kubeconfig

如果HelmRelease或者Kustomization中的spec.kubeconfig字段被指定了，helm-controller和kustomize-controller将在host cluster对remote cluster中的helm或者kustomization应用进行调谐。

helm-controller使用secret对remote cluster进行访问，该secret和HelmRelease需要在同一个namespace下，spec.kubeConfig.secretRef默认的key是value和value.yaml

ref: https://fluxcd.io/docs/components/helm/helmreleases/#remote-clusters--cluster-api

3. 创建helm应用

• pod-info-helmrepo.yaml

---
apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
metadata:
  name: podinfo
  namespace: devops-project
spec:
  interval: 1m0s
  url: https://stefanprodan.github.io/podinfo

• pod-info-helmrelease.yaml

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: backend
  namespace: devops-project
spec:
  storageNamespace: remote-devops-project
  targetNamespace: app
  install: 
    createNamespace: true
  kubeConfig:
    secretRef:
      name: remote-cluster
  interval: 1m
  chart:
    spec:
      chart: podinfo
      version: ">=4.0.0 <5.0.0"
      sourceRef:
        kind: HelmRepository
        name: podinfo
        namespace: devops-project
      interval: 1m
  upgrade:
    remediation:
      remediateLastFailure: true
  values:
    service:
      grpcService: backend
    resources:
      requests:
        cpu: 100m
        memory: 64Mi
---
apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: frontend
  namespace: devops-project
spec:
  storageNamespace: remote-devops-project
  targetNamespace: app
  install: 
    createNamespace: true
  kubeConfig:
    secretRef:
      name: remote-cluster
  interval: 1m
  chart:
    spec:
      chart: podinfo
      version: ">=4.0.0 <5.0.0"
      sourceRef:
        kind: HelmRepository
        name: podinfo
        namespace: devops-project
      interval: 1m
  dependsOn:
    - name: backend
  upgrade:
    remediation:
      remediateLastFailure: true
  values:
    backend: http://backend-podinfo:9898/echo
    resources:
      requests:
        cpu: 100m
        memory: 64Mi

manifest ref: https://fluxcd.io/docs/components/helm/helmreleases/#helmrelease-dependencies

kubectl apply -f pod-info-helmrepo.yaml,kubectl apply -f pod-info-helmrelease.yaml

在host cluster中可以看到helmrepository和helmrelease

[root@VM-16-8-centos ~]# kubectl get helmrepository -n devops-project
NAME      URL                                      AGE   READY   STATUS
podinfo   https://stefanprodan.github.io/podinfo   64m   True    stored artifact for revision 'c49e10861dd9c1c4682b0e50dfddddb092d840718ed3afbf3bafda65f467cfa7'
[root@VM-16-8-centos ~]# kubectl get helmrelease -n devops-project
NAME       AGE   READY   STATUS
backend    47m   True    Release reconciliation succeeded
frontend   47m   True    Release reconciliation succeeded

因为ArgoCD中的argoproj.io/Application只能被安装在argocd namespace中，所以kubesphere中有可能多个devops项目用了同一个argocd app的名称而导致的命名冲突问题。而flux中的helmrepository和helmrelease可以被创建在任意namespace内，所以可以将helmrelease安装在对应的kubesphere devops项目的名称空间下，就可以避免命名冲突问题。

在remote cluster中可以看到

[root@VM-16-14-centos ~]# helm ls -n remote-devops-project
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION
app-backend     app             1               2022-06-10 02:16:43.168724103 +0000 UTC deployed        podinfo-4.0.6   4.0.6
app-frontend    app             1               2022-06-10 02:17:13.198405467 +0000 UTC deployed        podinfo-4.0.6   4.0.6
[root@VM-16-14-centos ~]# kubectl get pods -n app
NAME                                   READY   STATUS    RESTARTS   AGE
app-backend-podinfo-594dbdb499-jn92z   1/1     Running   0          48m
app-frontend-podinfo-6cd78fb6c-krllh   1/1     Running   0          48m
[root@VM-16-14-centos ~]# kubectl get svc -n app
NAME                   TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                         AGE
app-backend-podinfo    ClusterIP   10.43.183.94    <none>        9898/TCP,9999/TCP               48m
app-frontend-podinfo   NodePort    10.43.197.179   <none>        9898:31336/TCP,9999:31205/TCP   48ms

HelmRelease如果设置了spec.storageNamespace（这里是remote-devops-project）字段，helm ls命令会在这个被指定的namespace中看见该helm storage，spec.targetNamespace（这里是app）可以指定helm应用安装的namespace。

将该svc改为NodePort，访问成功。