Skip to content

Latest commit

ย 

History

History
589 lines (517 loc) ยท 24.2 KB

File metadata and controls

589 lines (517 loc) ยท 24.2 KB

1. ๋ฉ€ํ‹ฐ ํด๋Ÿฌ์Šคํ„ฐ ๋ฐ๋ชจ๋ฅผ ์œ„ํ•œ ํ™˜๊ฒฝ ๋ณ€์ˆ˜ ์„ค์ •

  • ๋‹จ์ผ ํ”„๋กœ์ ํŠธ ํ™˜๊ฒฝ ์—์„œ ์„œ๋กœ ๋‹ค๋ฅธ Region ์— ์žˆ๋Š” ๋‘ GKE ํด๋Ÿฌ์Šคํ„ฐ

    # CLUSTER-1
    export PROJECT_1=kiwonlee-appmod-sandbox
    export CLUSTER_1=multi-cluster-demo-us
    export LOCATION_1=us-central1-c
    
    # CLUSTER-2
    export PROJECT_2=kiwonlee-appmod-sandbox
    export CLUSTER_2=multi-cluster-demo-asia
    export LOCATION_2=asia-northeast1-c
    
    # namespace for application deploy
    export NAMESPACE=whereami
    

2. Cluster_1 ์ƒ์„ฑ๊ณผ ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ๋ฐฐํฌ(whereami)

  • GKE ํด๋Ÿฌ์Šคํ„ฐ ์ƒ์„ฑ ${CLUSTER_1}

    gcloud container clusters create ${CLUSTER_1} \
     --project=${PROJECT_1} \
     --zone=${LOCATION_1} \
     --machine-type=e2-standard-4 \
     --num-nodes=3 \
     --workload-pool=${PROJECT_1}.svc.id.goog
    
  • ์ƒ์„ฑํ•œ ํด๋Ÿฌ์Šคํ„ฐ์˜ ์ธ์ฆ์ •๋ณด์™€ ์—”๋“œํฌ์ธํŠธ ์ •๋ณด๋ฅผ kubeconfig์— ์—…๋ฐ์ดํŠธ

    gcloud container clusters get-credentials ${CLUSTER_1} \
     --project=${PROJECT_1} \
     --zone=${LOCATION_1}
     
    export CTX_1="gke_${PROJECT_1}_${LOCATION_1}_${CLUSTER_1}"
    
  • namespace ์ƒ์„ฑ, ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ (whereami) ๋ฐฐํฌ

    • ๋‘ ํด๋Ÿฌ์Šคํ„ฐ์— ๋™์ผํ•œ namespace๋ฅผ ์ƒ์„ฑํ•˜๊ณ , ๋™์ผ namespace ์— ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜์„ ๋ฐฐํฌํ•จ. Namespace sameness
    kubectl create --context=${CTX_1} namespace ${NAMESPACE}
    kubectl --context=${CTX_1} apply -f ./kube/whereami.yaml --namespace ${NAMESPACE}
    
  • ๋ฐฐํฌํ•œ ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ๋™์ž‘ ํ…Œ์ŠคํŠธ

    ๋ชจ๋“  ํŠธ๋ž˜ํ”ฝ์€ ํด๋Ÿฌ์Šคํ„ฐ ๋‚ด์˜ Pod๋กœ๋งŒ ์ „๋‹ฌ๋จ

    $ kubectl get po,svc --context=${CTX_1} --namespace ${NAMESPACE}
    NAME                                       READY   STATUS        RESTARTS   AGE
    pod/whereami-deployment-86bc7496d8-86pxc   1/1     Running   0          6m34s
    pod/whereami-deployment-86bc7496d8-9dffb   1/1     Running   0          29m
    
    NAME                       TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
    service/whereami-service   ClusterIP   10.24.8.127   <none>        80/TCP    76s
    
    $ kubectl --context=${CTX_1} --namespace ${NAMESPACE} exec pod/whereami-deployment-86bc7496d8-9dffb -it -- /bin/sh
    $ curl whereami-service.whereami.svc.cluster.local
    {
      "cluster_name": "multi-cluster-demo-us",
      "host_header": "whereami-service.whereami.svc.cluster.local",
      "pod_name": "whereami-deployment-86bc7496d8-86pxc",
      "pod_name_emoji": "๐Ÿ‡น๐Ÿ‡ด",
      "project_id": "kwlee-goog-sandbox",
      "timestamp": "2022-04-28T05:36:20",
      "zone": "us-central1-c"
    }
    $ curl whereami-service.whereami.svc.cluster.local
    {
      "cluster_name": "multi-cluster-demo-us",
      "host_header": "whereami-service.whereami.svc.cluster.local",
      "pod_name": "whereami-deployment-86bc7496d8-9dffb",
      "pod_name_emoji": "๐Ÿ‘จโš–๏ธ",
      "project_id": "kwlee-goog-sandbox",
      "timestamp": "2022-04-28T05:36:21",
      "zone": "us-central1-c"
    }
    

3. Cluster_2 ์ƒ์„ฑ๊ณผ ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ๋ฐฐํฌ(whereami)

  • GKE ํด๋Ÿฌ์Šคํ„ฐ ์ƒ์„ฑ ${CLUSTER_2}

    gcloud container clusters create ${CLUSTER_2} \
     --project=${PROJECT_2} \
     --zone=${LOCATION_2} \
     --machine-type=e2-standard-4 \
     --num-nodes=3 \
     --workload-pool=${PROJECT_2}.svc.id.goog
    
    gcloud container clusters get-credentials ${CLUSTER_2} \
     --project=${PROJECT_2} \
     --zone=${LOCATION_2}
    
    export CTX_2="gke_${PROJECT_2}_${LOCATION_2}_${CLUSTER_2}"
    
    kubectl create --context=${CTX_2} namespace ${NAMESPACE}
    kubectl --context=${CTX_2} apply -f ./kube/whereami.yaml --namespace ${NAMESPACE}
    
  • ๋ฐฐํฌํ•œ ์• ํ”Œ๋ฆฌ์ผ€์ด์…˜ ๋™์ž‘ ํ…Œ์ŠคํŠธ

    ๋ชจ๋“  ํŠธ๋ž˜ํ”ฝ์€ ํด๋Ÿฌ์Šคํ„ฐ ๋‚ด์˜ Pod๋กœ๋งŒ ์ „๋‹ฌ๋จ

    $ kubectl get po,svc --context=${CTX_2} --namespace ${NAMESPACE}
    NAME                                       READY   STATUS    RESTARTS   AGE
    pod/whereami-deployment-86bc7496d8-m2knq   1/1     Running   0          21m
    pod/whereami-deployment-86bc7496d8-xlsxh   1/1     Running   0          8m40s
    
    NAME                       TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
    service/whereami-service   ClusterIP   10.88.11.39   <none>        80/TCP    20s
       
    $ kubectl --context=${CTX_2} --namespace ${NAMESPACE} exec pod/whereami-deployment-86bc7496d8-m2knq -it -- /bin/sh
    $ curl whereami-service.whereami.svc.cluster.local
    {
      "cluster_name": "multi-cluster-demo-asia",
      "host_header": "whereami-service.whereami.svc.cluster.local",
      "pod_name": "whereami-deployment-86bc7496d8-xlsxh",
      "pod_name_emoji": "๐Ÿ’‘๐Ÿพ",
      "project_id": "kwlee-goog-sandbox",
      "timestamp": "2022-04-28T05:39:37",
      "zone": "asia-northeast1-c"
    }
    $ curl whereami-service.whereami.svc.cluster.local
    {
      "cluster_name": "multi-cluster-demo-asia",
      "host_header": "whereami-service.whereami.svc.cluster.local",
      "pod_name": "whereami-deployment-86bc7496d8-m2knq",
      "pod_name_emoji": "๐Ÿ‘จ๐Ÿพโš•๏ธ",
      "project_id": "kwlee-goog-sandbox",
      "timestamp": "2022-04-28T05:39:38",
      "zone": "asia-northeast1-c"
    }
    

4. ASM ์„ค์น˜

  • download asmcli to install ASM

    curl https://storage.googleapis.com/csm-artifacts/asm/asmcli_1.13 > asmcli
    chmod +x asmcli
    
  • install ASM to ${CLUSTER_1}. macOS isn't supported for installation ASM Also, the ingress gateway is not installed now.

    ./asmcli install \
    --project_id ${PROJECT_1} \
    --cluster_name ${CLUSTER_1} \
    --cluster_location ${LOCATION_1} \
    --output_dir ./anthos-service-mesh \
    --enable_all  \
    --ca mesh_ca
    
  • install ASM to ${CLUSTER_2}.

    ./asmcli install \
    --project_id ${PROJECT_2} \
    --cluster_name ${CLUSTER_2} \
    --cluster_location ${LOCATION_2} \
    --output_dir ./anthos-service-mesh \
    --enable_all  \
    --ca mesh_ca
    
  • Injecting sidecar proxies

    export REVISION=$(kubectl get deploy -n istio-system -l app=istiod -o jsonpath={.items[*].metadata.labels.'istio\.io\/rev'}'{"\n"}')
    ## REVISION=asm-1132-2
    
    kubectl --context=${CTX_1} label namespace ${NAMESPACE} istio-injection- istio.io/rev=${REVISION} --overwrite
    kubectl --context=${CTX_1} rollout restart deployment whereami-deployment --namespace ${NAMESPACE}
    kubectl --context=${CTX_2} label namespace ${NAMESPACE} istio-injection- istio.io/rev=${REVISION} --overwrite
    kubectl --context=${CTX_2} rollout restart deployment whereami-deployment --namespace ${NAMESPACE}
    
  • Envoy Proxy ์„ค์น˜ ํ™•์ธ

๊ฐ pod ๋งˆ๋‹ค container ๊ฐ€ 2๊ฐœ์”ฉ (main container + sidecar) ์ƒ์„ฑ๋œ ๊ฒƒ ํ™•์ธ

$ kubectl get po,svc --context=${CTX_1} --namespace ${NAMESPACE}
NAME                                       READY   STATUS    RESTARTS   AGE
pod/whereami-deployment-5755d8b68b-kx4ss   2/2     Running   0          2m9s
pod/whereami-deployment-5755d8b68b-kxzzx   2/2     Running   0          2m12s

$ kubectl get po,svc --context=${CTX_2} --namespace ${NAMESPACE}
NAME                                       READY   STATUS    RESTARTS   AGE
pod/whereami-deployment-764cbfccdb-dw8ct   2/2     Running   0          2m2s
pod/whereami-deployment-764cbfccdb-vlzfg   2/2     Running   0          2m12s   

5. istio-ingrssgateway ์„ค์น˜, ${CLUSTER-1}์—๋งŒ

  • Install an ingress gateway

    export GATEWAY_NAMESPACE=istio-ingress
    
    kubectl create namespace ${GATEWAY_NAMESPACE} --context=${CTX_1} 
    kubectl --context=${CTX_1} label namespace ${GATEWAY_NAMESPACE} istio-injection- istio.io/rev=${REVISION} --overwrite
    
    kubectl apply --context=${CTX_1} -n ${GATEWAY_NAMESPACE} -f ./anthos-service-mesh/samples/gateways/istio-ingressgateway
    

    Output

    $ kubectl --context=${CTX_1} -n ${GATEWAY_NAMESPACE} get po,svc
    NAME                                        READY   STATUS    RESTARTS   AGE
    pod/istio-ingressgateway-66d9b945dc-46852   1/1     Running   0          31s
    pod/istio-ingressgateway-66d9b945dc-ftn8z   1/1     Running   0          31s
    pod/istio-ingressgateway-66d9b945dc-kfnsv   1/1     Running   0          31s
    
    NAME                           TYPE           CLUSTER-IP    EXTERNAL-IP      PORT(S)                                      AGE
    service/istio-ingressgateway   LoadBalancer   10.24.1.196   34.132.129.229   15021:30640/TCP,80:30051/TCP,443:31968/TCP   35s
    
  • Gateway, VirtualService ์ •์˜

    $ kubectl --context=${CTX_1} apply -f ./kube/asm-nw-ingress.yaml --namespace ${NAMESPACE}
    $ kubectl --context=${CTX_1} --namespace ${NAMESPACE} get gateway,virtualservice
    NAME                                           AGE
    gateway.networking.istio.io/whereami-gateway   49s
    
    NAME                                             GATEWAYS            HOSTS   AGE
    virtualservice.networking.istio.io/whereami-vs   ["whereami-gateway"]   ["*"]   46s
    
  • istio-ingressgateway ์˜ EXTERNAL-IP(L4 LoadBalancer)๋กœ ํ˜ธ์ถœ ํ™•์ธ (34.132.129.229)

    ๋ชจ๋“  ํŠธ๋ž˜ํ”ฝ์€ ํด๋Ÿฌ์Šคํ„ฐ ๋‚ด์˜ Pod๋กœ๋งŒ ์ „๋‹ฌ๋จ,๋‹จ ์™ธ๋ถ€์—์„œ๋„ ํ˜ธ์ถœ ๊ฐ€๋Šฅ

    $ curl 34.132.129.229
    {
      "cluster_name": "multi-cluster-demo-us",
      "host_header": "34.132.129.229",
      "pod_name": "whereami-deployment-5755d8b68b-kxzzx",
      "pod_name_emoji": "โน",
      "project_id": "kwlee-goog-sandbox",
      "timestamp": "2022-04-28T06:22:41",
      "zone": "us-central1-c"
    }
    $ curl 34.132.129.229
    {
      "cluster_name": "multi-cluster-demo-us",
      "host_header": "34.132.129.229",
      "pod_name": "whereami-deployment-5755d8b68b-kx4ss",
      "pod_name_emoji": "๐Ÿ˜…",
      "project_id": "kwlee-goog-sandbox",
      "timestamp": "2022-04-28T06:22:43",
      "zone": "us-central1-c"
    }
    $ curl 34.132.129.229
    {
      "cluster_name": "multi-cluster-demo-us",
      "host_header": "34.132.129.229",
      "pod_name": "whereami-deployment-5755d8b68b-kxzzx",
      "pod_name_emoji": "โน",
      "project_id": "kwlee-goog-sandbox",
      "timestamp": "2022-04-28T06:22:47",
      "zone": "us-central1-c"
    }
    $ curl 34.132.129.229
    {
      "cluster_name": "multi-cluster-demo-us",
      "host_header": "34.132.129.229",
      "pod_name": "whereami-deployment-5755d8b68b-kx4ss",
      "pod_name_emoji": "๐Ÿ˜…",
      "project_id": "kwlee-goog-sandbox",
      "timestamp": "2022-04-28T06:22:51",
      "zone": "us-central1-c"
    }
    
    

image

6. ๋ฉ€ํ‹ฐ ํด๋Ÿฌ์Šคํ„ฐ ๋ฉ”์‹œ ์„ค์ •

  • ๋‘ ํด๋Ÿฌ์Šคํ„ฐ๋ฅผ ๋‹จ์ผ Anthos Service Mesh์— ๊ฒฐํ•ฉ ํ•˜๊ณ  ํด๋Ÿฌ์Šคํ„ฐ ๊ฐ„ ๋ถ€ํ•˜ ๋ถ„์‚ฐ์„ ์‚ฌ์šฉ ์„ค์ •

  • create firewall rule

    function join_by { local IFS="$1"; shift; echo "$*"; }
    ALL_CLUSTER_CIDRS=$(gcloud container clusters list --project $PROJECT_1 --format='value(clusterIpv4Cidr)' | sort | uniq)
    ALL_CLUSTER_CIDRS=$(join_by , $(echo "${ALL_CLUSTER_CIDRS}"))
    ALL_CLUSTER_NETTAGS=$(gcloud compute instances list --project $PROJECT_1 --format='value(tags.items.[0])' | sort | uniq)
    ALL_CLUSTER_NETTAGS=$(join_by , $(echo "${ALL_CLUSTER_NETTAGS}"))   
    
    gcloud compute firewall-rules create istio-multicluster-pods \
     --allow=tcp,udp,icmp,esp,ah,sctp \
     --direction=INGRESS \
     --priority=900 \
     --source-ranges="${ALL_CLUSTER_CIDRS}" \
     --target-tags="${ALL_CLUSTER_NETTAGS}" --quiet
    
  • ํด๋Ÿฌ์Šคํ„ฐ๊ฐ„ ์—”๋“œํฌ์ธํŠธ ๊ฒ€์ƒ‰ ๊ตฌ์„ฑ

    ./asmcli create-mesh \
     ${PROJECT_1} \
     ${PROJECT_1}/${LOCATION_1}/${CLUSTER_1} \
     ${PROJECT_2}/${LOCATION_2}/${CLUSTER_2}
    
  • ๊ตฌ์„ฑ ํ™•์ธ

    $ gcloud container hub memberships list
    NAME: multi-cluster-demo-us
    EXTERNAL_ID: 43f55ae6-fc47-4fa6-b144-dabd47452d9a
    
    NAME: multi-cluster-demo-asia
    EXTERNAL_ID: 461327dc-8ab5-474b-afa5-ea182c502e8c
    

7. ๋ฉ€ํ‹ฐ ํด๋Ÿฌ์Šคํ„ฐ ๋ฉ”์‹œ ํ…Œ์ŠคํŠธ

๋ชจ๋“  ํŠธ๋ž˜ํ”ฝ์€ ๋‹จ์ผ ๋ฉ”์‹œ๋กœ ์„ค์ •ํ•œ ๋‘ ํด๋Ÿฌ์Šคํ„ฐ ${CLUSTER_1} ๊ดด ${CLUSTER_2}์˜ Pod๋กœ ์ „๋‹ฌ๋จ

$ curl http://34.132.129.229/
{
  "cluster_name": "multi-cluster-demo-us",
  "host_header": "34.132.129.229",
  "pod_name": "whereami-deployment-5755d8b68b-kxzzx",
  "pod_name_emoji": "โน",
  "project_id": "kwlee-goog-sandbox",
  "timestamp": "2022-04-28T06:37:06",
  "zone": "us-central1-c"
}
$ curl http://34.132.129.229/
{
  "cluster_name": "multi-cluster-demo-asia",
  "host_header": "34.132.129.229",
  "pod_name": "whereami-deployment-764cbfccdb-vlzfg",
  "pod_name_emoji": "๐Ÿง‘๐Ÿฝโœˆ",
  "project_id": "kwlee-goog-sandbox",
  "timestamp": "2022-04-28T06:37:08",
  "zone": "asia-northeast1-c"
}
$ curl http://34.132.129.229/
{
  "cluster_name": "multi-cluster-demo-us",
  "host_header": "34.132.129.229",
  "pod_name": "whereami-deployment-5755d8b68b-kx4ss",
  "pod_name_emoji": "๐Ÿ˜…",
  "project_id": "kwlee-goog-sandbox",
  "timestamp": "2022-04-28T06:37:10",
  "zone": "us-central1-c"
}
$ curl http://34.132.129.229/
{
  "cluster_name": "multi-cluster-demo-asia",
  "host_header": "34.132.129.229",
  "pod_name": "whereami-deployment-764cbfccdb-vlzfg",
  "pod_name_emoji": "๐Ÿง‘๐Ÿฝโœˆ",
  "project_id": "kwlee-goog-sandbox",
  "timestamp": "2022-04-28T06:37:12",
  "zone": "asia-northeast1-c"
}

image

8. istio-ingrssgateway ์„ค์น˜, ${CLUSTER-2}์—๋งŒ

  • Install an ingress gateway

    export GATEWAY_NAMESPACE=istio-ingress
    
    kubectl create namespace ${GATEWAY_NAMESPACE} --context=${CTX_2} 
    kubectl --context=${CTX_2} label namespace ${GATEWAY_NAMESPACE} istio-injection- istio.io/rev=${REVISION} --overwrite
    
    kubectl apply --context=${CTX_2} -n ${GATEWAY_NAMESPACE} -f ./anthos-service-mesh/samples/gateways/istio-ingressgateway
    
    

    Output

    $ kubectl --context=${CTX_2} -n ${GATEWAY_NAMESPACE} get po,svc
    NAME                                        READY   STATUS    RESTARTS   AGE
    pod/istio-ingressgateway-66d9b945dc-hlw7q   1/1     Running   0          8s
    pod/istio-ingressgateway-66d9b945dc-jlmtg   1/1     Running   0          8s
    pod/istio-ingressgateway-66d9b945dc-lnjxv   1/1     Running   0          8s
    
    NAME                           TYPE           CLUSTER-IP    EXTERNAL-IP         PORT(S)                                      AGE
    service/istio-ingressgateway   LoadBalancer   10.88.5.210   35.200.122.133      15021:32628/TCP,80:32344/TCP,443:32249/TCP   8s
    
  • Gateway, VirtualService ์ •์˜

    $ kubectl --context=${CTX_2} apply -f ./kube/asm-nw-ingress.yaml --namespace ${NAMESPACE}
    $ kubectl --context=${CTX_2} --namespace ${NAMESPACE} get gateway,virtualservice
    NAME                                           AGE
    gateway.networking.istio.io/whereami-gateway   49s
    
    NAME                                             GATEWAYS            HOSTS   AGE
    virtualservice.networking.istio.io/whereami-vs   ["whereami-gateway"]   ["*"]   46s
    
  • istio-ingressgateway ์˜ EXTERNAL-IP(L4 LoadBalancer)๋กœ ํ˜ธ์ถœ ํ™•์ธ (35.200.122.133)

    $ curl 35.200.122.133
    {
      "cluster_name": "multi-cluster-demo-us",
      "host_header": "35.200.122.133",
      "pod_name": "whereami-deployment-5755d8b68b-kx4ss",
      "pod_name_emoji": "๐Ÿ˜…",
      "project_id": "kwlee-goog-sandbox",
      "timestamp": "2022-04-28T07:39:46",
      "zone": "us-central1-c"
    }
    $ curl 35.200.122.133
    {
      "cluster_name": "multi-cluster-demo-asia",
      "host_header": "35.200.122.133",
      "pod_name": "whereami-deployment-764cbfccdb-vlzfg",
      "pod_name_emoji": "๐Ÿง‘๐Ÿฝโœˆ",
      "project_id": "kwlee-goog-sandbox",
      "timestamp": "2022-04-28T07:39:48",
      "zone": "asia-northeast1-c"
    }
    $ curl 35.200.122.133
    {
      "cluster_name": "multi-cluster-demo-asia",
      "host_header": "35.200.122.133",
      "pod_name": "whereami-deployment-764cbfccdb-dw8ct",
      "pod_name_emoji": "๐Ÿคฆ๐Ÿพ",
      "project_id": "kwlee-goog-sandbox",
      "timestamp": "2022-04-28T07:39:48",
      "zone": "asia-northeast1-c"
    }
    $ curl 35.200.122.133
    {
      "cluster_name": "multi-cluster-demo-asia",
      "host_header": "35.200.122.133",
      "pod_name": "whereami-deployment-764cbfccdb-dw8ct",
      "pod_name_emoji": "๐Ÿคฆ๐Ÿพ",
      "project_id": "kwlee-goog-sandbox",
      "timestamp": "2022-04-28T07:39:49",
      "zone": "asia-northeast1-c"
    }
    

image

9. MultiClusterIngress ์„ค์ •

  • fleet ์— ์ด๋ฏธ ๋“ฑ๋กํ–ˆ๊ธฐ ๋•Œ๋ฌธ์—, fleet ์— ํด๋Ÿฌ์Šคํ„ฐ ๋“ฑ๋ก ์€ skip

    • ์ •์ƒ ๋“ฑ๋ก ์—ฌ๋ถ€๋งŒ ์•„๋ž˜์—์„œ ํ™•์ธ
    $ gcloud container hub memberships list --project=${PROJECT_1}
    NAME: multi-cluster-demo-us
    EXTERNAL_ID: 43f55ae6-fc47-4fa6-b144-dabd47452d9a
    
    NAME: multi-cluster-demo-asia
    EXTERNAL_ID: 461327dc-8ab5-474b-afa5-ea182c502e8c
    
  • ConfigCluster ์ง€์ •

    $ gcloud beta container hub ingress enable --config-membership=${CLUSTER_1}
    $ gcloud beta container hub ingress update
    
    $ gcloud beta container hub ingress describe
    createTime: '2021-08-19T07:28:13.980348616Z'
    membershipStates:
      projects/852689693404/locations/global/memberships/asm-multi-neg-1:
        state:
          code: OK
          updateTime: '2022-04-28T08:06:12.073973312Z'
      projects/852689693404/locations/global/memberships/asm-multi-neg-2:
        state:
          code: OK
          updateTime: '2022-04-28T08:06:12.073974684Z'
    name: projects/kwlee-goog-sandbox/locations/global/features/multiclusteringress
    resourceState:
      state: ACTIVE
    spec:
      multiclusteringress:
        configMembership: projects/kwlee-goog-sandbox/locations/global/memberships/asm-multi-neg-1
    state:
      state:
        code: OK
        description: Ready to use
        updateTime: '2022-04-28T08:05:01.291033008Z'
    updateTime: '2022-04-28T08:06:22.733507873Z'
    

    ๋งŒ์•ฝ AVMBR111 ์—๋Ÿฌ ๋ฐœ์ƒ ์‹œ, ํ•ด๋‹น ๋งํฌ ์ฐธ์กฐ

  • MCS ์ƒ์„ฑ

    • ConfigCluster ์— MCS ๋ฅผ ์ƒ์„ฑํ•˜๋ฉด, fleet ์— ๋“ฑ๋ก๋œ ํด๋Ÿฌ์Šคํ„ฐ๋“ค์— Headless Service ๊ฐ€ ์ƒ์„ฑ
    $ kubectl apply -f ./kube/mcs.yaml --context=${CTX_1} --namespace=${NAMESPACE} 
    multiclusterservice.networking.gke.io/mcs-for-asm-ingressgateway created
    
    kubectl --context=${CTX_1} get mcs --namespace=${GATEWAY_NAMESPACE}
    NAME                         AGE
    mcs-for-asm-ingressgateway   15s
    
    $ kubectl --context=${CTX_1} --namespace=${NAMESPACE} get svc
    NAME                           TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)                                      AGE
    istio-ingressgateway           LoadBalancer   10.76.6.163   34.69.217.99   15021:32737/TCP,80:31856/TCP,443:31521/TCP   19m
    mci-mcs-svc-6v1bymw1gtk1xw8k   ClusterIP      None          <none>         15021/TCP,80/TCP,443/TCP                     39s
    whereami-service               ClusterIP      10.76.9.249   <none>         80/TCP                                       51m
    
    $ kubectl --context=${CTX_2} --namespace=${NAMESPACE} get svc
    NAME                           TYPE           CLUSTER-IP    EXTERNAL-IP     PORT(S)                                      AGE
    istio-ingressgateway           LoadBalancer   10.96.12.66   35.243.92.154   15021:32620/TCP,80:32518/TCP,443:32409/TCP   8m55s
    mci-mcs-svc-6v1bymw1gtk1xw8k   ClusterIP      None          <none>          15021/TCP,80/TCP,443/TCP                     51s
    whereami-service               ClusterIP      10.96.9.147   <none>          80/TCP                                       41m
    
  • MCI ์ƒ์„ฑ

    $ kubectl apply -f ./kube/mci.yaml --context=${CTX_1} --namespace=${NAMESPACE} 
    multiclusteringress.networking.gke.io/mci-for-asm-ingressgateway created
    
    $ kubectl --context=${CTX_1} --namespace=${NAMESPACE} describe mci mci-for-asm-ingressgateway | grep VIP
         f:VIP:
      VIP:        34.111.155.196
    
    $ curl 34.111.155.196
    {
      "cluster_name": "multi-cluster-demo-asia",
      "host_header": "34.111.155.196",
      "pod_name": "whereami-deployment-895464769-d98jj",
      "pod_name_emoji": "โ˜ธ๏ธ",
      "project_id": "kwlee-goog-sandbox",
      "timestamp": "2022-04-29T01:27:15",
      "zone": "asia-northeast1-c"
    }
    $ curl 34.111.155.196
    {
      "cluster_name": "multi-cluster-demo-asia",
      "host_header": "34.111.155.196",
      "pod_name": "whereami-deployment-895464769-d98jj",
      "pod_name_emoji": "โ˜ธ๏ธ",
      "project_id": "kwlee-goog-sandbox",
      "timestamp": "2022-04-29T01:27:15",
      "zone": "asia-northeast1-c"
    }
    $ curl 34.111.155.196
    {
      "cluster_name": "multi-cluster-demo-us",
      "host_header": "34.111.155.196",
      "pod_name": "whereami-deployment-895464769-7mbf7",
      "pod_name_emoji": "โ›ธ๏ธ",
      "project_id": "kwlee-goog-sandbox",
      "timestamp": "2022-04-29T01:27:16",
      "zone": "us-central1-c"
    }
    $ curl 34.111.155.196
    {
      "cluster_name": "multi-cluster-demo-us",
      "host_header": "34.111.155.196",
      "pod_name": "whereami-deployment-895464769-clfgv",
      "pod_name_emoji": "๐Ÿคž๐Ÿป",
      "project_id": "kwlee-goog-sandbox",
      "timestamp": "2022-04-29T01:27:20",
      "zone": "us-central1-c"
    }
    

๊ธฐ๋ณธ์ ์œผ๋กœ HTTP Load Balancing ๋กœ ๋“ค์–ด์˜ค๋Š” ์š”์ฒญ์€ ํด๋ผ์ด์–ธํŠธ์™€ ๊ฐ€์žฅ ๊ฐ€๊นŒ์šด PoP(GFE) ์„ ํ†ตํ•ด ๊ฐ€๊นŒ์šด ์ง€์—ญ์˜ ํด๋Ÿฌ์Šคํ„ฐ๋กœ ๋“ค์–ด๊ฐ. ํ˜„์žฌ ๊ตฌ์„ฑ์—์„œ๋„ ๋™์ผํ•˜์ง€๋งŒ, ํด๋Ÿฌ์Šคํ„ฐ ๋‚ด๋ถ€์˜ MultiClusterMesh ์„ค์ •์œผ๋กœ ์ธํ•ด ํด๋ผ์ด์–ธํŠธ ์œ„์น˜์™€ ๊ด€๊ณ„ ์—†์ด ํŠธ๋ž˜ํ”ฝ์ด ๋‘ ํด๋Ÿฌ์Šคํ„ฐ๋กœ ๋ถ€ํ•˜ ๋ถ„์‚ฐ๋จ multicluster mesh ๋ฅผ ์„ค์ •ํ•˜์ง€ ์•Š์œผ๋ฉด, ๋‹จ์ผ ํด๋Ÿฌ์Šค๋กœ๋งŒ ํŠธ๋ž˜ํ”ฝ ์ „๋‹ฌ๋จ..

ํ•„์š” ์‹œ, istio-ingressgateway ์˜ Service Type ์„ clusterip ๋กœ ๋ณ€๊ฒฝํ•˜๋ฉด, ๊ธฐ์กด ์ƒ์„ฑํ•œ L4 ๋ฅผ ํ†ตํ•œ ์™ธ๋ถ€ ์ ‘๊ทผ์„ ๋ง‰์„ ์ˆ˜ ์žˆ์Œ.

$ kubectl --context=${CTX_1} --namespace=${NAMESPACE} get svc
NAME                           TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                    AGE
istio-ingressgateway           ClusterIP   10.76.6.163   <none>        15021/TCP,80/TCP,443/TCP   88m
mci-mcs-svc-6v1bymw1gtk1xw8k   ClusterIP   None          <none>        15021/TCP,80/TCP,443/TCP   69m
whereami-service               ClusterIP   10.76.9.249   <none>        80/TCP                     121m

$ kubectl --context=${CTX_2} --namespace=${NAMESPACE} get svc
NAME                           TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                    AGE
istio-ingressgateway           ClusterIP   10.96.12.66   <none>        15021/TCP,80/TCP,443/TCP   78m
mci-mcs-svc-6v1bymw1gtk1xw8k   ClusterIP   None          <none>        15021/TCP,80/TCP,443/TCP   69m
whereami-service               ClusterIP   10.96.9.147   <none>        80/TCP                     110m
admin_@cloudshell:~/multi-cluster-with-asm (kwlee-goog-sandbox)$

image