[Analysis] Deploying with centralized infra

[zelima]

As a developer, I want to follow the README and be able to see running instance in the browser

Acceptance Criteria

• Able to run empty ckan, with just following the REAMDE
• Able to run existing image from ckan-cloud-docker (Eg: honduras, or any)

Taks

• Try follwoing REAMDE
• Document steps
• List outstanding issue
• Do analysis and try to fix them

Analysis

Deployment process worked great itself. All serveces are up and running (including centralized infra). Only problem I have atm is that CKAN is not seem to be talking with centralized DB. Getting authentication errors

Password authentication fials to connecto to db. SQLALCEHMY_URL from secrets does not work

kubectl exec -it ckan-9fdf6b774-mbk4v -n first bash
cat /etc/ckan-conf/secrets/secrets.sh | grep SQL
psql postgresql://first:73021fa192850023c05cbc86@db.ckan-cloud/first

Commands to deoloy ckan with centralized infra

export KUBERNETES_VERSION=v1.11.10
minikube start --kubernetes-version "${KUBERNETES_VERSION}" --vm-driver=virtualbox

# Helm stuff
kubectl --context minikube --namespace kube-system create serviceaccount tiller
kubectl --context minikube create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
helm --kube-context=minikube init --service-account tiller --history-max 2 --upgrade --wait
# Check versions
helm --kube-context=minikube version
kubectl --context minikube -n kube-system delete service tiller-deploy &&\
kubectl --context minikube -n kube-system patch deployment tiller-deploy --patch '
spec:
  template:
    spec:
      containers:
        - name: tiller
          ports: []
          command: ["/tiller"]
          args: ["--listen=localhost:44134"]
'

# Centralized infra
kubectl --context minikube create ns ckan-cloud &&helm upgrade --namespace ckan-cloud "ckan-cloud-infra" ckan --install      --set centralizedInfraOnly=true

# Create namsepace and service account perms
kubectl --context minikube create ns "${CKAN_NAMESPACE}" &&\
kubectl --context minikube --namespace "${CKAN_NAMESPACE}" \
    create serviceaccount "ckan-${CKAN_NAMESPACE}-operator" &&\
kubectl --context minikube --namespace "${CKAN_NAMESPACE}" \
    create role "ckan-${CKAN_NAMESPACE}-operator-role" --verb list,get,create \
                                                       --resource secrets,pods,pods/exec,pods/portforward &&\
kubectl --context minikube --namespace "${CKAN_NAMESPACE}" \
    create rolebinding "ckan-${CKAN_NAMESPACE}-operator-rolebinding" --role "ckan-${CKAN_NAMESPACE}-operator-role" \
                                                                     --serviceaccount "${CKAN_NAMESPACE}:ckan-${CKAN_NAMESPACE}-operator"

# Shortcats
export CCA_HELM_FUNCTIONS_KUBECTL_ARGS="--context minikube"
export CCA_HELM_FUNCTIONS_HELM_ARGS="--kube-context minikube"
export CKAN_CHART=ckan
source cca_helm_functions.sh

# Creating SOLR collection

SOLRCLOUD_POD_NAME=$(kubectl --context minikube -n ckan-cloud get pods -l "app=solr" -o 'jsonpath={.items[0].metadata.name}')
kubectl --context minikube -n ckan-cloud exec $SOLRCLOUD_POD_NAME -- \
    bin/solr create_collection -c ${CKAN_NAMESPACE} -d ckan_default -n ckan_default -force

# Deploy (needs some time? or hang?)
cca_helm_upgrade --install --set replicas=1 --set nginxReplicas=1 --set disableJobs=true --set useCentralizedInfra=true --set noProbes=true --set ckanOperatorImage=viderum/ckan-cloud-docker:cca-operator-latest

# Port forward to ngnix pod
cca_kubectl port-forward $(cca_pod_name nginx) 8080

[akariv]

what is the purpose of cca_helm_upgrade and cca_kubectl here? Where do these commands come from?

…

On Wed, Nov 13, 2019 at 2:09 PM Irakli Mchedlishvili < ***@***.***> wrote: As a developer, I want to follow the README and be able to see running instance in the browser Acceptance Criteria - Able to run empty ckan, with just following the REAMDE - Able to run existing image from ckan-cloud-docker (Eg: honduras, or any) Taks - Try follwoing REAMDE - Document steps - List outstanding issue - Do analysis and try to fix them Analysis Deployment process worked great itself. All serveces are up and running (including centralized infra). Only problem I have atm is that CKAN is not seem to be talking with centralized DB. Getting authentication errors Password authentication fials to connecto to db. SQLALCEHMY_URL from secrets does not work kubectl exec -it ckan-9fdf6b774-mbk4v -n first bash cat /etc/ckan-conf/secrets/secrets.sh | grep SQL psql ***@***.***/first Commands to deoloy ckan with centralized infra export KUBERNETES_VERSION=v1.11.10 minikube start --kubernetes-version "${KUBERNETES_VERSION}" --vm-driver=virtualbox # Helm stuff kubectl --context minikube --namespace kube-system create serviceaccount tiller kubectl --context minikube create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller helm --kube-context=minikube init --service-account tiller --history-max 2 --upgrade --wait # Check versions helm --kube-context=minikube version kubectl --context minikube -n kube-system delete service tiller-deploy &&\ kubectl --context minikube -n kube-system patch deployment tiller-deploy --patch ' spec: template: spec: containers: - name: tiller ports: [] command: ["/tiller"] args: ["--listen=localhost:44134"] ' # Centralized infra kubectl --context minikube create ns ckan-cloud &&helm upgrade --namespace ckan-cloud "ckan-cloud-infra" ckan --install --set centralizedInfraOnly=true # Create namsepace and service account perms kubectl --context minikube create ns "${CKAN_NAMESPACE}" &&\ kubectl --context minikube --namespace "${CKAN_NAMESPACE}" \ create serviceaccount "ckan-${CKAN_NAMESPACE}-operator" &&\ kubectl --context minikube --namespace "${CKAN_NAMESPACE}" \ create role "ckan-${CKAN_NAMESPACE}-operator-role" --verb list,get,create \ --resource secrets,pods,pods/exec,pods/portforward &&\ kubectl --context minikube --namespace "${CKAN_NAMESPACE}" \ create rolebinding "ckan-${CKAN_NAMESPACE}-operator-rolebinding" --role "ckan-${CKAN_NAMESPACE}-operator-role" \ --serviceaccount "${CKAN_NAMESPACE}:ckan-${CKAN_NAMESPACE}-operator" # Shortcats export CCA_HELM_FUNCTIONS_KUBECTL_ARGS="--context minikube" export CCA_HELM_FUNCTIONS_HELM_ARGS="--kube-context minikube" export CKAN_CHART=ckan source cca_helm_functions.sh # Creating SOLR collection SOLRCLOUD_POD_NAME=$(kubectl --context minikube -n ckan-cloud get pods -l "app=solr" -o 'jsonpath={.items[0].metadata.name}') kubectl --context minikube -n ckan-cloud exec $SOLRCLOUD_POD_NAME -- \ bin/solr create_collection -c ${CKAN_NAMESPACE} -d ckan_default -n ckan_default -force # Deploy (needs some time? or hang?) cca_helm_upgrade --install --set replicas=1 --set nginxReplicas=1 --set disableJobs=true --set useCentralizedInfra=true --set noProbes=true --set ckanOperatorImage=viderum/ckan-cloud-docker:cca-operator-latest # Port forward to ngnix pod cca_kubectl port-forward $(cca_pod_name nginx) 8080 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#11>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACAY5IFG23AQ3WG64CYSMTQTPU6RANCNFSM4JM2LRVQ> .

[akariv]

Also - what is CKAN_NAMESPACE?

…

On Wed, Nov 13, 2019 at 5:12 PM Adam Kariv ***@***.***> wrote: what is the purpose of cca_helm_upgrade and cca_kubectl here? Where do these commands come from? On Wed, Nov 13, 2019 at 2:09 PM Irakli Mchedlishvili < ***@***.***> wrote: > As a developer, I want to follow the README and be able to see running > instance in the browser > Acceptance Criteria > > - Able to run empty ckan, with just following the REAMDE > - Able to run existing image from ckan-cloud-docker (Eg: honduras, or > any) > > Taks > > - Try follwoing REAMDE > - Document steps > - List outstanding issue > - Do analysis and try to fix them > > Analysis > > Deployment process worked great itself. All serveces are up and running > (including centralized infra). Only problem I have atm is that CKAN is not > seem to be talking with centralized DB. Getting authentication errors > > Password authentication fials to connecto to db. SQLALCEHMY_URL from > secrets does not work > > kubectl exec -it ckan-9fdf6b774-mbk4v -n first bash > cat /etc/ckan-conf/secrets/secrets.sh | grep SQL > psql ***@***.***/first > > Commands to deoloy ckan with centralized infra > > export KUBERNETES_VERSION=v1.11.10 > minikube start --kubernetes-version "${KUBERNETES_VERSION}" --vm-driver=virtualbox > > # Helm stuff > kubectl --context minikube --namespace kube-system create serviceaccount tiller > kubectl --context minikube create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller > helm --kube-context=minikube init --service-account tiller --history-max 2 --upgrade --wait > # Check versions > helm --kube-context=minikube version > kubectl --context minikube -n kube-system delete service tiller-deploy &&\ > kubectl --context minikube -n kube-system patch deployment tiller-deploy --patch ' > spec: > template: > spec: > containers: > - name: tiller > ports: [] > command: ["/tiller"] > args: ["--listen=localhost:44134"] > ' > > # Centralized infra > kubectl --context minikube create ns ckan-cloud &&helm upgrade --namespace ckan-cloud "ckan-cloud-infra" ckan --install --set centralizedInfraOnly=true > > # Create namsepace and service account perms > kubectl --context minikube create ns "${CKAN_NAMESPACE}" &&\ > kubectl --context minikube --namespace "${CKAN_NAMESPACE}" \ > create serviceaccount "ckan-${CKAN_NAMESPACE}-operator" &&\ > kubectl --context minikube --namespace "${CKAN_NAMESPACE}" \ > create role "ckan-${CKAN_NAMESPACE}-operator-role" --verb list,get,create \ > --resource secrets,pods,pods/exec,pods/portforward &&\ > kubectl --context minikube --namespace "${CKAN_NAMESPACE}" \ > create rolebinding "ckan-${CKAN_NAMESPACE}-operator-rolebinding" --role "ckan-${CKAN_NAMESPACE}-operator-role" \ > --serviceaccount "${CKAN_NAMESPACE}:ckan-${CKAN_NAMESPACE}-operator" > > # Shortcats > export CCA_HELM_FUNCTIONS_KUBECTL_ARGS="--context minikube" > export CCA_HELM_FUNCTIONS_HELM_ARGS="--kube-context minikube" > export CKAN_CHART=ckan > source cca_helm_functions.sh > > # Creating SOLR collection > > SOLRCLOUD_POD_NAME=$(kubectl --context minikube -n ckan-cloud get pods -l "app=solr" -o 'jsonpath={.items[0].metadata.name}') > kubectl --context minikube -n ckan-cloud exec $SOLRCLOUD_POD_NAME -- \ > bin/solr create_collection -c ${CKAN_NAMESPACE} -d ckan_default -n ckan_default -force > > # Deploy (needs some time? or hang?) > cca_helm_upgrade --install --set replicas=1 --set nginxReplicas=1 --set disableJobs=true --set useCentralizedInfra=true --set noProbes=true --set ckanOperatorImage=viderum/ckan-cloud-docker:cca-operator-latest > > # Port forward to ngnix pod > cca_kubectl port-forward $(cca_pod_name nginx) 8080 > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#11>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AACAY5IFG23AQ3WG64CYSMTQTPU6RANCNFSM4JM2LRVQ> > . >

[zelima]

@akariv they are just executing kubectl and helm command with some checks prior, with -n CKAN_NAMSPACE flag. They are from here https://github.com/ViderumGlobal/ckan-cloud-helm/blob/master/cca_helm_functions.sh

CKAN_NAMESPACE is ckan instance id. Any name you give it basically...

[akariv]

I'm guessing it would be good to incorporate these scripts in CCO

…

On Mon, Nov 18, 2019 at 8:35 AM Irakli Mchedlishvili < ***@***.***> wrote: @akariv <https://github.com/akariv> they are just executing kubectl and helm command with some checks prior, with -n CKAN_NAMSPACE flag. They are from here https://github.com/ViderumGlobal/ckan-cloud-helm/blob/master/cca_helm_functions.sh CKAN_NAMESPACE is ckan instance id. Any name you give it basically... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#11>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACAY5ILHQMEFLSZE34CRALQUIZUDANCNFSM4JM2LRVQ> .