ingress

[joshuacox]

How do I get ingress to the ml-pipeline-ui service if I'm on k3s?

I had to install istio from istioctl with the values.global.platform=k3s flag:

istioctl install --set profile=default --set values.global.platform=k3s --skip-confirmation

and then installed the pipelines with:

  kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=${THIS_KUBEFLOW_PIPELINE_VERSION}"
  kubectl wait --for condition=established --timeout=60s crd/applications.app.k8s.io
  kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/env/platform-agnostic?ref=${THIS_KUBEFLOW_PIPELINE_VERSION}"

kubectl port-forward works just fine:

kubectl port-forward -n kubeflow svc/ml-pipeline-ui 8080:80
Forwarding from 127.0.0.1:8080 -> 3000
Forwarding from [::1]:8080 -> 3000
Handling connection for 8080

http://127.0.0.1:8080/#/pipelines

^works just fine in my browser. However ingress does not seem to be able to contact the service:

apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
  name: ml-pipeline-ui-gateway
  namespace: kubeflow
spec:
  # The selector matches the ingress gateway pod labels.
  # If you installed Istio using Helm following the standard documentation, this would be "istio=ingress"
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "kfpui.example.com"
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: ml-pipeline-ui
  namespace: kubeflow
spec:
  hosts:
  - "kfpui.example.com"
  gateways:
  - ml-pipeline-ui-gateway
  http:
  - name: "ml-pipeline-ui-routes"
    route:
    - destination:
        port:
          number: 80 
        host: ml-pipeline-ui.kubeflow.svc.cluster.local

cat /etc/hosts
10.0.0.3 kfpui.example.com

curl -s -I -HHost:kfpui.example.com "http://10.0.0.3/" -v   

istio-proxy@istio-ingressgateway-865c448856-j9vdg:/$ curl ml-pipeline-ui.kubeflow.svc.cluster.local -v
* Host ml-pipeline-ui.kubeflow.svc.cluster.local:80 was resolved.
* IPv6: (none)
* IPv4: 10.43.205.59
*   Trying 10.43.205.59:80...
* connect to 10.43.205.59 port 80 from 10.42.1.5 port 54498 failed: Connection refused
* Failed to connect to ml-pipeline-ui.kubeflow.svc.cluster.local port 80 after 6128 ms: Couldn't connect to server
* Closing connection
curl: (7) Failed to connect to ml-pipeline-ui.kubeflow.svc.cluster.local port 80 after 6128 ms: Couldn't connect to server
istio-proxy@istio-ingressgateway-865c448856-j9vdg:/$ curl ml-pipeline-ui.kubeflow.svc.cluster.local:3000 -v
* Host ml-pipeline-ui.kubeflow.svc.cluster.local:3000 was resolved.
* IPv6: (none)
* IPv4: 10.43.205.59
*   Trying 10.43.205.59:3000...
* connect to 10.43.205.59 port 3000 from 10.42.1.5 port 33400 failed: Connection timed out
* Failed to connect to ml-pipeline-ui.kubeflow.svc.cluster.local port 3000 after 136660 ms: Couldn't connect to server
* Closing connection
curl: (28) Failed to connect to ml-pipeline-ui.kubeflow.svc.cluster.local port 3000 after 136660 ms: Couldn't connect to server

some more details:

k get po -n kubeflow
NAME                                               READY   STATUS    RESTARTS      AGE
cache-deployer-deployment-5b569ff4-pcvhd           1/1     Running   1 (41h ago)   41h
cache-server-5589754fb5-cvrgf                      1/1     Running   0             41h
metadata-envoy-deployment-bf4db48c8-s7fqr          1/1     Running   0             41h
metadata-grpc-deployment-7cd994d449-qcf2q          1/1     Running   7 (41h ago)   41h
metadata-writer-bb74cb4b-t7927                     1/1     Running   0             41h
ml-pipeline-556c9c6b55-sfjtr                       1/1     Running   1 (41h ago)   41h
ml-pipeline-persistenceagent-784b777bdd-jk4k2      1/1     Running   0             41h
ml-pipeline-scheduledworkflow-84d49cd5d5-rkhwg     1/1     Running   1 (41h ago)   41h
ml-pipeline-ui-648c7b645c-ls2nr                    1/1     Running   0             41h
ml-pipeline-viewer-crd-7d584d5978-czmqh            1/1     Running   0             41h
ml-pipeline-visualizationserver-77d9bbcc85-b2rxr   1/1     Running   0             41h
mysql-5bb7cf8b95-zbjpt                             1/1     Running   0             41h
seaweedfs-58b784d48c-5nsf8                         1/1     Running   0             41h
workflow-controller-6b89c5dd44-fl27r               1/1     Running   3 (40h ago)   41h

k get svc -n kubeflow
NAME                              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                   AGE
cache-server                      ClusterIP   10.43.229.191   <none>        443/TCP                                                   41h
metadata-envoy-service            ClusterIP   10.43.145.109   <none>        9090/TCP                                                  41h
metadata-grpc-service             ClusterIP   10.43.183.226   <none>        8080/TCP                                                  41h
minio-service                     ClusterIP   10.43.70.215    <none>        9000/TCP                                                  41h
ml-pipeline                       ClusterIP   10.43.111.125   <none>        8888/TCP,8887/TCP                                         41h
ml-pipeline-ui                    ClusterIP   10.43.205.59    <none>        80/TCP                                                    41h
ml-pipeline-visualizationserver   ClusterIP   10.43.79.170    <none>        8888/TCP                                                  41h
mysql                             ClusterIP   10.43.153.181   <none>        3306/TCP                                                  41h
seaweedfs                         ClusterIP   10.43.101.10    <none>        8111/TCP,8333/TCP,9333/TCP,19333/TCP,18888/TCP,8888/TCP   41h

[juliusvonkohout]

See #12851