404 Not Found error with KServe model using Istio Ingress Gateway despite following guide

[csanadpoda]

Validation Checklist

• I confirm that this is a Kubeflow-related issue.
• I am reporting this in the appropriate repository.
• I have followed the Kubeflow installation guidelines.
• The issue report is detailed and includes version numbers where applicable.
• I have considered adding my company to the adopters page to support Kubeflow and help the community, since I expect help from the community for my issue (see 1. and 2.).
• This issue pertains to Kubeflow development.
• I am available to work on this issue.
• You can join the CNCF Slack and access our meetings at the Kubeflow Community website. Our channel on the CNCF Slack is here #kubeflow-platform.

Version

1.10.2

Detailed Description

Issue: 404 Not Found error with KServe model using Istio Ingress Gateway

Description:

I followed the official guide to deploy a KServe model (sklearn-iris) using Istio as the ingress gateway. However, I'm encountering a 404 Not Found error when accessing the model through the sslip.io URL.

Steps Taken:

1. Followed the guide from the KServe GitHub repository Istio + Dex Setup.

   • Specifically, I followed the section Why I am getting 404 not found error? to configure Istio and KServe.
   • I deployed an InferenceService with the sklearn-iris model.

2. DNS Configuration:

   • Applied the Magic DNS (sslip.io) configuration by running:

     kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.21.0/serving-default-domain.yaml

3. Istio Configuration:

   • I followed the steps under the section Why I am getting 403 Forbidden RBAC access denied error? to set:

     sidecar.istio.io/inject: "false"

4. Model URL:
   The model's URL is http://sklearn-iris-my-namespace.IP.sslip.io.

5. Current Issue:
   Despite these configurations, I still encounter a 404 Not Found nginx landing site when attempting to access the model through the configured URL (http://sklearn-iris-my-namespace.IP.sslip.io).

   • Logs show a 200 OK status but routing to an incorrect service (kserve-models-web-app.kubeflow.svc.cluster.local) rather than the intended model service.
   • NGINX is responding with the 404 error.
   • Port forwarding does not work due to the ExternalName service (mistral-7b-instr-vllm-predictor) being incorrectly defined.

Expected Behavior:

I expect the 404 error to be resolved, and the traffic should reach the correct KServe model service via Istio's ingress gateway.

Steps Taken So Far:

1. Rechecked the VirtualService configuration to ensure correct routing to the KServe model.
2. Verified that the DNS resolves correctly to the assigned IP.
3. Attempted to port-forward but encountered issues with the ExternalName service.
4. Checked Istio's logs but encountered misleading 200 OK responses (since the website does load, but it loads to show me the nginx 404 error page).

Suggestions/Requests:

• Any suggestions on correctly configuring the service for the KServe model to route via Istio.
• Insights into why NGINX is returning a 404 despite the service running successfully.

Steps to Reproduce

1. Deploy Kubeflow via Manifests
2. Expose Central Dashboard via sslip.io
3. Deploy example inference service
4. Get public url and visit it
5. Get 404 nginx site
6. Same happens if I try the dex auth via Python: print("Response: ", res) returns

Status Code:  404
Response:  <Response [404]>

Screenshots or Videos (Optional)

No response

[juliusvonkohout]

Whatever you reference is not owned by Kubeflow Please take a look at https://github.com/kubeflow/manifests/blob/master/.github/workflows/kserve_test.yaml

[csanadpoda]

Whatever you reference is not owned by Kubeflow Please take a look at https://github.com/kubeflow/manifests/blob/master/.github/workflows/kserve_test.yaml

I did, I still get

TOKEN=$(kubectl create token default-editor -n my-user --audience=istio-ingressgateway.istio-system.svc.cluster.local --duration=24h)
curl -ik -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" https://mistral-7b-instruct-vllm-predictor-my-user.my-ip.sslip.io/v1/models/mistral-7b-instruct-vllm:predict
HTTP/2 401 
date: Fri, 30 Jan 2026 10:39:42 GMT
content-type: text/plain
content-length: 28
www-authenticate: Bearer realm="http://mistral-7b-instruct-vllm-predictor-my-user.my-ip.sslip.io/v1/models/mistral-7b-instruct-vllm:predict", error="invalid_token"
strict-transport-security: max-age=31536000; includeSubDomains

Jwt issuer is not configured%                           

Btw the documentation I linked is directly referenced in the Kubeflow official documentation here:

[juliusvonkohout]

Whatever you reference is not owned by Kubeflow Please take a look at https://github.com/kubeflow/manifests/blob/master/.github/workflows/kserve_test.yaml

I did, I still get

TOKEN=$(kubectl create token default-editor -n my-user --audience=istio-ingressgateway.istio-system.svc.cluster.local --duration=24h)
curl -ik -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" https://mistral-7b-instruct-vllm-predictor-my-user.my-ip.sslip.io/v1/models/mistral-7b-instruct-vllm:predict
HTTP/2 401 
date: Fri, 30 Jan 2026 10:39:42 GMT
content-type: text/plain
content-length: 28
www-authenticate: Bearer realm="http://mistral-7b-instruct-vllm-predictor-my-user.my-ip.sslip.io/v1/models/mistral-7b-instruct-vllm:predict", error="invalid_token"
strict-transport-security: max-age=31536000; includeSubDomains

Jwt issuer is not configured%                           

Btw the documentation I linked is directly referenced in the Kubeflow official documentation here:

@LogicalGuy77 @Griffin-Sullivan then we also have to update the incorrect documentation on the website. may you raise a PR or include this in GSOC ? We should link to https://github.com/kubeflow/manifests/blob/master/.github/workflows/kserve_test.yaml. @csanadpoda reagrding "Jwt issuer is not configured%" did you configure it if you are not using the default kind? See https://github.com/kubeflow/manifests#oauth2-proxy