Bulk API fails with 503/422 - Unable to connect to the datasource error #1788
Description
Describe the bug
Bulk API fails with below error, with no experiments created and recommendations generated
❌ Bulk Job FAILED with {'503': {'type': 'ERROR', 'message': 'HttpHostConnectException: Unable to connect to the data source. Please try again later. (Unexpected response status: 422)', 'code': 503}}
How to reproduce it
Run Bulk demo on openshift or minikube:
./bulk_service_demo.sh -c openshift/minikube
Expected behavior
No datasource connection error and bulk API should create experiments and generate recommendations
Relevant logs
./bulk_service_demo.sh -c minikube -o quay.io/dinogun210/kruize-operator:0.0.4-rc1
#######################################
# Kruize Demo Setup on minikube
#######################################
🔍 Checking if Kruize deployment is running... Found!
🔄 Cleaning up existing Kruize deployment (including database)...✅ Cleanup complete!
⏳ Waiting for resources to be fully removed... Done!
🔄 Checking if minikube exists...✅ minikube exists!
🔄 Installing Kruize! Please wait...
⏳ Waiting for Kruize service to fully initialize... Done!
✅ Installation of Kruize complete!
🔄 Invoking bulk service...✅ Invoked job_id b8081d7d-1a85-4c8f-8c2b-af0f4f0f4ff5
❌ Bulk Job FAILED with {'503': {'type': 'ERROR', 'message': 'HttpHostConnectException: Unable to connect to the data source. Please try again later. (Unexpected response status: 422)', 'code': 503}}
📌 Job status is available in job_status.json
For detailed logs, look in kruize-bulk-demo.log
ℹ️ Access Kruize UI at http://192.168.49.2:30614
🔖 To explore further, access Kruize UI to list and create experiments, and to view or generate recommendations!
ℹ️ For Kruize CLI commands, refer to the end of ./kruize-bulk-demo.log
🛠️ Kruize installation took 67 seconds
🚀 Kruize experiment creation and recommendations generation took 10 seconds
🕒 Success! Kruize demo setup took 121 seconds
For detailed logs, look in kruize-bulk-demo.log
Environment:
- Kubernetes Cluster: Openshift
- Version: 4.20.6
- OS: Fedora
Additional context
Kruize Pod logs
2026-02-0514:01:33.525 INFO [pool-11-thread-1][DataSourceMetadataOperator.java(245)]-namespaceQuery: sum by (namespace) (avg_over_time(kube_namespace_status_phase{namespace!='' }[15m]))
2026-02-0514:01:33.525 INFO [pool-11-thread-1][DataSourceMetadataOperator.java(246)]-workloadQuery: sum by (namespace, workload, workload_type) (avg_over_time(namespace_workload_pod:kube_pod_owner:relabel{workload!='' }[15m]))
2026-02-0514:01:33.525 INFO [pool-11-thread-1][DataSourceMetadataOperator.java(247)]-containerQuery: sum by (container, image, workload, workload_type, namespace) (avg_over_time(kube_pod_container_info{container!='' }[15m]) * on (pod, namespace) group_left(workload, workload_type) avg_over_time(namespace_workload_pod:kube_pod_owner:relabel{workload!="" }[15m]))
2026-02-0514:01:33.540 ERROR [pool-11-thread-1][BulkJobManager.java(237)]-Error in bulk job manager for job_id b8081d7d-1a85-4c8f-8c2b-af0f4f0f4ff5 due to Unexpected response status: 422
Metadata
Metadata
Assignees
Labels
Type
Projects
Status