Skip to content

Use insights ros ingress#24

Merged
gciavarrini merged 19 commits intoinsights-onprem:mainfrom
masayag:use-insights-ros-ingress
Sep 26, 2025
Merged

Use insights ros ingress#24
gciavarrini merged 19 commits intoinsights-onprem:mainfrom
masayag:use-insights-ros-ingress

Conversation

@masayag
Copy link

@masayag masayag commented Sep 10, 2025

The insights-ingress-go is being replaced with a leaner version of ingress service to serve the minimal functionality required to process upload types from the cost-management metrics operator to be processed by the resource optimization processor.
In this PR, both the helm chart and docker-compose are being updated to deploy, configure, use and test the new ingress service.

@masayag masayag force-pushed the use-insights-ros-ingress branch 2 times, most recently from c054519 to dad033f Compare September 11, 2025 16:43
masayag and others added 16 commits September 22, 2025 11:45
@masayag @claude @jordigilh
Fix authentication issues in test-ros-ocp-dataflow.sh and update to i…
86765ce 
…nsights-ros-ingress

This commit resolves HTTP 401 authentication failures and improves the complete data flow test:

**Authentication Fixes:**
- Add automatic KIND cluster setup with Kubernetes service account authentication
- Implement fallback authentication using DEV_SERVICE_ACCOUNT_TOKEN
- Fix kubeconfig endpoint configuration for container network access
- Add comprehensive authentication environment loading and setup

**Service Migration:**
- Replace insights-ingress-go with insights-ros-ingress service
- Update docker-compose configuration for new service requirements
- Add proper environment variables for storage, Kafka, and authentication
- Update health check endpoints from /api/ingress/v1/version to /health

**Test Improvements:**
- Add teardown-test-resources.sh script for complete cleanup
- Fix MinIO verification logic to use recursive CSV file search
- Update test flow to work with automated CSV extraction and processing
- Improve error handling and debugging output for authentication issues

**CI/CD Updates:**
- Update GitHub Actions workflow for new health check endpoints
- Remove trailing whitespace from workflow files

The complete end-to-end data flow now works successfully from upload through
CSV processing to MinIO storage and Kafka event publishing.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@masayag @claude @jordigilh
Update Kubernetes configurations for insights-ros-ingress migration
6cff5fa 
This commit updates the Kubernetes/Helm configurations to support the migration
from insights-ingress-go to insights-ros-ingress service:

**Helm Chart Updates:**
- Update deployment-ingress.yaml with new environment variables for insights-ros-ingress
- Add proper storage, Kafka, and authentication configuration
- Update MinIO statefulset configuration for ros-data bucket support
- Update values.yaml with new service configuration parameters

**Kubernetes Scripts:**
- Update deploy-kind.sh for insights-ros-ingress compatibility
- Update test-k8s-dataflow.sh to work with new service endpoints and authentication

These changes ensure consistency between Docker Compose and Kubernetes deployments
for the insights-ros-ingress service migration.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@masayag @claude @jordigilh
Update GitHub Actions workflows for insights-ros-ingress authentication
311ddf1 
This commit updates both Docker Compose and Helm chart CI/CD workflows to support
the new insights-ros-ingress service with proper Kubernetes authentication:

**Docker Compose Workflow Updates:**
- Add KIND and kubectl installation for authentication setup
- Add authentication setup step that runs setup-ingress-auth.sh
- Verify authentication environment file creation
- Update cleanup to remove authentication resources (KIND cluster, kubeconfig, auth files)
- Enhance test report to highlight authentication flow and insights-ros-ingress testing

**Helm Chart Workflow Updates:**
- Add Podman installation for container inspection capabilities
- Add authentication verification step to check service accounts and secrets
- Enhance failure debugging with authentication status information
- Add specific ingress service log collection for troubleshooting

**Key Improvements:**
- Workflows now properly test the complete authentication flow
- Better error reporting and debugging for authentication issues
- Comprehensive cleanup of authentication resources
- Updated test descriptions to reflect insights-ros-ingress migration

These workflows now test the complete end-to-end flow including:
- Kubernetes authentication setup (KIND cluster + service accounts)
- Authenticated file upload to insights-ros-ingress
- CSV extraction and MinIO storage
- Kafka event publishing and data processing

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@masayag @claude @jordigilh
Add cluster_alias configuration for insights-ros-ingress service
927c511 
This commit addresses the missing cluster_alias field issue in Kafka messages
published by insights-ros-ingress service:

**Docker Compose Configuration:**
- Add multiple cluster alias environment variables (DEFAULT_CLUSTER_ALIAS, CLUSTER_ALIAS, ROS_CLUSTER_ALIAS)
- Try different configuration options to ensure cluster_alias is included in Kafka messages

**Test Script Updates:**
- Update x-rh-identity header to include cluster_alias in the identity metadata
- Add x-rh-cluster-alias header as an additional way to provide cluster information
- Include cluster_alias in test upload requests to prevent validation errors

**Issue Status:**
The insights-ros-ingress service is not currently reading cluster_alias from the
request headers or environment variables and including it in Kafka messages. This
causes ROS processor validation failures due to the required cluster_alias field
being empty.

**Next Steps:**
This issue requires either:
1. Configuration update to insights-ros-ingress service to read cluster_alias from headers/env
2. Update to insights-ros-ingress to support cluster_alias extraction
3. Temporary workaround to disable cluster_alias validation in test environment

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@masayag @claude @jordigilh
Fix CSV validation failure by creating proper ROS-OCP test data
3ed664b 
- Create ros-ocp-test-data.tar.gz with complete 37-column CSV structure
- Fix x-rh-identity header to match test cluster ID (023d9b0e-7ca6-481d-b04f-ea606becd54e)
- Update test script to use new test data instead of incompatible cost-mgmt.tar.gz
- Add ROS_TEST_DATA.md documentation explaining new test data format
- Update bucket path checking to ros-data/ros
- Enhance processor log checking for successful processing

The original cost-mgmt.tar.gz contained only 6 columns which failed validation
in aggregator.go causing "Invalid records in CSV - 1" errors. The new test data
contains all required columns for proper ROS-OCP processing pipeline testing.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@masayag @jordigilh
Remove useless messages
f6d425f 
Signed-off-by: Moti Asayag <masayag@redhat.com>
@masayag @claude @jordigilh
Fix GitHub Actions workflow: reorder steps and eliminate env var dupl…
28d3e07 
…ication

- Move authentication setup before service startup (fixes ingress dependency)
- Export KUBECONFIG to GITHUB_ENV for all subsequent steps
- Remove duplicate MINIO environment variable exports
- Add environment verification logging for debugging
- Include internal code changes in workflow triggers

The ingress service requires Kubernetes authentication to start successfully,
so KIND cluster setup must happen before podman-compose up.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@masayag @claude @jordigilh
Fix GitHub Actions: configure rootless KIND with systemd delegation
5c2a9cf 
- Add systemd delegate configuration for rootless KIND support
- Set KIND_EXPERIMENTAL_PROVIDER=podman globally in GITHUB_ENV
- Add verification of podman socket status
- Remove duplicate KIND provider configuration

Fixes error: "running kind with rootless provider requires setting
systemd property Delegate=yes"

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Moti Asayag <masayag@redhat.com>
@masayag @jordigilh
Remove unsupported argument by podman port
6fc9949 
Signed-off-by: Moti Asayag <masayag@redhat.com>
@masayag @claude @jordigilh
Fix authentication issues in test-ros-ocp-dataflow.sh
4d23d4a 
- Add fix_kubeconfig_ip() function to detect and correct incorrect Kubernetes API server addresses in kubeconfig
- Automatically fix kubeconfig when it points to 0.0.0.0, 127.0.0.1, or localhost instead of actual KIND container IP
- Call kubeconfig fix proactively during service startup to prevent authentication failures
- Resolves HTTP 500 "Authentication failed" errors caused by TokenReview API connection refused

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Moti Asayag <masayag@redhat.com>
@masayag @claude @jordigilh
Ensure GitHub Actions fails when upload test fails
88ee402 
- Add proper exit codes (exit 1) when upload_test_data() fails
- Add exit codes for missing test files and fallback file failures
- Add exit codes for critical verification failures (Kafka events, database)
- Ensure upload failures cause immediate test termination with clear error messages
- Maintain warnings for non-critical failures (MinIO bucket, recommendations)

This ensures the GitHub Actions workflow properly fails when authentication or upload issues occur, preventing false positive test results.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Moti Asayag <masayag@redhat.com>
@masayag @claude @jordigilh
Fix kubeconfig authentication in GitHub Actions workflow
a5c14a9 
- Add explicit kubeconfig IP fixing step before running data flow test
- Get KIND container IP and update kubeconfig server address
- Restart ingress service after kubeconfig update to pick up changes
- Add debugging output to show current and updated kubeconfig server
- Ensure authentication works in CI environment by fixing container network access

This resolves the "Authentication failed" HTTP 500 errors in GitHub Actions by ensuring the kubeconfig points to the correct KIND cluster IP address instead of 0.0.0.0.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@masayag @jordigilh
Fix warning for watching the processror logs
1e3c33e 
Signed-off-by: Moti Asayag <masayag@redhat.com>
@masayag @claude @jordigilh
Fix ingress service port configuration in deploy-kind.sh
f8ed7e4 
The deploy-kind.sh script was failing health checks because it was trying to patch the ingress service with port 3000 instead of the actual port 8080. This caused the NodePort patch to fail, leaving the service as ClusterIP and making the ingress API inaccessible at localhost:30080.

Changes:
- Updated create_nodeport_services() function to use port 8080 for ingress service
- This ensures the ingress service is properly exposed as NodePort on host port 30080
- All health checks now pass successfully

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@masayag @claude @jordigilh
Improve test-k8s-dataflow.sh with proper manifest.json format
d853305 
Enhanced the test script to create a proper manifest.json file that matches the expected format for insights-ros-ingress uploads. This addresses upload processing issues and ensures compatibility with the ingress service.

Changes:
- Generate UUID-based filenames following the pattern {uuid}_openshift_usage_report.{number}.csv
- Create comprehensive manifest.json with all required metadata fields
- Include manifest.json in the upload archive alongside CSV files
- Add cluster metadata and configuration details to match expected format

This improves the test coverage for the insights-ros-ingress upload functionality.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@masayag @claude @jordigilh
Add JWT_SECRET environment variable to ingress deployment
03231b8 
Added JWT_SECRET configuration to the ingress service deployment template and values.yaml. This provides the JWT secret key needed for token generation and validation in the insights-ros-ingress service.

Changes:
- Added JWT_SECRET environment variable to deployment-ingress.yaml template
- Added jwtSecret configuration field to values.yaml with default dev value
- Ensures ingress service has proper JWT configuration for authentication

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@jordigilh jordigilh force-pushed the use-insights-ros-ingress branch 5 times, most recently from 347cacc to da529d3 Compare September 24, 2025 22:21
@jordigilh
Changelog:
5b52a4a 
- Rename oauth2 to auth in Helm chart configuration
- Add ClusterRole/ClusterRoleBinding for insights-ros-ingress
- Update deployment templates and API middleware
- Enhance test and deployment scripts

Signed-off-by: Jordi Gil <jgil@redhat.com>
@jordigilh jordigilh force-pushed the use-insights-ros-ingress branch from da529d3 to 5b52a4a Compare September 24, 2025 23:42
@jordigilh
Disable docker-compose-test-yml workflow for now as it is unclear wha…
14f04dc 
…t is the purpose of running the compose with kind and oauth2 when compose should be ideal for x-rh-identity scenarios, not oauth2

Signed-off-by: Jordi Gil <jgil@redhat.com>
@jordigilh
Copy link

jordigilh commented Sep 25, 2025

@gciavarrini PTAL

gciavarrini
gciavarrini approved these changes Sep 25, 2025
View reviewed changes
deployment/kubernetes/helm/ros-ocp/values.yaml
auth:
enabled: true
allowedOrgs: []
jwtSecret: "dev-jwt-secret-key-for-ros-ingress"
Copy link

@gciavarrini gciavarrini Sep 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this create an infoSec issue? not sure, just wondering

Copy link

@jordigilh jordigilh Sep 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's just a filler, it is not used in our context but it seems like it needs to be provided somehow. 🤷

Copy link

@jordigilh jordigilh Sep 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're safe to remove it. I'll purge it from the ingress code and then create a new PR to remove the references in here.

deployment/kubernetes/helm/ros-ocp/values.yaml
image:
repository: quay.io/insights-onprem/ros-ocp-backend
#TODO: Change to quay.io/insights-onprem/ros-ocp-backend when access is granted
repository: quay.io/jordigilh/ros-ocp-backend
Copy link

@gciavarrini gciavarrini Sep 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to merge it like this? to be sure.

Copy link

@jordigilh jordigilh Sep 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can move it now that I have permissions, I'll let you know.

@jordigilh
fix: Update Kubernetes health checks and resolve test script issues
f5b8431 
- Switch health check endpoints from /api/ingress/v1/version to /ready
  - Update install-helm-chart.sh to use /ready endpoint for ingress checks
  - Update test-k8s-dataflow.sh and test-ocp-dataflow.sh health checks
  - Add /ready endpoint to ingress.yaml for external access
  - Remove unused /health endpoint exposure from deployment-ingress.yaml

- Fix Kafka message format validation in test-k8s-dataflow.sh
  - Generate single-line JSON messages to prevent kafka-console-producer splitting
  - Add all required fields (request_id, b64_identity, metadata, files, object_keys)
  - Ensure topic creation before publishing messages
  - Fix schema validation errors in rosocp-processor

- Fix ID_PROVIDER detection in test-k8s-dataflow.sh
  - Correct kubectl jsonpath query to use .items[0] for label selectors
  - Eliminate false warnings about missing oauth2 configuration

These changes resolve authentication issues, message validation errors, and
improve the reliability of Kubernetes deployment health checks and testing.
@gciavarrini gciavarrini merged commit 8fdcf2b into insights-onprem:main Sep 26, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jordigilh jordigilh jordigilh left review comments

@gciavarrini gciavarrini gciavarrini approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@masayag @jordigilh @gciavarrini