Skip to content

Remove sources-api-go Dependency for On-Prem Deployments#46

Closed
ELK4N4 wants to merge 7 commits intoinsights-onprem:mainfrom
ELK4N4:feat/sources-by-koku
Closed

Remove sources-api-go Dependency for On-Prem Deployments#46
ELK4N4 wants to merge 7 commits intoinsights-onprem:mainfrom
ELK4N4:feat/sources-by-koku

Conversation

@ELK4N4
Copy link

@ELK4N4 ELK4N4 commented Jan 6, 2026

Overview

This PR removes the dependency on sources-api-go for on-prem deployments by eliminating the application type lookup and configuring the housekeeper to work directly with Koku's Kafka events.

Motivation

As part of the migration to make Koku the single source of truth for sources:

  • ros-ocp-backend previously made a REST call to sources-api-go at startup to get the cost-management application type ID
  • This is no longer necessary since Koku now publishes source deletion events directly
  • Simplifies the on-prem architecture by removing inter-service dependencies

Changes

Modified Files

File Changes
internal/config/config.go Added ONPREM configuration flag
internal/services/housekeeper/sourcesCleaner.go Modified to skip application type filtering when ONPREM is enabled

Key Changes

1. New ONPREM Configuration Flag

Added ONPREM environment variable to the config:

// config.go
ONPREM bool

2. Simplified Sources Cleaner

When ONPREM=true:

  • Skips the REST call to sources-api-go to fetch application type ID
  • Uses a constant application type ID (0) for cost-management
  • Still filters Kafka events by Application_type_id == 0

Application Type ID

For on-prem deployments, the application type ID is hardcoded to 0 (constant OnPremCostAppID). This is consistent with Koku, which publishes Kafka events with Application_type_id: 0. The housekeeper still filters events by application type ID, but uses the hardcoded value 0 instead of fetching it from sources-api-go.

Before (cloud mode):

1. On startup: GET /api/sources/v1.0/application_types?filter[name][eq]=/insights/platform/cost-management
2. Store application_type_id from response
3. On Kafka event: Filter by application_type_id

After (on-prem mode):

1. On startup: Set cost_app_id = 0 (constant)
2. On Kafka event: Filter by Application_type_id == 0

Configuration

Set the following environment variable to enable on-prem mode:

ONPREM=true

Testing

  • Verified housekeeper correctly processes Kafka events from Koku
  • Tested cluster cleanup when sources are deleted
  • Confirmed no REST calls to sources-api-go when ONPREM is enabled

Related PRs

  • koku: Enables Sources API and publishes Kafka events on source deletion
  • cost-helm-chart: Helm chart updates to configure ONPREM flag for ros-ocp-backend

@ELK4N4 ELK4N4 changed the title On-Prem Source Deletion Event Handling Remove sources-api-go Dependency for On-Prem Deployments Jan 25, 2026
@ELK4N4 ELK4N4 requested a review from masayag January 25, 2026 16:53
@ELK4N4 ELK4N4 closed this Jan 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants