feat: Collect CRD metrics from kube-state-metrics as dimensional metrics

[NRhzhao]

Description

This PR adds support for collecting Custom Resource Definition (CRD) metrics from kube-state-metrics as dimensional metrics sent to the New Relic Metric API.

Key Features:

• CRD Metrics Collection: New ksm.enableCustomResourceMetrics config enables collection of all kube_customresource_* metrics from KSM
• Dimensional Metrics: CRD metrics are sent as dimensional metrics to the Metric API (not as entity-based samples), preserving all CRD labels as metric attributes
• Configurable Harvest Period: New ksm.harvestPeriod config allows batching metrics before sending (defaults to scrape interval for consistency)
• Custom Metric Endpoint: New ksm.metricAPIURL config supports proxies or alternative endpoints
• Graceful Shutdown: Proper signal handling (SIGTERM/SIGINT) ensures pending metrics are flushed before pod termination, preventing data loss during rolling updates
• Performance Optimization: Single KSM scrape per cycle (instead of two) reduces KSM endpoint load by 50% when CRD metrics are enabled

Configuration Example:

ksm:
  enableCustomResourceMetrics: true  # Default: false
  harvestPeriod: "30s"               # Optional, defaults to scrape interval (typically 15s)
  metricAPIURL: ""                   # Optional, uses production endpoint by default

Testing:

• ✅ All unit tests pass
• ✅ Feature disabled: zero regression confirmed
• ✅ Live cluster testing: 13,809 data points verified in New Relic staging (4 metric types)
• ✅ Custom harvest period (30s batching) working correctly
• ✅ Graceful shutdown with metric flush confirmed
• ✅ Single scrape optimization verified (50% load reduction)

See COMPREHENSIVE_TEST_REPORT.md for detailed test results.

Type of change

[] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] New feature / enhancement (non-breaking change which adds functionality)
[] Security fix
[] Bug fix (non-breaking change which fixes an issue)

Checklist:

[x] Add changelog entry following the https://github.com/newrelic/nri-kubernetes/blob/main/CONTRIBUTING.md#pull-requests
[x]Documentation has been updated

• This change requires changes in testing:
  • unit tests
  • E2E tests (not required - dimensional metrics don't create entities)

[codecov]

Codecov Report

❌ Patch coverage is 81.10236% with 24 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.00%. Comparing base (0a260a3) to head (7835c74).
⚠️ Report is 57 commits behind head on main.

Files with missing lines	Patch %	Lines
src/ksm/scraper.go	63.33%	16 Missing and 6 partials ⚠️
src/prometheus/metric.go	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1390      +/-   ##
==========================================
+ Coverage   74.41%   75.00%   +0.58%     
==========================================
  Files          53       54       +1     
  Lines        3666     3788     +122     
==========================================
+ Hits         2728     2841     +113     
- Misses        766      771       +5     
- Partials      172      176       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.