PI Dashboard Development - Data Integration - Data Processing & API linking & Error Handling

[jimmysway]

Motivation

When billing scripts fail, services go down we need to handle errors properly, to provide a smooth user experience, we need a strategy to handle these gaps gracefully rather than showing a broken line, while still being transparent about data quality.

Completion Criteria

• The system can accept raw usage data and identify missing periods.
• "Gaps" in data are mathematically interpolated to provide a continuous visual trend.
• Missing data points are clearly distinguishable from actual reported data in the data structure.

Description

• Define Data Interface: Establish how usage data will be structured and passed from the backend to the frontend.
• Implement Interpolation Logic: Create a utility that can take sparse data and fill in the gaps (e.g., using linear interpolation) so the chart appears continuous.
• Handle Edge Cases: Ensure the logic is robust enough to handle scenarios where data is missing for a day due to billing errors, handling cases where the actual cluster services go down, difference between planned/unplanned outages etc.

Completion dates

Desired - 12-04-2025

[jimmysway]

Progress on this has stalled due security policies regarding, allocation attributes and resource attributes in Coldfront. We will likely need a custom solution in order to pass data into the chart. @knikolla

[joachimweyl]

We try to avoid issues of size 8, I suggest cutting this into manageable pieces.