[Feature Request] Performance Analyzer should index data into Opensearch to make it easy to generate performance dashboards over time

[jhenley-icr-team]

Is your feature request related to a problem? Please describe

"Performance Analyzer is a plugin that contains an agent and REST API that allow you to query numerous cluster performance metrics, including aggregations of those metrics.
...
Performance Analyzer uses /dev/shm for temporary storage. During heavy cluster workloads, Performance Analyzer can use up to 1 GB of space."

Currently we periodically have periods of increased load and would like an easier way to track when our clusters performance is being negatively impacted, as well as see a longer duration of performance over time.

Describe the solution you'd like

Currently performance analyzer puts data in /dev/shm, would be useful to also have this data present indexed in Opensearch to make performance oriented dashboards of the clusters performance over time.

Related component

Other

Describe alternatives you've considered

Have considered using PerfTop dashboards or a thrid party option but thought this might be a useful solution for others to track their Opensearch's cluster's performance in Opensearch.

Additional context

No response

[andrross]

Thanks @jhenley-icr-team. I think this belongs in the performance-analyzer repository. The core offers extension points for plugins to create indexes if desired. I think this is a question as to what the PA plugin itself should use for storage, as opposed to a core feature. Do you agree?

[jhenley-icr-team]

@andrross oh excellent point, my bad we should move this over to PA

[andrross]

@opensearch-project/admin Can this be transferred to the performance analyzer repo?

[andrross]

Catch All Triage - 1 2