You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
RCAEval is an open-source framework for multimodal root cause analysis (RCA). It provides 15+ ready-to-use RCA tools including metric-based, trace-based, and multi-source RCA approaches. It also provides comprehensive datasets containing 735 failure cases collected from real-world software systems.
26
-
RCAEval is a Python framework, open-source on GitHub, and installable via PyPi. The datasets are available in Zenodo which can be programmatically downloaded. RCAEval is the first framework to support many reproducible RCA tools and comprehensive benchmark datasets with diverse fault types and modality, enabling researchers to evaluate RCA methods under realistic conditions.
25
+
RCAEval is an open-source framework for multimodal root cause analysis (RCA). It provides 20+ ready-to-use RCA tools including metric-based, trace-based, and multi-source RCA approaches. It is accompanied by comprehensive datasets containing 735 failure cases from real-world software systems.
26
+
RCAEval is a Python framework, open-source on GitHub, and installable via PyPi. The datasets are available in Zenodo which can be programmatically downloaded. RCAEval provides many reproducible RCA tools and multimodal benchmark datasets, enabling researchers to evaluate RCA methods under realistic conditions.
Several tools and datasets exist for RCA, but none provide comprehensive coverage of multimodal telemetry with reproducible methods. Existing libraries focus on metric-based RCA, supporting methods like Bayesian networks and Granger causality, but lack support for log and trace analysis. Available datasets provide limited fault types and no benchmarking framework for systematic evaluation. Commercial observability platforms offer automated root cause analysis features, but their proprietary nature prevents reproducible research comparisons.
41
41
42
-
RCAEval distinguishes itself by providing the first open-source benchmark framework that combines 15 reproducible RCA tools[@pham2025rcaeval] with large-scale datasets with multimodal observability data. This enables fair, systematic comparison of RCA methods under realistic failure scenarios.
42
+
RCAEval distinguishes itself by providing the first open-source benchmark framework that combines 20+ reproducible RCA tools with large-scale datasets with multimodal observability data. This enables fair, systematic comparison of RCA methods under realistic failure scenarios.
43
43
44
44
{ width=100% }
45
45
@@ -87,6 +87,8 @@ The datasets are available in the `data` directory after download. Details about
87
87
88
88
# Acknowledgements
89
89
90
-
We would like to express our sincere gratitude to the researchers and developers who created the baselines used in our library. Their work has been instrumental in making this project possible. We deeply appreciate the time, effort, and expertise that have gone into developing and maintaining these resources. This project would not have been feasible without their contributions. This library is built upon my previous published work [@pham2024baro;@pham2024root;@pham2025rcaeval].
90
+
We would like to express our sincere gratitude to the researchers and developers who created the baselines used in our library. Their work has been instrumental in making this project possible. We deeply appreciate the time, effort, and expertise that have gone into developing and maintaining these resources. This project would not have been feasible without their contributions.
91
+
92
+
This framework is built upon my previously published academic work [@pham2024baro;@pham2024root;@pham2025rcaeval] on RCA for microservice systems. As I am working toward general root cause analysis, this software paper positions the framework for general RCA task without limiting itself to microservice systems. Future improvement of this framework focuses on the inclusion of RCA methods and datasets from different fields.
0 commit comments