Skip to content

Commit ff6fd48

Browse files
committed
update paper
1 parent 11679ef commit ff6fd48

File tree

1 file changed

+7
-5
lines changed

1 file changed

+7
-5
lines changed

paper.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: 'RCAEval: A Benchmark for Multimodal Root Cause Analysis'
2+
title: 'RCAEval: A Framework for Multimodal Root Cause Analysis'
33
tags:
44
- Python
55
- root cause analysis
@@ -22,8 +22,8 @@ archive_doi: 10.5281/zenodo.15616876
2222

2323
# Summary
2424

25-
RCAEval is an open-source framework for multimodal root cause analysis (RCA). It provides 15+ ready-to-use RCA tools including metric-based, trace-based, and multi-source RCA approaches. It also provides comprehensive datasets containing 735 failure cases collected from real-world software systems.
26-
RCAEval is a Python framework, open-source on GitHub, and installable via PyPi. The datasets are available in Zenodo which can be programmatically downloaded. RCAEval is the first framework to support many reproducible RCA tools and comprehensive benchmark datasets with diverse fault types and modality, enabling researchers to evaluate RCA methods under realistic conditions.
25+
RCAEval is an open-source framework for multimodal root cause analysis (RCA). It provides 20+ ready-to-use RCA tools including metric-based, trace-based, and multi-source RCA approaches. It is accompanied by comprehensive datasets containing 735 failure cases from real-world software systems.
26+
RCAEval is a Python framework, open-source on GitHub, and installable via PyPi. The datasets are available in Zenodo which can be programmatically downloaded. RCAEval provides many reproducible RCA tools and multimodal benchmark datasets, enabling researchers to evaluate RCA methods under realistic conditions.
2727

2828
# Statement of need
2929

@@ -39,7 +39,7 @@ RCAEval targets researchers developing new RCA algorithms, practitioners evaluat
3939

4040
Several tools and datasets exist for RCA, but none provide comprehensive coverage of multimodal telemetry with reproducible methods. Existing libraries focus on metric-based RCA, supporting methods like Bayesian networks and Granger causality, but lack support for log and trace analysis. Available datasets provide limited fault types and no benchmarking framework for systematic evaluation. Commercial observability platforms offer automated root cause analysis features, but their proprietary nature prevents reproducible research comparisons.
4141

42-
RCAEval distinguishes itself by providing the first open-source benchmark framework that combines 15 reproducible RCA tools [@pham2025rcaeval] with large-scale datasets with multimodal observability data. This enables fair, systematic comparison of RCA methods under realistic failure scenarios.
42+
RCAEval distinguishes itself by providing the first open-source benchmark framework that combines 20+ reproducible RCA tools with large-scale datasets with multimodal observability data. This enables fair, systematic comparison of RCA methods under realistic failure scenarios.
4343

4444
![Overview of RCAEval benchmark framework.\label{fig:overview}](docs/readme.jpg){ width=100% }
4545

@@ -87,6 +87,8 @@ The datasets are available in the `data` directory after download. Details about
8787

8888
# Acknowledgements
8989

90-
We would like to express our sincere gratitude to the researchers and developers who created the baselines used in our library. Their work has been instrumental in making this project possible. We deeply appreciate the time, effort, and expertise that have gone into developing and maintaining these resources. This project would not have been feasible without their contributions. This library is built upon my previous published work [@pham2024baro;@pham2024root;@pham2025rcaeval].
90+
We would like to express our sincere gratitude to the researchers and developers who created the baselines used in our library. Their work has been instrumental in making this project possible. We deeply appreciate the time, effort, and expertise that have gone into developing and maintaining these resources. This project would not have been feasible without their contributions.
91+
92+
This framework is built upon my previously published academic work [@pham2024baro;@pham2024root;@pham2025rcaeval] on RCA for microservice systems. As I am working toward general root cause analysis, this software paper positions the framework for general RCA task without limiting itself to microservice systems. Future improvement of this framework focuses on the inclusion of RCA methods and datasets from different fields.
9193

9294
# References

0 commit comments

Comments
 (0)