Skip to content

Commit e27085f

Browse files
committed
update paper
1 parent 4616e60 commit e27085f

File tree

2 files changed

+6
-27
lines changed

2 files changed

+6
-27
lines changed

paper.bib

Lines changed: 1 addition & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,3 @@
1-
@article{Soldani2018microservice,
2-
title = {The pains and gains of microservices: A Systematic grey literature review},
3-
journal = {Journal of Systems and Software},
4-
volume = {146},
5-
pages = {215-232},
6-
year = {2018},
7-
doi = {10.1016/j.jss.2018.09.016},
8-
author = {Jacopo Soldani and Damian Andrew Tamburri and Willem-Jan {Van Den Heuvel}}
9-
}
10-
11-
@article{Soldani2022rcasurvey,
12-
author = {Soldani, Jacopo and Brogi, Antonio},
13-
title = {Anomaly Detection and Failure Root Cause Analysis in (Micro) Service-Based Cloud Applications: A Survey},
14-
year = {2022},
15-
volume = {55},
16-
number = {3},
17-
doi = {10.1145/3501297},
18-
journal = {ACM Computing Surveys}
19-
}
20-
21-
221
@inproceedings{pham2024baro,
232
title = {BARO: Robust Root Cause Analysis for Microservices via Multivariate Bayesian Online Change Point Detection},
243
author = {Pham, Luan and Ha, Huong and Zhang, Hongyu},
@@ -46,7 +25,7 @@ @inproceedings{pham2024root
4625
booktitle = {Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering},
4726
pages = {706--715},
4827
year = {2024},
49-
doi = {10.1145/3691620.3695063}
28+
doi = {10.1145/3691620.3695065}
5029
}
5130

5231
@inproceedings{pham2025rcaeval,

paper.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -22,19 +22,19 @@ archive_doi: 10.5281/zenodo.15616876
2222

2323
# Summary
2424

25-
RCAEval is an open-source Python framework for root cause analysis (RCA) methods using multimodal data. When failures or incidents occur in software systems, engineers must quickly identify root causes from massive amounts of observable data including time-series metrics, textual logs, and topological tracing data. RCAEval addresses the lack of standardized, reproducible tools and benchmarks in this domain by providing (1) ready-to-use RCA methods spanning metric-based, trace-based, and multi-source approaches, and (2) comprehensive datasets containing 735 failure cases collected from real-world software systems.
25+
RCAEval is an open-source Python framework for root cause analysis (RCA) methods using multimodal data. When failures or incidents occur in large and dynamic systems, humans must quickly identify root causes from massive amounts of observable data including time-series metrics, textual logs, and topological tracing data. RCAEval addresses the lack of standardized, reproducible tools and benchmarks in this domain by providing (1) ready-to-use RCA methods spanning metric-based, trace-based, and multi-source approaches, and (2) comprehensive datasets containing 735 failure cases collected from real-world software systems.
2626

2727
RCAEval is the first framework to support many reproducible RCA tools and comprehensive benchmark datasets with diverse fault types and modality, enabling researchers to evaluate RCA methods under realistic conditions. The library is pip-installable, provides a simple Python API for running experiments, and includes standardized evaluation metrics (AC@k, Avg@k) for fair comparison across methods.
2828

2929
# Statement of need
3030

31-
Modern cloud applications generate massive amounts of telemetry data including metrics, logs, and traces [@Soldani2022rcasurvey]. When failures occur, they can propagate across multiple components, making it challenging for operators to identify root causes from the overwhelming volume of observable data. Root cause analysis (RCA) aims to pinpoint the faulty component and the specific indicators (e.g., CPU usage, error logs) responsible for the failure [@Soldani2022rcasurvey].
31+
Modern large and dynamic systems generate massive amounts of observability data including metrics, logs, and traces. When failures occur, they can propagate across multiple components, making it challenging for operators to identify root causes from the overwhelming volume of observable data. Root cause analysis (RCA) aims to pinpoint the faulty component and the specific indicators responsible for the failure.
3232

33-
Despite growing research interest in automated RCA, the field lacks a standardized, reproducible benchmark. Existing studies typically evaluate on limited systems with few fault types, often using private datasets that prevent fair comparison [@pham2024root]. Available resources provide only single-modality analysis (e.g., metrics only) without support for multimodal data combining logs and traces. Commercial observability platforms offer RCA capabilities but are proprietary and not reproducible for research purposes.
33+
Despite growing research interest in automated RCA, the field lacks a standardized, reproducible benchmark. Existing studies typically evaluate on limited systems with few fault types, often using private datasets that prevent fair comparison. Available resources provide only single-modality analysis (e.g., metrics only) without support for multimodal data combining logs and traces. Commercial observability platforms offer RCA capabilities but are proprietary and not reproducible for research purposes.
3434

3535
RCAEval fills this gap by providing: (1) three large-scale datasets with 735 failure cases across three software systems, covering resource faults (CPU, memory, disk), network faults (delay, packet loss), and code-level faults; (2) multimodal telemetry data including metrics, logs, and traces; (3) 15 reproducible RCA tools implementations including state-of-the-art methods, e.g., BARO [@pham2024baro] and CausalRCA [@Xin2023CausalRCA]; and (4) standardized evaluation metrics for consistent comparison.
3636

37-
RCAEval targets researchers developing new RCA algorithms, practitioners evaluating methods for production deployment, and educators teaching AIOps and site reliability engineering concepts.
37+
RCAEval targets researchers developing new RCA algorithms, practitioners evaluating methods for production deployment, and educators teaching algorithms.
3838

3939
# State of the field
4040

@@ -46,6 +46,6 @@ RCAEval distinguishes itself by providing the first open-source benchmark framew
4646

4747
# Acknowledgements
4848

49-
We acknowledge contributions from the open-source community and the developers of the software systems (Online Boutique, Sock Shop, Train Ticket) used in our benchmark datasets.
49+
We would like to express our sincere gratitude to the researchers and developers who created the baselines used in our library. Their work has been instrumental in making this project possible. We deeply appreciate the time, effort, and expertise that have gone into developing and maintaining these resources. This project would not have been feasible without their contributions. This library is built upon my previous published work [@pham2024baro;@pham2024root;@pham2025rcaeval].
5050

5151
# References

0 commit comments

Comments
 (0)