2
2
tile : Intelligent Logging Pipeline
3
3
project : Intelligent Log Analysis for the HSF Conditions Database
4
4
author : Osama Ahmed Tahir
5
- date : 06.09.2025\
5
+ date : 06.09.2025
6
6
year : 2025
7
7
layout : blog_post
8
8
intro : |
@@ -24,7 +24,7 @@ stores metadata and file URLs instead of payloads. However, NopayloadDB
24
24
lacks a centralized logging subsystem. To address these limitations,
25
25
this project proposes an intelligent logging pipeline integrated with
26
26
NopayloadDB. The pipeline combines advanced log aggregation, scalable
27
- storage, and deep learning-- based anomaly detection to reduce downtime
27
+ storage, and deep learning-based anomaly detection to reduce downtime
28
28
and improve operation. The result is enhanced reliability,
29
29
maintainability, and scalability of conditions database services in
30
30
modern HEP experiments.
@@ -138,13 +138,13 @@ DeepLog in a random sequence.
138
138
139
139
<p align =" justify " >
140
140
Here the model thinks "Login" is most likely next event, then "Select
141
- File" and then "Upload File" etc. Hence, the sequence will be \ [ Login,
142
- Select File, Upload File, Submit File, Logout\ ] and with their
143
- respective unique IDs, it will be \ [ 0, 2, 1, 4, 3\ ] . With k=2, the model
144
- predicts the top 2 event IDs as \ [ Login, Select File\ ] , while the true
141
+ File" and then "Upload File" etc. Hence, the sequence will be [ Login,
142
+ Select File, Upload File, Submit File, Logout] and with their
143
+ respective unique IDs, it will be [ 0, 2, 1, 4, 3] . With k=2, the model
144
+ predicts the top 2 event IDs as [ Login, Select File] , while the true
145
145
event is Upload File. Since the true event does not appear in the top 2
146
146
predictions, this case is flagged as an anomaly. When k=3, the top 3
147
- event IDs are \ [ Login, Select File, Upload File\ ] , and the true event
147
+ event IDs are [ Login, Select File, Upload File] , and the true event
148
148
Upload File is included, so it is considered normal. In practice, the
149
149
model checks whether the true event ID appears within the top-k
150
150
predicted IDs: if the true event is not present, the sequence is
@@ -167,7 +167,7 @@ This research will establish a baseline for how the observability and
167
167
diagnostics of a system can benefit the most from artificial
168
168
intelligence. In addition, it will also be beneficial for the open
169
169
source community, scientific research, and enterprise applications. From
170
- the experiment\ ' s point of view, it will provide more reliable and
170
+ the experiment's point of view, it will provide more reliable and
171
171
reproducible physics experiments. This will also enable HEP to
172
172
efficiently allocate resources from insights gained from the system. In
173
173
addition, it will also pave the way for how cutting-edge techniques can
0 commit comments