Intelligent Log Analysis for the HSF CondDB

Author: michmx

_gsocorgs/2025/bnl.md

@@ -0,0 +1,12 @@
+---
+title: "Brookhaven National Laboratory"
+author: "Michel Hernandez Villanueva"
+layout: default
+organization: BNL
+logo: BNL-logo.png
+description: |
+  Brookhaven National Laboratory (BNL) is a multipurpose research laboratory located in Upton, New York. 
+  It is operated by Brookhaven Science Associates for the U.S. Department of Energy. 
+---
+
+{% include gsoc_proposal.ext %}

_gsocprojects/2025/project_HSFCondDB.md

@@ -0,0 +1,20 @@
+---
+project: HSFCondDB
+title: HSF Conditions Database 
+layout: default
+description: |
+  In high-energy physics (HEP), conditions databases play a critical role in managing non-event data. 
+  This includes calibration constants, alignment parameters, and detector conditions, which evolve over time. 
+  These databases ensure that analysis software can access the correct calibration and alignment data corresponding to 
+  the detector’s state at any given time, enabling accurate physics measurements.
+  
+  The [HEP Software Foundation](https://hepsoftwarefoundation.org/) (HSF) proposes a Conditions Database reference 
+  for HEP and Nuclear Physics experiments around the world. Several experts have converged on a common design for 
+  conditions data access management [arXiv:1901.05429](https://arxiv.org/abs/1901.05429). This reference implementation 
+  has been successfully operating within the [sPHENIX](https://www.sphenix.bnl.gov/) experiment for nearly two years 
+  and is currently being adopted by [Belle II](https://www.belle2.org/). Additionally, other collaborations, including 
+  [ePIC](https://www.bnl.gov/eic/epic.php) and the [Einstein Telescope](https://www.et-gw.eu/), have expressed interest 
+  in evaluating its suitability for their needs.
+---
+
+{% include gsoc_project.ext %}

_gsocproposals/2025/proposal_HSFCondDB_AILogAnalysis.md

@@ -0,0 +1,59 @@
+---
+title: Intelligent Log Analysis for the HSF Conditions Database 
+layout: gsoc_proposal
+project: HSFCondDB
+year: 2025
+difficulty: medium
+duration: 350
+mentor_avail: June-October
+organization:
+  - BNL
+---
+
+## Description
+
+The [nopayloaddb](https://github.com/BNLNPPS/nopayloaddb) project works as an implementation of the Conditions Database 
+reference for the HSF. It provides a RESTful API for managing payloads, global tags, payload types, and associated data. 
+
+Our current system, composed of Nginx, Django, and database ([link to helm chart](https://github.com/BNLNPPS/nopayloaddb-charts)), 
+lacks a centralized logging solution making it difficult to effectively monitor and troubleshoot issues. 
+This task will address this deficiency by implementing a centralized logging system aggregating logs from multiple 
+components, and develop a machine learning model to perform intelligent log analysis.  The model will identify unusual 
+log entries indicative of software bugs, database bottlenecks, or other performance issues, allowing us to address 
+problems before they escalate. Additionally, by analyzing system metrics, the model will provide insights for an optimal 
+adjustment of parameters during periods of increased request rates.
+
+## Steps
+
+1. Set up a centralized logging system   
+2. Collect and structure logs from Nginx, Django, and the database  
+3. Develop an ML model for log grouping and anomaly detection 
+4. Implement Kubernetes-based database with replication 
+5. Train an ML model to optimize Kubernetes parameters dynamically
+
+
+## Expected Results
+
+* A centralized logging system for improved monitoring and troubleshooting 
+* ML-powered anomaly detection
+* ML-driven dynamic configuration for optimal performance
+
+## Requirements
+
+* Python and basic understanding of ML frameworks
+* Kubernetes, basic understanding, k8s, Helm, Operators, OpenShift
+* Django and Nginx, basic understanding of web frameworks and logging
+* Database knowledge, PostgreSQL, database replication
+
+
+## Mentors
+
+- **Ruslan Mashinistov [[email protected]](mailto:[email protected]) BNL** 
+- John S. De Stefano Jr. [[email protected]](mailto:[email protected]) BNL
+- Michel Hernandez Villanueva [[email protected]](mailto:[email protected]) BNL
+
+
+## Links
+
+* Django REST API: https://github.com/BNLNPPS/nopayloaddb 
+* Automized deployment with helm-chart: https://github.com/BNLNPPS/nopayloaddb-charts

gsoc/2025/mentors.md

@@ -18,13 +18,16 @@ layout: plain
 * Stephan Lachnit [[email protected]](mailto:[email protected]) DESY
 * David Lange [[email protected]](mailto:[email protected]) CompRes
 * Serguei Linev [[email protected]](mailto:[email protected]) GSI 
+* Ruslan Mashinistov [[email protected]](mailto:[email protected]) BNL
 * Peter McKeown [[email protected]](mailto:[email protected]) CERN
 * Felice Pantaleo [[email protected]](mailto:[email protected]) CERN
 * Giacomo Parolini [[email protected]](mailto:[email protected]) CERN
 * Alexander Penev [[email protected]](mailto:[email protected]) CompRes/University of Plovdiv, BG
 * Mayank Sharma [[email protected]](mailto:[email protected]) UMich
 * Simon Spannagel [[email protected]](mailto:[email protected]) DESY
+* John De Stefano [[email protected]](mailto:[email protected]) BNL
 * Graeme Stewart [[email protected]](mailto:[email protected]) CERN
 * Martin Vasilev [[email protected]](mailto:[email protected]) University of Plovdiv, BG
 * Vassil Vassilev [[email protected]](mailto:[email protected]) CompRes
+* Michel Hernandez Villanueva [[email protected]](mailto:[email protected]) BNL
 * Valentin Volkl [[email protected]](mailto:[email protected]) CERN