Merge pull request #351 from Vaibhavsahu2810/updates/2025/week-6

chore(docs): Add osselot week 6,7,8 docs

Reviewrd-by: shaheem.azmal@siemens.com

Author: shaheemazmalmmd

docs/2025/osselot/updates/2025-07-14.md

@@ -0,0 +1,63 @@
+---
+title: Week 6
+author: Vaibhav Sahu
+tags: [gsoc25, OSSelot]
+---
+
+<!--
+SPDX-License-Identifier: CC-BY-SA-4.0
+SPDX-FileCopyrightText: 2025 Vaibhav Sahu <sahusv4527@gmail.com>
+-->
+
+# Week 6
+
+*(July 8, 2025 – July 14, 2025)*
+
+## Meeting 6
+
+*(July 11, 2025)*
+
+### Attendees
+
+* [Vaibhav Sahu](https://github.com/Vaibhavsahu2810)
+* [Jan Altenberg](https://github.com/JanAltenberg)
+
+### Discussions
+
+* Demonstrated the working prototype of parallelized `ScanCode` scanning within FOSSology.
+* Discussed runtime behavior with multiple workers and how resource constraints are enforced.
+* Reviewed performance benchmarks from busybox test uploads.
+* Collected feedback on user-configurable CLI arguments, some bugs while testing and their integration.
+
+## Updates
+
+* **Parallel ScanCode implementation completed**
+
+  * Fully implemented multiprocessing support in `runscanonfiles.py`.
+  * Introduced CLI parameters for customization:
+    parser.add_argument("--parallel", type=int, default=1, help="Number of parallel processes (will be adjusted based on available memory)")
+    parser.add_argument("--nice-level", type=int, default=10, help="Process nice level (0-19)")
+    parser.add_argument("--max-tasks", type=int, default=1000, help="Max tasks per worker process")
+    parser.add_argument("--heartbeat-interval", type=int, default=60, help="Heartbeat interval in seconds")
+  * Enforced memory and CPU resource limits for each worker process using OS-level controls.
+
+* **Performance testing**
+
+  * Tested using busybox-1.36.1.tar.bz2 uploads.
+  * Scanning time improved drastically — from **\~12 minutes to just \~4 minutes**.
+  * Below are side-by-side comparisons:
+
+    **Single-process run**
+    ![Single-process result](/img/osselot/single.png)
+
+    **Parallelized run (4 processes)**
+    ![Parallel-process result](/img/osselot/parallel.png)
+
+## Plan for Next Week
+
+1. Avoid starting worker threads if available memory is too low. Ensure threads gracefully shut down if they don’t receive sufficient memory (without hard memory limits).
+2. Implement proper cleanup to ensure all worker processes terminate when the user cancels the agent.
+3. Reduce the default number of parallel jobs to a safer baseline.
+4. Finalize and verify heartbeat handling for worker monitoring.
+
+

docs/2025/osselot/updates/2025-07-21.md

@@ -0,0 +1,46 @@
+---
+title: Week 7
+author: Vaibhav Sahu
+tags: [gsoc25, OSSelot]
+---
+
+<!--
+SPDX-License-Identifier: CC-BY-SA-4.0
+SPDX-FileCopyrightText: 2025 Vaibhav Sahu <sahusv4527@gmail.com>
+-->
+
+
+# Week 7
+
+*(July 15, 2025 – July 21, 2025)*
+
+## Meeting 7
+
+*(July 18, 2025)*
+
+### Attendees
+
+* [Vaibhav Sahu](https://github.com/Vaibhavsahu2810)
+* [Jan Altenberg](https://github.com/JanAltenberg)
+* [Shaheem Azmal M MD](https://github.com/shaheemazmalmmd)
+
+### Discussions
+
+* Walked through heartbeat implementation and memory-aware parallelization logic.
+* Discussed challenges with SIGTERM signal handling for agent cancellation.
+* Shaheem shared helpful documentation on process signal handling strategies in Python.
+* Reviewed improvements in process management and configuration defaults.
+
+## Updates
+
+  * Heartbeat functionality is now fully integrated and working as intended for all worker processes.
+  * Removed hardcoded memory limit configuration.
+  * Implemented a dynamic adjustment mechanism that reduces the number of worker processes based on available system memory.
+  * Lowered the default number of parallel jobs to a safer.
+  * Began work on capturing `SIGTERM` to cleanly shut down all worker processes when the agent is canceled.
+  * Facing some technical challenges in implementation—discussed with Shaheem and received relevant documentation for further debugging.
+
+## Plan for Next Week
+
+1. Finalize and fully implement shutdown behavior when a user cancels the agent (SIGTERM handling).
+2. Ensure job status and disclaimers are properly reflected in the **Job View** UI.

docs/2025/osselot/updates/2025-07-28.md

@@ -0,0 +1,43 @@
+---
+title: Week 8
+author: Vaibhav Sahu
+tags: [gsoc25, OSSelot]
+---
+
+<!--
+SPDX-License-Identifier: CC-BY-SA-4.0
+SPDX-FileCopyrightText: 2025 Vaibhav Sahu <sahusv4527@gmail.com>
+-->
+
+
+# Week 8
+
+*(July 22, 2025 – July 28, 2025)*
+
+## Meeting 8
+
+*(July 25, 2025)*
+
+### Attendees
+
+* [Vaibhav Sahu](https://github.com/Vaibhavsahu2810)
+* [Jan Altenberg](https://github.com/JanAltenberg)
+
+### Discussions
+
+* Confirmed that thread shutdown now works correctly when a user cancels the agent.
+* Reviewed heartbeat logging enhancements and added ETA .
+* Discussed current inconsistencies in the **FOSSology Dump** mechanism.
+* Jan advised testing of the dump system to help identify bugs.
+
+## Updates
+
+  * Finalized SIGTERM capture logic; worker processes now gracefully stop when the agent is cancelled.
+  * Improved heartbeat log structure for better traceability.
+  * Began preliminary testing of the FOSSology Dump system to identify existing issues.
+  * Jan pointed out that some bugs might be affecting reports.
+
+## Plan for Next Week
+
+1. Conduct thorough testing of all PRs made up to this point.
+2. Continue testing and investigation into the FOSSology Dump system to help identify potential bugs.