0.1.1

Author: mcrespoae

README.md

@@ -6,7 +6,7 @@
 
 ## Overview
 
-`metrit` is a Python package designed to simplify the process of measuring the execution resources of your functions through a straightforward decorator.
+`metrit` is a Python package designed to simplify the process of measuring the execution resources of your functions through a straightforward decorator. `metrit` decorator is a powerful tool for detailed performance analysis of Python functions, offering insights into resource utilization and potential bottlenecks. Its robust handling of process isolation and monitoring ensures accurate measurements, even for complex functions that involve multiple processes.
 
 ## Installation
 
@@ -214,6 +214,42 @@ If an error occurs while executing the decorated function in non-isolated mode o
 
 - Deprecation warnings will be added before removing a feature.
 
+## How metrit works
+
+Here's a detailed explanation of how `metrit` operates:
+
+1. Function Process Execution:
+
+    - The decorated function is initially executed in a separate process if `isolate = True`, referred to as the function process.
+    - If this process crashes, the function will be executed in the main process.
+
+2. Monitor Process Setup:
+
+    - Before starting the function process, another process, called the monitor process, is created to monitor the resources of the function process.
+    - The monitor process takes an initial snapshot of the resources in the function process to use as a baseline.
+
+3. Resource Monitoring:
+
+    - The monitor process continuously checks the resources of the function process, starting with an interval of 0.1 seconds, which can scale up to 5 seconds for longer-running functions.
+    - This adaptive refresh rate minimizes the monitoring overhead for time-consuming functions.
+
+4. Completion and Data Collection:
+
+    - Upon the function process completion, a signal is sent to the monitor process to stop monitoring.
+    - The data collected by the monitor process is adjusted by subtracting the initial snapshot to ensure precision.
+    - The final values are then printed.
+
+5. Handling Failures and Parameters:
+
+    - If the function process fails, or if the `isolate` parameter is set to `False`, the function will execute in the main process while the monitor process continues its monitoring.
+    - The recursion checker for isolated functions uses queues to communicate the function call stack.
+    - Methods are not isolated and will always run in the main process, as if `isolate` is `False`.
+
+6. Child Process Monitoring:
+
+    - When `find_children` is set to `True`, the library measures all processes spawned by the function process.
+    - This feature is useful for nested functions or functions that spawn additional processes.
+
 ## Contributing
 
 Contributions are welcome! Please follow these guidelines when contributing:

examples/examples.py

@@ -60,7 +60,7 @@ def cpu_intensive(a: int = 1, b: int = 2) -> int:
     return a + b
 
 
-@metrit(isolate=True)
+@metrit
 def recursive_func(n):
     if n < 2:
         return n
@@ -92,7 +92,7 @@ def simulate_writes_and_reads(num_writes=5_000, data_size=1024):
     return num_writes + data_size
 
 
-# @metrit(verbose=True, find_children=True, isolate=True)
+@metrit(verbose=True, find_children=True, isolate=True)
 def main():
 
     print("---CLASS EXAMPLES---")

metrit/Monitoring.py

@@ -157,6 +157,9 @@ def pool_stats(self):
             stats = self.stat_collector.collect_cpu_ram_stats(
                 stats, self.find_children, refresh_rate, self.success_queue
             )
+
+            # This call will help if find children = True and some of them die before the STOP signal is received so IO counters for dead processes will not be collected.
+            stats = self.stat_collector.collect_io_counters(stats, self.find_children, self.success_queue)
             if not self.stop_queue.empty():
                 message = self.stop_queue.get()
                 if message == "STOP":

metrit/StatCollector.py

@@ -63,8 +63,6 @@ def collect_cpu_ram_stats(
                         stats[child.pid].rss_bytes.append(rss_bytes)
                         stats[child.pid].vms_bytes.append(vms_bytes)
                     except Exception:
-                        if child.pid in stats:
-                            del stats[child.pid]
                         continue  # It is possible for some children processes to not exist, so we skip them
 
             sucesss_queue.put(True)
@@ -150,8 +148,14 @@ def collect_io_counters(self, stats, find_children: bool, sucesss_queue: Queue)
                         stats[child.pid].io_read_bytes = io_read_bytes
                         stats[child.pid].io_write_bytes = io_write_bytes
                     except Exception:
-                        if child.pid in stats:
-                            del stats[child.pid]
+                        if child.pid in stats and (
+                            stats[child.pid].io_read_count != 0
+                            and stats[child.pid].io_write_count != 0  # noqa: W503
+                            and stats[child.pid].io_read_bytes != 0  # noqa: W503
+                            and stats[child.pid].io_write_bytes != 0  # noqa: W503
+                        ):
+                            continue  # This continue is to make the code more clear, but the whole condition could be removed
+
                         continue  # It is possible for some children processes to not exist, so we skip them
 
             sucesss_queue.put(True)