Merge branch 'development' into alex_devenum_update

Author: alexandraBara

docs/PLUGIN_DOC.md

@@ -17,8 +17,8 @@
 | MemoryPlugin | free -b<br>/usr/bin/lsmem<br>wmic OS get FreePhysicalMemory /Value; wmic ComputerSystem get TotalPhysicalMemory /Value | - | [MemoryDataModel](#MemoryDataModel-Model) | [MemoryCollector](#Collector-Class-MemoryCollector) | [MemoryAnalyzer](#Data-Analyzer-Class-MemoryAnalyzer) |
 | NvmePlugin | nvme smart-log {dev}<br>nvme error-log {dev} --log-entries=256<br>nvme id-ctrl {dev}<br>nvme id-ns {dev}{ns}<br>nvme fw-log {dev}<br>nvme self-test-log {dev}<br>nvme get-log {dev} --log-id=6 --log-len=512<br>nvme telemetry-log {dev} --output-file={dev}_{f_name} | - | [NvmeDataModel](#NvmeDataModel-Model) | [NvmeCollector](#Collector-Class-NvmeCollector) | - |
 | OsPlugin | sh -c '( lsb_release -ds \|\| (cat /etc/*release \| grep PRETTY_NAME) \|\| uname -om ) 2>/dev/null \| head -n1'<br>cat /etc/*release \| grep VERSION_ID<br>wmic os get Version /value<br>wmic os get Caption /Value | **Analyzer Args:**<br>- `exp_os`: Union[str, list]<br>- `exact_match`: bool | [OsDataModel](#OsDataModel-Model) | [OsCollector](#Collector-Class-OsCollector) | [OsAnalyzer](#Data-Analyzer-Class-OsAnalyzer) |
-| PackagePlugin | dnf list --installed<br>dpkg-query -W<br>pacman -Q<br>cat /etc/*release<br>wmic product get name,version | **Analyzer Args:**<br>- `exp_package_ver`: Dict[str, Optional[str]]<br>- `regex_match`: bool | [PackageDataModel](#PackageDataModel-Model) | [PackageCollector](#Collector-Class-PackageCollector) | [PackageAnalyzer](#Data-Analyzer-Class-PackageAnalyzer) |
-| PciePlugin | lspci -d {vendor_id}: -nn<br>lspci -x<br>lspci -xxxx<br>lspci -PP<br>lspci -PP -d {vendor_id}:{dev_id}<br>lspci -vt<br>lspci -vvv | **Analyzer Args:**<br>- `exp_speed`: int<br>- `exp_width`: int<br>- `exp_sriov_count`: int<br>- `exp_gpu_count_override`: Optional[int]<br>- `exp_max_payload_size`: Union[Dict[int, int], int, NoneType]<br>- `exp_max_rd_req_size`: Union[Dict[int, int], int, NoneType]<br>- `exp_ten_bit_tag_req_en`: Union[Dict[int, int], int, NoneType] | [PcieDataModel](#PcieDataModel-Model) | [PcieCollector](#Collector-Class-PcieCollector) | [PcieAnalyzer](#Data-Analyzer-Class-PcieAnalyzer) |
+| PackagePlugin | dnf list --installed<br>dpkg-query -W<br>pacman -Q<br>cat /etc/*release<br>wmic product get name,version | **Analyzer Args:**<br>- `exp_package_ver`: Dict[str, Optional[str]]<br>- `regex_match`: bool<br>- `rocm_regex`: Optional[str]<br>- `enable_rocm_regex`: bool | [PackageDataModel](#PackageDataModel-Model) | [PackageCollector](#Collector-Class-PackageCollector) | [PackageAnalyzer](#Data-Analyzer-Class-PackageAnalyzer) |
+| PciePlugin | lspci -d {vendor_id}: -nn<br>lspci -x<br>lspci -xxxx<br>lspci -PP<br>lspci -PP -d {vendor_id}:{dev_id}<br>lspci -vvv<br>lspci -vvvt | **Analyzer Args:**<br>- `exp_speed`: int<br>- `exp_width`: int<br>- `exp_sriov_count`: int<br>- `exp_gpu_count_override`: Optional[int]<br>- `exp_max_payload_size`: Union[Dict[int, int], int, NoneType]<br>- `exp_max_rd_req_size`: Union[Dict[int, int], int, NoneType]<br>- `exp_ten_bit_tag_req_en`: Union[Dict[int, int], int, NoneType] | [PcieDataModel](#PcieDataModel-Model) | [PcieCollector](#Collector-Class-PcieCollector) | [PcieAnalyzer](#Data-Analyzer-Class-PcieAnalyzer) |
 | ProcessPlugin | top -b -n 1<br>rocm-smi --showpids<br>top -b -n 1 -o %CPU  | **Analyzer Args:**<br>- `max_kfd_processes`: int<br>- `max_cpu_usage`: float | [ProcessDataModel](#ProcessDataModel-Model) | [ProcessCollector](#Collector-Class-ProcessCollector) | [ProcessAnalyzer](#Data-Analyzer-Class-ProcessAnalyzer) |
 | RocmPlugin | {rocm_path}/opencl/bin/*/clinfo<br>env \| grep -Ei 'rocm\|hsa\|hip\|mpi\|openmp\|ucx\|miopen'<br>ls /sys/class/kfd/kfd/proc/<br>grep -i -E 'rocm' /etc/ld.so.conf.d/*<br>{rocm_path}/bin/rocminfo<br>ls -v -d /opt/rocm*<br>ls -v -d /opt/rocm-[3-7]* \| tail -1<br>ldconfig -p \| grep -i -E 'rocm'<br>/opt/rocm/.info/version-rocm<br>/opt/rocm/.info/version | **Analyzer Args:**<br>- `exp_rocm`: Union[str, list]<br>- `exp_rocm_latest`: str | [RocmDataModel](#RocmDataModel-Model) | [RocmCollector](#Collector-Class-RocmCollector) | [RocmAnalyzer](#Data-Analyzer-Class-RocmAnalyzer) |
 | StoragePlugin | sh -c 'df -lH -B1 \| grep -v 'boot''<br>wmic LogicalDisk Where DriveType="3" Get DeviceId,Size,FreeSpace | - | [StorageDataModel](#StorageDataModel-Model) | [StorageCollector](#Collector-Class-StorageCollector) | [StorageAnalyzer](#Data-Analyzer-Class-StorageAnalyzer) |
@@ -428,7 +428,7 @@ class for collection of PCIe data only supports Linux OS type.
 
     This class will collect important PCIe data from the system running the commands
     - `lspci -vvv` : Verbose collection of PCIe data
-    - `lspci -vt`: Tree view of PCIe data
+    - `lspci -vvvt`: Verbose tree view of PCIe data
     - `lspci -PP`: Path view of PCIe data for the GPUs
     - If system interaction level is set to STANDARD or higher, the following commands will be run with sudo:
         - `lspci -xxxx`: Hex view of PCIe data for the GPUs
@@ -448,7 +448,7 @@ class for collection of PCIe data only supports Linux OS type.
 
 - **SUPPORTED_OS_FAMILY**: `{<OSFamily.LINUX: 3>}`
 - **CMD_LSPCI_VERBOSE**: `lspci -vvv`
-- **CMD_LSPCI_TREE**: `lspci -vt`
+- **CMD_LSPCI_VERBOSE_TREE**: `lspci -vvvt`
 - **CMD_LSPCI_PATH**: `lspci -PP`
 - **CMD_LSPCI_HEX_SUDO**: `lspci -xxxx`
 - **CMD_LSPCI_HEX**: `lspci -x`
@@ -466,8 +466,8 @@ PcieDataModel
 - lspci -xxxx
 - lspci -PP
 - lspci -PP -d {vendor_id}:{dev_id}
-- lspci -vt
 - lspci -vvv
+- lspci -vvvt
 
 ## Collector Class ProcessCollector
 
@@ -810,6 +810,8 @@ Pacakge data contains the package data for the system
 ### Model annotations and fields
 
 - **version_info**: `dict[str, str]`
+- **rocm_regex**: `str`
+- **enable_rocm_regex**: `bool`
 
 ## PcieDataModel Model
 
@@ -1322,6 +1324,8 @@ Check sysctl matches expected sysctl details
 
 - **exp_package_ver**: `Dict[str, Optional[str]]`
 - **regex_match**: `bool`
+- **rocm_regex**: `Optional[str]`
+- **enable_rocm_regex**: `bool`
 
 ## Analyzer Args Class PcieAnalyzerArgs
 

nodescraper/plugins/inband/package/analyzer_args.py

@@ -34,7 +34,15 @@
 class PackageAnalyzerArgs(AnalyzerArgs):
     exp_package_ver: Dict[str, Optional[str]] = Field(default_factory=dict)
     regex_match: bool = False
+    # rocm_regex is optional and should be specified in plugin_config.json if needed
+    rocm_regex: Optional[str] = None
+    enable_rocm_regex: bool = False
 
     @classmethod
     def build_from_model(cls, datamodel: PackageDataModel) -> "PackageAnalyzerArgs":
-        return cls(exp_package_ver=datamodel.version_info)
+        # Use custom rocm_regex from collection_args if enable_rocm_regex is true
+        rocm_regex = None
+        if datamodel.enable_rocm_regex and datamodel.rocm_regex:
+            rocm_regex = datamodel.rocm_regex
+
+        return cls(exp_package_ver=datamodel.version_info, rocm_regex=rocm_regex)

nodescraper/plugins/inband/package/package_collector.py

@@ -34,10 +34,11 @@
 from nodescraper.models import TaskResult
 from nodescraper.utils import get_exception_details
 
+from .analyzer_args import PackageAnalyzerArgs
 from .packagedata import PackageDataModel
 
 
-class PackageCollector(InBandDataCollector[PackageDataModel, None]):
+class PackageCollector(InBandDataCollector[PackageDataModel, PackageAnalyzerArgs]):
     """Collecting Package information from the system"""
 
     DATA_MODEL = PackageDataModel
@@ -181,9 +182,34 @@ def _handle_command_failure(self, command_artifact: CommandArtifact):
         self.result.message = "Failed to run Package Manager command"
         self.result.status = ExecutionStatus.EXECUTION_FAILURE
 
-    def collect_data(self, args=None) -> tuple[TaskResult, Optional[PackageDataModel]]:
+    def _filter_rocm_packages(self, packages: dict[str, str], rocm_pattern: str) -> dict[str, str]:
+        """Filter ROCm-related packages from a package dictionary.
+
+        This method searches package names for ROCm-related patterns and returns
+        only the matching packages.
+
+        Args:
+            packages (dict[str, str]): Dictionary with package names as keys and versions as values.
+            rocm_pattern (str): Regex pattern to match ROCm-related package names.
+
+        Returns:
+            dict[str, str]: Filtered dictionary containing only ROCm-related packages.
+        """
+        rocm_packages = {}
+        pattern = re.compile(rocm_pattern, re.IGNORECASE)
+        for package_name, version in packages.items():
+            if pattern.search(package_name):
+                rocm_packages[package_name] = version
+        return rocm_packages
+
+    def collect_data(
+        self, args: Optional[PackageAnalyzerArgs] = None
+    ) -> tuple[TaskResult, Optional[PackageDataModel]]:
         """Collect package information from the system.
 
+        Args:
+            args (Optional[PackageAnalyzerArgs]): Optional arguments containing ROCm regex pattern.
+
         Returns:
             tuple[TaskResult, Optional[PackageDataModel]]: tuple containing the task result and a PackageDataModel instance
             with the collected package information, or None if there was an error.
@@ -205,8 +231,36 @@ def collect_data(self, args=None) -> tuple[TaskResult, Optional[PackageDataModel
             self.result.message = "Unsupported OS"
             self.result.status = ExecutionStatus.NOT_RAN
             return self.result, None
+
+        # Filter and log ROCm packages if on Linux and rocm_regex is provided
+        if self.system_info.os_family == OSFamily.LINUX and packages:
+            # Get ROCm pattern from args if provided
+            rocm_pattern = args.rocm_regex if args else None
+            if rocm_pattern:
+                self.logger.info("Using rocm_pattern: %s", rocm_pattern)
+                rocm_packages = self._filter_rocm_packages(packages, rocm_pattern)
+                if rocm_packages:
+                    self.result.message = (
+                        f"Found {len(rocm_packages)} ROCm-related packages installed"
+                    )
+                    self.result.status = ExecutionStatus.OK
+                    self._log_event(
+                        category=EventCategory.OS,
+                        description=f"Found {len(rocm_packages)} ROCm-related packages installed",
+                        priority=EventPriority.INFO,
+                        data={"rocm_packages": sorted(rocm_packages.keys())},
+                    )
+            else:
+                self.logger.info("No rocm_regex provided, skipping ROCm package filtering")
+
+        # Extract rocm_regex and enable_rocm_regex from args if provided
+        rocm_regex = args.rocm_regex if (args and args.rocm_regex) else ""
+        enable_rocm_regex = getattr(args, "enable_rocm_regex", False) if args else False
+
         try:
-            package_model = PackageDataModel(version_info=packages)
+            package_model = PackageDataModel(
+                version_info=packages, rocm_regex=rocm_regex, enable_rocm_regex=enable_rocm_regex
+            )
         except ValidationError as val_err:
             self._log_event(
                 category=EventCategory.RUNTIME,

nodescraper/plugins/inband/package/packagedata.py

@@ -32,6 +32,10 @@ class PackageDataModel(DataModel):
     Attributes:
         version_info (dict[str, str]): The version information for the package
             Key is the package name and value is the version of the package
+        rocm_regex (str): Regular expression pattern for ROCm package filtering
+        enable_rocm_regex (bool): Whether to use custom ROCm regex from collection_args
     """
 
     version_info: dict[str, str]
+    rocm_regex: str = ""
+    enable_rocm_regex: bool = False

nodescraper/plugins/inband/pcie/pcie_collector.py

@@ -64,7 +64,7 @@ class PcieCollector(InBandDataCollector[PcieDataModel, None]):
 
     This class will collect important PCIe data from the system running the commands
     - `lspci -vvv` : Verbose collection of PCIe data
-    - `lspci -vt`: Tree view of PCIe data
+    - `lspci -vvvt`: Verbose tree view of PCIe data
     - `lspci -PP`: Path view of PCIe data for the GPUs
     - If system interaction level is set to STANDARD or higher, the following commands will be run with sudo:
         - `lspci -xxxx`: Hex view of PCIe data for the GPUs
@@ -83,7 +83,7 @@ class PcieCollector(InBandDataCollector[PcieDataModel, None]):
     DATA_MODEL = PcieDataModel
 
     CMD_LSPCI_VERBOSE = "lspci -vvv"
-    CMD_LSPCI_TREE = "lspci -vt"
+    CMD_LSPCI_VERBOSE_TREE = "lspci -vvvt"
     CMD_LSPCI_PATH = "lspci -PP"
     CMD_LSPCI_HEX_SUDO = "lspci -xxxx"
     CMD_LSPCI_HEX = "lspci -x"
@@ -142,8 +142,8 @@ def show_lspci_verbose(self, sudo=True) -> Optional[str]:
         return self._run_os_cmd(self.CMD_LSPCI_VERBOSE, sudo=sudo)
 
     def show_lspci_verbose_tree(self, sudo=True) -> Optional[str]:
-        """Show lspci with -vt."""
-        return self._run_os_cmd(self.CMD_LSPCI_TREE, sudo=sudo)
+        """Show lspci with -vvvt (verbose tree view)."""
+        return self._run_os_cmd(self.CMD_LSPCI_VERBOSE_TREE, sudo=sudo)
 
     def show_lspci_path(self, sudo=True) -> Optional[str]:
         """Show lspci with -PP."""
@@ -548,13 +548,13 @@ def _log_pcie_artifacts(
         self,
         lspci_pp: Optional[str],
         lspci_hex: Optional[str],
-        lspci_tree: Optional[str],
+        lspci_verbose_tree: Optional[str],
         lspci_verbose: Optional[str],
     ):
         """Log the file artifacts for the PCIe data collector."""
         name_log_map = {
             "lspci_hex.txt": lspci_hex,
-            "lspci_tree.txt": lspci_tree,
+            "lspci_verbose_tree.txt": lspci_verbose_tree,
             "lspci_verbose.txt": lspci_verbose,
             "lspci_pp.txt": lspci_pp,
         }
@@ -629,7 +629,7 @@ def _get_pcie_data(
             self._log_pcie_artifacts(
                 lspci_pp=lspci_path,
                 lspci_hex=lspci_hex,
-                lspci_tree=lspci_verbose_tree,
+                lspci_verbose_tree=lspci_verbose_tree,
                 lspci_verbose=lspci_verbose,
             )
             pcie_data = PcieDataModel(

test/functional/fixtures/package_plugin_config.json

@@ -2,6 +2,10 @@
   "global_args": {},
   "plugins": {
     "PackagePlugin": {
+      "collection_args": {
+        "rocm_regex": "rocm|hip|hsa|amdgpu",
+        "enable_rocm_regex": true
+      },
       "analysis_args": {
         "exp_package_ver": {
           "gcc": "11.4.0"

test/unit/plugin/test_package_collector.py

@@ -222,3 +222,112 @@ def test_bad_splits_ubuntu(collector, conn_mock, command_results):
     ]
     res, _ = collector.collect_data()
     assert res.status == ExecutionStatus.OK
+
+
+def test_rocm_package_filtering_custom_regex(collector, conn_mock, command_results):
+    """Test ROCm package filtering with custom regex pattern."""
+    from nodescraper.plugins.inband.package.analyzer_args import PackageAnalyzerArgs
+
+    # Mock Ubuntu system with ROCm packages
+    ubuntu_packages = """rocm-core 5.7.0
+                    hip-runtime-amd 5.7.0
+                    hsa-rocr 1.9.0
+                    amdgpu-dkms 6.3.6
+                    gcc 11.4.0
+                    python3 3.10.12"""
+
+    conn_mock.run_command.side_effect = [
+        CommandArtifact(
+            command="",
+            exit_code=0,
+            stdout=command_results["ubuntu_rel"],
+            stderr="",
+        ),
+        CommandArtifact(
+            command="",
+            exit_code=0,
+            stdout=ubuntu_packages,
+            stderr="",
+        ),
+    ]
+
+    # Use custom regex that only matches 'rocm' and 'hip'
+    args = PackageAnalyzerArgs(rocm_regex="rocm|hip")
+    res, data = collector.collect_data(args)
+    assert res.status == ExecutionStatus.OK
+    # Check that ROCm packages are found
+    assert "found 2 rocm-related packages" in res.message.lower()
+    assert data is not None
+
+
+def test_rocm_package_filtering_no_matches(collector, conn_mock, command_results):
+    """Test ROCm package filtering when no ROCm packages are installed."""
+    from nodescraper.plugins.inband.package.analyzer_args import PackageAnalyzerArgs
+
+    # Mock Ubuntu system without ROCm packages
+    ubuntu_packages = """gcc 11.4.0
+                    python3 3.10.12
+                    vim 8.2.3995"""
+
+    conn_mock.run_command.side_effect = [
+        CommandArtifact(
+            command="",
+            exit_code=0,
+            stdout=command_results["ubuntu_rel"],
+            stderr="",
+        ),
+        CommandArtifact(
+            command="",
+            exit_code=0,
+            stdout=ubuntu_packages,
+            stderr="",
+        ),
+    ]
+
+    args = PackageAnalyzerArgs(rocm_regex="rocm|hip|hsa")
+    res, data = collector.collect_data(args)
+    assert res.status == ExecutionStatus.OK
+    # No ROCm packages found, so message should not mention them
+    assert "rocm" not in res.message.lower() or res.message == ""
+    assert data is not None
+    assert len(data.version_info) == 3
+
+
+def test_filter_rocm_packages_method(collector):
+    """Test _filter_rocm_packages method directly."""
+    packages = {
+        "rocm-core": "5.7.0",
+        "hip-runtime-amd": "5.7.0",
+        "hsa-rocr": "1.9.0",
+        "amdgpu-dkms": "6.3.6",
+        "gcc": "11.4.0",
+        "python3": "3.10.12",
+    }
+
+    # Test with default-like pattern
+    rocm_pattern = "rocm|hip|hsa|amdgpu"
+    filtered = collector._filter_rocm_packages(packages, rocm_pattern)
+
+    assert len(filtered) == 4
+    assert "rocm-core" in filtered
+    assert "hip-runtime-amd" in filtered
+    assert "hsa-rocr" in filtered
+    assert "amdgpu-dkms" in filtered
+    assert "gcc" not in filtered
+    assert "python3" not in filtered
+
+
+def test_filter_rocm_packages_case_insensitive(collector):
+    """Test that ROCm package filtering is case-insensitive."""
+    packages = {
+        "ROCM-Core": "5.7.0",
+        "HIP-Runtime-AMD": "5.7.0",
+        "gcc": "11.4.0",
+    }
+
+    rocm_pattern = "rocm|hip"
+    filtered = collector._filter_rocm_packages(packages, rocm_pattern)
+
+    assert len(filtered) == 2
+    assert "ROCM-Core" in filtered
+    assert "HIP-Runtime-AMD" in filtered