ansible_test_splitter - extend feature for roles (#109)

* add splitter for role

* changed files

* add more log

* Display command

* add optional argument

* Remove base_ref input

* fix role matching

* add documentation for ansible_test_splitter github action

* update after code review

Author: abikouo

.github/actions/ansible_test_splitter/REAME.md

@@ -0,0 +1,90 @@
+# ansible_test_splitter
+
+This action identifies the targets impacted by the changes on a pull request and split them into a number of jobs defined by the user.
+
+## Usage
+
+<!-- start usage -->
+
+```yaml
+- uses: ansible-network/github_actions/.github/actions/ansible_test_splitter@main
+  with:
+    # Path to a list of collections
+    collections_to_test: |
+      path_to_collection_1
+      path_to_collection_2
+      (...)
+      path_to_collection_n
+
+    # The total number of jobs to share
+    total_jobs: 5
+```
+
+The action output is a variable `test_targets` containing a list of chunk for each collection with the targets for each chunk.
+e.g: `community.aws-1:dynamodb_table;community.aws-2:elb_target;community.aws-3:msk_cluster-auth;community.aws-4:secretsmanager_secret;community.aws-5:redshift,ec2_transit_gateway_vpc_attachment`
+
+<!-- end usage -->
+
+## Relationship between plugins/roles and targets
+
+This action reads elements to test from `plugins` and `roles` directories and corresponding tests from `tests/integration/targets` directory. Here after more details on the relationship between plugins/roles and integration tests targets:
+
+- `modules`, the test target should have the same name as the module or defines the module name into the `aliases` file
+
+_Example_:
+
+```
+    |___plugins/modules/my_module.py
+    |___tests
+        |___integration
+            |___targets
+                |___my_module
+                |___another_test
+                    |___aliases (contains this line my_module)
+```
+
+For any change on `plugins/modules/my_module.py`, this action will produce `my_module` and `another_test` as impacted targets.
+
+- `roles`, the test target should defines the role name with the prefix `role` into the `aliases` file.
+
+_Example_:
+
+```
+    |___roles/some_role
+    |___tests
+        |___integration
+            |___targets
+                |___test_of_some_role
+                    |___aliases (contains this line role/some_role)
+```
+
+For any change on `roles/some_role`, this action will produce `test_of_some_role` as impacted target.
+
+- For any other plugin (inventory, connection, module_utils, plugin_utils, lookup), the test target should have the same name as the plugin or defines the plugin name prefixed by the plugin type and underscore (e.g: **inventory_myinventory**) into the `aliases` file.
+
+_Example_:
+
+```
+    |___plugins/lookup/random.py
+    |___tests
+        |___integration
+            |___targets
+                |___lookup_random
+                |___test_random
+                    |___aliases (contains this line lookup_random)
+```
+
+For any change on `plugins/lookup/random.py`, this action will produce `lookup_random` and `test_random` as impacted targets.
+
+## Debugging
+
+- Set the label `test-all-the-targets` on the pull request to run the full test suite instead of the impacted changes.
+- Use `TargetsToTest=collection1:target01,target02;collection2:target03,target4` in the pull request description to run a specific list of targets.
+  _Example_: You need to test the following targets for a pull request
+
+```yaml
+- collection1: some_test_1 some_test_2
+- collection2: another_test
+```
+
+The pull request should contain the following line `TargetsToTest=collection1:some_test_1,some_test_2;collection2:another_test`.

.github/actions/ansible_test_splitter/list_changed_common.py

@@ -173,9 +173,9 @@ def changed_files(self) -> list[PosixPath]:
         :returns: a list of pathlib.PosixPath
         """
         if not self.files:
-            stdout = run_command(
-                command=f"git diff origin/{self.base_ref} --name-only", chdir=self.collection_path
-            )
+            changed_files_cmd = f"git diff origin/{self.base_ref} --name-only"
+            print(f"Command for changed files => {changed_files_cmd}")
+            stdout = run_command(command=changed_files_cmd, chdir=self.collection_path)
             self.files = [PosixPath(p) for p in stdout.split("\n") if p]
         return self.files
 
@@ -229,6 +229,15 @@ def modules(self) -> Generator[PosixPath, None, None]:
         """
         yield from self._path_matches("plugins/modules/")
 
+    def roles(self) -> Generator[str, None, None]:
+        """List the roles impacted by the change.
+
+        :yields: path to a role change
+        """
+        for changed_file in self.changed_files():
+            if str(changed_file).startswith("roles/"):
+                yield str(changed_file).split("/", maxsplit=2)[1]
+
     def _util_matches(
         self, base_path: str, import_path: str
     ) -> Generator[tuple[PosixPath, str], None, None]:

.github/actions/ansible_test_splitter/list_changed_targets.py

@@ -77,6 +77,9 @@ def _add_changed_target(
             elif plugin_type == "modules":
                 file_name = PosixPath(ref_path).stem
                 plugin_file_name = file_name
+            elif plugin_type == "roles":
+                file_name = str(ref_path)
+                plugin_file_name = f"role/{ref_path}"
             else:
                 file_name = PosixPath(ref_path).stem
                 plugin_file_name = f"{plugin_type}_{PosixPath(ref_path).stem}"
@@ -85,6 +88,7 @@ def _add_changed_target(
                 collection.add_target_to_plan(plugin_file_name)
 
         for whc in [WhatHaveChanged(path, self.base_ref) for path in self.collections_to_test]:
+            print(f"changed file for collection [{whc.collection_name}] => {whc.changed_files()}")
             listed_changes[whc.collection_name] = {
                 "modules": [],
                 "inventory": [],
@@ -93,6 +97,7 @@ def _add_changed_target(
                 "plugin_utils": [],
                 "lookup": [],
                 "targets": [],
+                "roles": [],
             }
             for path in whc.modules():
                 _add_changed_target(whc.collection_name, path, "modules")
@@ -112,6 +117,8 @@ def _add_changed_target(
                 _add_changed_target(whc.collection_name, path, "lookup")
             for target in whc.targets():
                 _add_changed_target(whc.collection_name, target, "targets")
+            for role in whc.roles():
+                _add_changed_target(whc.collection_name, role, "roles")
 
         print("----------- Listed Changes -----------\n", json.dumps(listed_changes, indent=2))
         return {x: make_unique(y["targets"]) for x, y in listed_changes.items()}