1.4.2 cherry picks (#39842)

* TC-CC-2.1: Fix checks for nullable attributes (#39566)

* TC-CC-2.1: Fix nullables

Fixes the null check for StartUpColorTemperatureMireds, add null
checks for other nullable values.

* Update src/python_testing/TC_CC_2_1.py

Co-authored-by: Tennessee Carmel-Veilleux <[email protected]>

* number of primaries can also be null

* minor fix to log

---------

Co-authored-by: Tennessee Carmel-Veilleux <[email protected]>

* [Fix] Fix CADMIN_1_19 test step 3 fabric filtered (#39609)

- Changed to make test step 3 unfiltered now as mentioned in issue #(636)[project-chip/certification-tool#636]

* TC-SC-4.3: Fix test for no TXT record returned from DUT (#39623)

* TC-SC-4.3: Fix test for no TXT record

The spec is a BIT fuzzy in this area, but in my re-reading, a TXT
record is not specifically mandated, only certain keys in certain
situations (which would then require a TXT record).

If there are no mandatory TXT keys (which can happen for operational)
then the TXT record can be omitted and that's fine.

* Apply suggestions from code review

* linter

aka here's a deeply unimportant thing I'm going to waste your time on

* TC-IDM-4.2: indicate the DUT needs to be commissioned (#39638)

Side note - yes, I know how terrible this is. It's now written
into the TH and will take a while to fix. We're working on it.

* Revert matter_testing back to tag

* Move app pipe and mark_all_remaining_steps_skipped back to master

* Revert "Adding helper function for commands running errors (#38799)"

This reverts commit 1bf9a3e.

* some mypy conformance was added after sve

* move opstate common back to the app pipe style on master

* Put back changes for TestClusters on master

* Take TBRM tests from master

see https://github.com/project-chip/connectedhomeip/pull/39555/files

* Rename from skip_all_remaining_steps_skipped to mark_all_remaining_steps_skipped (#38729)

* Updated name from skip_all_remaining_steps_skipped to mark_all_remaining_steps_skipped. Updated test cases using the old name to new one.

* fix: missing method name replace.

* fix: updated test cases

* Update step start to skip at  src/python_testing/TC_CNET_4_3.py

Co-authored-by: Andrey Khodyrev <[email protected]>

* fix: updated docstring and method args type as the step method.

* Added missing returns after marked as skip

---------

Co-authored-by: Andrey Khodyrev <[email protected]>

* Move all camera tests back up to master

These weren't run through SVE anyway, won't be selected by the
1.4.2 PICS and this spec is enough is flux that there were
changes to the cluster definitions between SVE and this branch
that are affecting the test.

* Proposal to remove --app-pid and --app-pipe-prefix for out of band commands. (#38778)

* Updated linux apps to read the FIFO without using the app_pid. Now uses a value from a env variable or static value. This will be usefull on tests to use only -app_pipe <app_prefix>+ instead of two paramters.

* Update write_to_app_pipe to only use app_pipe argument which is the full path to pipe file <prefix>_<id>, raises FileNotFound error if the pipe file is not found from start. Removed arguments app_pid and app_pipe_prefix as they are not needed anymore.

* Removed arguments --app-pid and --app-pipe-prefix

* Updated app-pipe_prefix to app-pipe with APP_ID in test cases

* Updated OPSTATE testcases and OVENOPSTATE test cases to use --app-pippe with id instead of PID of the app.

* Fix not found when the parameter is not available

* Removed unused ci actions

* Added APP_IDs for workflows. Currently those are static values for CI.

* Remove automatically appended --app-pid on execution. Now it dependes on the CI parameters provided by the user.

* Fix typo and restyle

* Restyled by whitespace

* Restyled by clang-format

* fix: Solved typo for TC_RVCRUM_2_1. Added quotes for APP_ID.

* fix: Fixed id for bridge app. Added --app-pipe for BRBINFO. Added missing app-pipe for TC_SWITCH

* Typo in variable name

* Typo RCV_CHIP

* Missing file exists check  when the app_pipe is send from the test case.

* fix: code review fix, optimize for string.

* Removed vars from yaml file

* Removed APP_ID from Test case CI arguments from --app-pipe

* Removed env variable for custom app-id. Replaced for app argument --app-id

* Restyled by whitespace

* Restore change.

* fix: Fixed from comments, app_id set to empty string instead of nullptr, this will avoid uneeded check on each of the apps.

* fix: removed comma

* Added app-pipe from scripts args to app-args if present. This is to avoid having the fifo path in scripts args and app-args and avoid posible run errors.

* Added custom fifo path for each one of the test cases.

* Removed prefix. Updated app-id to app-pipe. Now app pipe gets the full path of the fifo path. If not defined avoid sChipNamedPipeCommands.Start()

* Restyled by clang-format

* Updated empty string compare

* Added --app-pipe into app-args. Removed auto append --app-pipe this will need repeated value but will not force same fifo and allow more scenarios.

* Updated tc_soil_2_2 with new fifo format.

---------

Co-authored-by: Restyled.io <[email protected]>

* add chime to do not run list

Good grief why are you fighing me so hard, CI? Chime tests
aren't even part of 1.4.2. If this continues, I'm going to
delete the lot of you.

---------

Co-authored-by: Tennessee Carmel-Veilleux <[email protected]>
Co-authored-by: Jake Ororke <[email protected]>
Co-authored-by: jtrejoespinoza-grid <[email protected]>
Co-authored-by: Andrey Khodyrev <[email protected]>
Co-authored-by: Restyled.io <[email protected]>

Author: 6 people

.github/workflows/lint.yml

@@ -325,12 +325,6 @@ jobs:
               if: always()
               run: |
                   ruff check
-                
-            # Run ChipStackError linter
-            - name: Check for errors using ruff Python linter
-              if: always()
-              run: |
-                  python3 scripts/check_chipstack.py
 
             # git grep exits with 0 if it finds a match, but we want
             # to fail (exit nonzero) on match.  And we want to exclude this file,

.github/workflows/mypy-validation.yml

@@ -58,9 +58,6 @@ jobs:
             src/python_testing/matter_testing_infrastructure/chip/testing/tasks.py \
             src/python_testing/matter_testing_infrastructure/chip/testing/taglist_and_topology_test.py \
             src/python_testing/matter_testing_infrastructure/chip/testing/pics.py \
-            src/python_testing/matter_testing_infrastructure/chip/testing/choice_conformance.py \
-            src/python_testing/matter_testing_infrastructure/chip/testing/commissioning.py \
-            src/python_testing/matter_testing_infrastructure/chip/testing/decorators.py \
             src/python_testing/matter_testing_infrastructure/chip/testing/basic_composition.py"
           
           # Print a reminder about expanding coverage

scripts/check_chipstack.py

@@ -3913,6 +3913,7 @@ tests:
                   23,
                   24,
                   25,
+                  26,
                   4294049962,
               ]
 

src/app/tests/suites/TestCluster.yaml

@@ -41,6 +41,7 @@ tests:
           values:
               - name: nodeId
                 value: nodeId
+
     # Step 1
     - label: "TH reads the ActiveDatasetTimestamp attribute from the DUT"
       command: readAttribute
@@ -49,25 +50,55 @@ tests:
           value: null
 
     # Step 2
+    - label: "TH send a GetActiveDatasetRequest command to the DUT"
+      command: GetActiveDatasetRequest
+      response:
+          values:
+              - name: Dataset
+                value: ""
+
+    # Step 3
     - label: "TH reads the PendingDatasetTimestamp attribute from the DUT"
       command: readAttribute
       attribute: PendingDatasetTimestamp
       response:
           value: null
 
-    # Step 3
+    # Step 4
+    - label: "TH send a GetPendingDatasetRequest command to the DUT"
+      command: GetPendingDatasetRequest
+      response:
+          values:
+              - name: Dataset
+                value: ""
+
+    # Step 5
+    # Note: Unsupported with darwin-framework-tool because it automatically performs a timed invoke
+    - label:
+          "TH sends a valid ActiveDatasetRequest command to the DUT without a
+          timed interaction"
+      command: SetActiveDatasetRequest
+      arguments:
+          values:
+              - name: ActiveDataset
+                value: PIXIT.TBRM.THREAD_ACTIVE_DATASET
+      response:
+          error: NEEDS_TIMED_INTERACTION
+
+    # Step 6
     - label:
           "TH sends a valid ActiveDatasetRequest command to the DUT without
           having armed the fail-safe"
       command: SetActiveDatasetRequest
+      timedInteractionTimeoutMs: 2000
       arguments:
           values:
               - name: ActiveDataset
                 value: PIXIT.TBRM.THREAD_ACTIVE_DATASET
       response:
           error: FAILSAFE_REQUIRED
 
-    # Step 4
+    # Step 7
     - label: "TH sends ArmFailSafe command to the DUT"
       cluster: General Commissioning
       command: ArmFailSafe
@@ -83,38 +114,40 @@ tests:
               - name: ErrorCode
                 value: CommissioningErrorEnum.OK
 
-    # Step 5
+    # Step 8
     - label: "TH sends an invalid ActiveDatasetRequest command to the DUT"
       command: SetActiveDatasetRequest
+      timedInteractionTimeoutMs: 2000
       arguments:
           values:
               - name: ActiveDataset
                 value: PIXIT.TBRM.THREAD_INVALID_DATASET
       response:
           error: INVALID_COMMAND
 
-    # Step 6
+    # Step 9
     - label: "TH sends a valid ActiveDatasetRequest command to the DUT"
       command: SetActiveDatasetRequest
+      timedInteractionTimeoutMs: 2000
       arguments:
           values:
               - name: ActiveDataset
                 value: PIXIT.TBRM.THREAD_ACTIVE_DATASET
 
-    # Step 7
+    # Step 10
     - label: "TH sends CommissioningComplete command to the DUT"
       cluster: General Commissioning
       command: CommissioningComplete
       endpoint: 0
 
-    # Step 8
+    # Step 11
     - label: "TH reads the InterfaceEnabled attribute from the DUT"
       command: readAttribute
       attribute: InterfaceEnabled
       response:
           value: true
 
-    # Step 9
+    # Step 12
     - label: "TH reads the ActiveDatasetTimestamp attribute from the DUT"
       command: readAttribute
       attribute: ActiveDatasetTimestamp
@@ -123,7 +156,7 @@ tests:
           constraints:
               type: int64u
 
-    # Step 10
+    # Step 13
     - label: "TH send a GetActiveDatasetRequest command to the DUT"
       command: GetActiveDatasetRequest
       response:

src/app/tests/suites/certification/Test_TC_TBRM_2_2.yaml

@@ -40,6 +40,7 @@ tests:
           values:
               - name: nodeId
                 value: nodeId
+
     # Step 1
     - label: "TH reads the ActiveDatasetTimestamp attribute from the DUT"
       command: readAttribute
@@ -84,6 +85,7 @@ tests:
     - label: "TH sends a valid ActiveDatasetRequest command to the DUT"
       runIf: noActiveDataset
       command: SetActiveDatasetRequest
+      timedInteractionTimeoutMs: 2000
       arguments:
           values:
               - name: ActiveDataset
@@ -106,14 +108,28 @@ tests:
               type: int64u
 
     # Step 6
-    - label: "TH sends a SetPendingDatasetRequest command to the DUT"
+    # Note: Unsupported with darwin-framework-tool because it automatically performs a timed invoke
+    - label:
+          "TH sends a SetPendingDatasetRequest command to the DUT without a
+          timed interaction"
       command: SetPendingDatasetRequest
       arguments:
           values:
               - name: PendingDataset
                 value: PIXIT.TBRM.THREAD_PENDING_DATASET
+      response:
+          error: NEEDS_TIMED_INTERACTION
 
     # Step 7
+    - label: "TH sends a SetPendingDatasetRequest command to the DUT"
+      command: SetPendingDatasetRequest
+      timedInteractionTimeoutMs: 2000
+      arguments:
+          values:
+              - name: PendingDataset
+                value: PIXIT.TBRM.THREAD_PENDING_DATASET
+
+    # Step 8
     - label: "TH sends a GetPendingDatasetRequest command to the DUT"
       command: GetPendingDatasetRequest
       response:
@@ -123,7 +139,7 @@ tests:
                     type: octet_string
                     # TODO: This should be PIXIT.TBRM.THREAD_PENDING_DATASET but ignoring the Delay Timer element if present
 
-    # Step 8
+    # Step 9
     - label: "TH reads the PendingDatasetTimestamp attribute from the DUT"
       command: readAttribute
       attribute: PendingDatasetTimestamp
@@ -132,7 +148,7 @@ tests:
               type: int64u
               notValue: initialPendingTimestamp
 
-    # Step 9
+    # Step 10
     - label:
           "TH subscribes to the ActiveDatasetTimestamp attribute from the DUT"
       command: subscribeAttribute
@@ -152,14 +168,14 @@ tests:
           constraints:
               hasValue: true
 
-    # Step 10
+    # Step 11
     - label: "TH reads the PendingDatasetTimestamp attribute from the DUT"
       command: readAttribute
       attribute: PendingDatasetTimestamp
       response:
           value: null
 
-    # Step 11
+    # Step 12
     - label: "TH sends a GetActiveDatasetRequest command to the DUT"
       command: GetActiveDatasetRequest
       response:
@@ -169,7 +185,7 @@ tests:
                 constraints:
                     type: octet_string
 
-    # Step 12
+    # Step 13
     - label: "TH reads the InterfaceEnabled attribute from the DUT"
       command: readAttribute
       attribute: InterfaceEnabled

src/app/tests/suites/certification/Test_TC_TBRM_2_3.yaml

@@ -13,8 +13,8 @@
 # limitations under the License.
 
 name:
-    "[TC-TBRM-2.4] Verify that getting Active or Pending Dataset in the PASE
-    session results in unsupported access"
+    "[TC-TBRM-2.4] Verify that getting or setting Active or Pending Dataset in a
+    PASE session results in unsupported access"
 
 PICS:
     - TBRM.S
@@ -31,6 +31,7 @@ config:
         defaultValue: "hex:d0e8a02db8629e9d172dfd40719c89204ff395651a6a2612839a71469880ec2404687d05cf0642b91242c712b5405b6905070c2a4bd80bdc8437ae5a2aded0cf3de91318d16f0ce9450d1c802cc01f39b8761de87cc7eeeb7f52b51308353da49a"
 
 tests:
+    # Step 1
     - label: "Wait for the commissioned device to be retrieved"
       cluster: DelayCommands
       command: WaitForCommissionee
@@ -39,6 +40,7 @@ tests:
               - name: nodeId
                 value: nodeId
 
+    # Step 2
     - label: "Open Commissioning Window"
       endpoint: 0
       cluster: Administrator Commissioning
@@ -57,6 +59,7 @@ tests:
               - name: Salt
                 value: "SPAKE2P Key Salt"
 
+    # Step 3
     - label: "TH2  establishes a PASE session with the DUT"
       identity: beta
       cluster: CommissionerCommands
@@ -69,20 +72,51 @@ tests:
               - name: payload
                 value: EcmPayload
 
+    # Step 4
     - label:
           "TH2 send GetActiveDatasetRequest command to the DUT in PASE session"
       identity: beta
       command: GetActiveDatasetRequest
       response:
           error: UNSUPPORTED_ACCESS
 
+    # Step 5
+    - label:
+          "TH2 send SetActiveDatasetRequest command to the DUT in PASE session"
+      identity: beta
+      command: SetActiveDatasetRequest
+      timedInteractionTimeoutMs: 2000
+      arguments:
+          values:
+              - name: ActiveDataset
+                value: "hex:0e080000000000010000"
+      response:
+          error: UNSUPPORTED_ACCESS
+
+    # Step 6
     - label:
           "TH2 send GetPendingDatasetRequest command to the DUT in PASE session"
       identity: beta
       command: GetPendingDatasetRequest
       response:
           error: UNSUPPORTED_ACCESS
 
+    # Step 7
+    - label:
+          "TH sends a SetPendingDatasetRequest command to the DUT in PASE
+          session"
+      PICS: TBRM.S.F00
+      identity: beta
+      command: SetPendingDatasetRequest
+      timedInteractionTimeoutMs: 2000
+      arguments:
+          values:
+              - name: PendingDataset
+                value: "hex:0e080000000000010000"
+      response:
+          error: UNSUPPORTED_ACCESS
+
+    # Step 8
     - label: "Cleanup: Close the commissioning window"
       endpoint: 0
       cluster: Administrator Commissioning