Verification Process Requirements Tooling alignment

Resolves: #36

Signed-off-by: aschemmel-tech <[email protected]>

Author: aschemmel-tech

process/process_areas/verification/guidance/verification_process_toolreqs.rst

@@ -28,19 +28,12 @@ Test Linking to Requirements
     * Verifies
         * PartiallyVerifies
         * FullyVerifies
-    * ASIL
-        * QM
-        * ASIL_B
     * Description
-    * Status
-        * valid
-        * invalid
     * TestType
         * Fault Injection (fault-injection)
         * Interface Test (interface-test)
         * Requirements-based Test (requirements-based)
         * Resource Usage Evaluation (resource-usage)
-
     * DerivationTechnique
         * Analysis of requirements (requirements-analysis)
         * Analysis of design (design-analysis)
@@ -69,8 +62,6 @@ Test Linking to Requirements
          public:
             void SetUp() override
             {
-               RecordProperty("Status", "<Status>");
-               RecordProperty("ASIL", "<ASIL>");
                RecordProperty("TestType", "<TestType>");
                RecordProperty("DerivationTechnique", "<DerivationTechnique>");
                ...
@@ -99,7 +90,6 @@ Test Linking to Requirements
 
       @pytest.fixture(scope="session", autouse=True)
       def add_metadata(record_testsuite_property):
-         record_testsuite_property("ASIL", "<ASIL>")
          record_testsuite_property("TestType", "<TestType>")
          record_testsuite_property("DerivationTechnique", "<DerivationTechnique>")
       class Testclass(TestSim):
@@ -123,8 +113,6 @@ Test Linking to Requirements
         #[record_property("PartiallyVerifies", "ID_2, ID_3, ...")]
         #[record_property("FullyVerifies", "ID_4, ID_5, ...")]
         #[record_property("Description", "<Description>")]
-        #[record_property("Status", "<Status>")]
-        #[record_property("ASIL", "<ASIL>")]
         #[record_property("TestType", "<TestType>")]
         #[record_property("DerivationTechnique", "<DerivationTechnique>")]
         #[test]
@@ -134,6 +122,33 @@ Test Linking to Requirements
 
 .. gd_req:: Test Independence
     :id: gd_req__verification_independence
-    :status: draft
+    :status: valid
 
     The approver of a pull request shall differ from the author(s) of the pull request in all pull requests.
+
+.. gd_req:: Verification Reporting
+    :id: gd_req__verification_reporting
+    :status: valid
+
+    Verification reports shall be automatically generated. These may be independent documents (i.e. not integrated into sphinx documentation).
+    The content of the reports is specified in :need:`gd_temp__platform_ver_report` and :need:`gd_temp__mod_ver_report`.
+
+.. gd_req:: Verification Documentation Checks
+    :id: gd_req__verification_checks
+    :status: valid
+
+    The following checks shall be implemented on test metadata:
+    - TestType and DerivationTechnique shall be set
+    - Description shall not be empty
+    - In a Platform Test Partially/FullyVerifies shall be set to a Platform Requirement
+    - If Partially/FullyVerifies are set in Feature Integration Test these shall link to Feature Requirements
+    - If Partially/FullyVerifies are set in Component Integration Test these shall link to Component Requirements
+    - If Partially/FullyVerifies are set in Unit Test these shall link to Component Requirements
+
+.. gd_req:: Verification Documentation Checks Extended
+    :id: gd_req__verification_checks_extended
+    :status: draft
+
+    The following checks shall be implemented on test metadata:
+    - If TestType is set to requirements-based then PartiallyVerifies or FullyVerifies shall contain a link to at least one requirement
+    - If TestType is set to interface-test then PartiallyVerifies or FullyVerifies shall contain a link to at least one interface

process/process_areas/verification/guidance/verification_report_template.rst

@@ -24,7 +24,8 @@ Verification Report Templates
 
     This document implements :need:`wp__verification__module_ver_report`.
 
-      | Place the document header here following the `REPLACE_doc__documentation_mgt_plan`.
+      | Place the document header here following the definitions in your module's
+      | Documentation Management Plan (doc__documentation_mgt_plan).
       |
       | This verification report is based on the :need:`wp__verification__plan`.
       | It covers all the components of the above stated module.
@@ -34,7 +35,7 @@ Verification Report Templates
       | **1. Verification Coverage**
       |
       | **1.1. on Requirements**
-      | - Lists of component requirements (incl. AoU satisfied by the component) tested by which test case, passed/failed and completeness verdict
+      | - Lists of component requirements (incl. AoU satisfied by the component) tested by which test case, passed/failed/not_run and completeness verdict
       |   (this shall be generated by tools and accompanied by progress charts to be usable also for project steering)
       | - For external component Assumptions of Use: coverage by platform safety manual or feature/components incl. test case, passed/failed and completeness verdict
       | - This is split in a list of QM requirements tested and a separate list of tests for ASIL rated requirements.
@@ -89,7 +90,8 @@ Verification Report Templates
 
     This document implements :need:`wp__verification__platform_ver_report`.
 
-      | Place the document header here following the `REPLACE_doc__documentation_mgt_plan`.
+      | Place the document header here following the definitions in your project's
+      | Documentation Management Plan (doc__documentation_mgt_plan).
       |
       | This verification report is based on the :need:`wp__verification__plan`.
       | It covers all the selected features of the SW platform.
@@ -99,7 +101,7 @@ Verification Report Templates
       | **1. Verification Coverage**
       |
       | **1.1. on Requirements**
-      | - Lists of feature requirements tested by which test case, passed/failed and completeness verdict
+      | - Lists of feature requirements tested by which test case, passed/failed/not_run and completeness verdict
       |   (this shall be generated by tools and accompanied by progress charts to be usable also for project steering)
       | - For external component Assumptions of Use: coverage by platform safety manual or feature
       | - This is split in a list of QM requirements tested and a separate list of tests for ASIL rated requirements.

process/process_areas/verification/guidance/verification_specification.rst

@@ -47,20 +47,12 @@ A test specification contains the following attributes.
        - Expected outcome (e.g. "A success message is displayed.")
        - Test environment (e.g. network configuration, clean system state)
      -
-   * - Status
-     -
-       - valid
-       - invalid
-     -
-     -
    * - TestType
      - Examples are:
 
        - requirements-based
        - interface-test
-       - boundary
        - coverage (various types apply, shall be tool supported)
-       - for :need:`wp__verification__sw_unit_test` also fault-injection
      - These are example values and an incomplete list.
        A full list of test types is available in :need:`doc_concept__verification__process` at
        :ref:`verification_concept_types_methods`.

process/process_areas/verification/verification_concept.rst

@@ -117,9 +117,10 @@ The derivation of test cases can also be based on certain methods.
  * Analysis of boundary values (boundary-values)
  * Analysis of equivalence classes (equivalence-classes)
  * Analysis of requirements (requirements-analysis)
+ * Analysis of design (design-analysis)
  * Error guessing based on knowledge or experience (error-guessing)
  * Fuzzy testing (fuzz-testing)
- * Exlporative testing (explorative-testing)
+ * Explorative testing (explorative-testing)
 
 Usually the defined methods are not applied on each verification level between unit and platform level.
 Also their execution may differ whether it is a QM or ASIL rated test case.