Merge pull request #58 from mathworks/DATF-1988-add-pdfTestReport

Adding PDF test report option for non-mac platforms

Author: hkhan17

CONFIGDOC.md

@@ -12,7 +12,7 @@ To configure the plugin for a freestyle project, select **Run MATLAB Tests** fro
 
   ![new_add_build_step](https://user-images.githubusercontent.com/47204011/55624172-be54a100-57c2-11e9-9596-52d3a60ee467.png)
 
-  ![new_enter_matlabroot](https://user-images.githubusercontent.com/51316279/70772996-28830400-1d9c-11ea-92d1-db9f054c304c.png)
+  ![new_enter_matlabroot](https://user-images.githubusercontent.com/51316279/72131234-a5a49780-33a1-11ea-8497-45a1d9161ecf.png)
 
 ### Option 1: Freestyle Project with Automatic Test Mode
 With the Jenkins plugin for MATLAB, you have the option to run your tests in either *automatic* or *custom* mode. The automatic test mode employs a default setting to run tests written using the MATLAB Unit Testing Framework and/or Simulink Test. If your source code is organized as files and folders within a project, the plugin will consider any test files in the project that have been tagged as **Test**. If your code does not leverage a project or uses a MATLAB version prior to R2019a, the plugin will consider all tests in the current Jenkins workspace including the subfolders. 
@@ -23,30 +23,32 @@ The automatic test execution feature of the plugin enables you to generate diffe
 
 1) From the **Test mode** drop-down list, select the **Automatic** option (**Automatic** is the default testing mode).
 
-  ![new_select_automatic_option](https://user-images.githubusercontent.com/51316279/70773784-6719be00-1d9e-11ea-9a6c-7b277d63e10a.png)
+  ![new_select_automatic_option](https://user-images.githubusercontent.com/51316279/72131293-cc62ce00-33a1-11ea-9573-813ac3060790.png)
 
 2) Select your desired test artifacts.
 
-  ![new_select_all_test_artifacts](https://user-images.githubusercontent.com/51316279/70773293-0dfd5a80-1d9d-11ea-8d24-28407e67c8a5.png)
+  ![new_select_all_test_artifacts](https://user-images.githubusercontent.com/51316279/72131319-e3a1bb80-33a1-11ea-99de-aef714bb5300.png)
 
   The selected artifacts will be saved in the **matlabTestArtifacts** folder of the Jenkins workspace.
 
   ![Workspace01](https://user-images.githubusercontent.com/47204011/55470859-1e621080-5626-11e9-98f2-044144272643.JPG)
 
-  ![Test_artifacts](https://user-images.githubusercontent.com/51316279/70776181-5bc89180-1da2-11ea-92e6-4c71deab7199.png)
+  ![Test_artifacts](https://user-images.githubusercontent.com/51316279/72140084-f625f000-33b5-11ea-8fd6-a4ebde6fcc5f.png)
 
   If you do not select any of the test artifact check boxes, the **matlabTestArtifacts** folder will not be created in the workspace. However, tests will still run and potential test failures will fail the build. 
 
   The **Automatic** test mode results in a MATLAB script file named **runMatlabTests.m** in the Jenkins workspace. The plugin uses this file to run the tests and generate the test artifacts. You can review the contents of the script to understand the testing workflow.
 
   ![Workspace01](https://user-images.githubusercontent.com/47204011/55470859-1e621080-5626-11e9-98f2-044144272643.JPG)
 
+  **Note:** Producing a PDF test report is not currently supported on MacOS platforms.
+
 ### Option 2: Freestyle Project with Custom Test Mode
 This option enables you to develop your custom MATLAB commands for running tests. To configure the Jenkins build where you can customize the MATLAB test execution, follow these steps.
 
 1) From the **Test mode** drop-down list, select the **Custom** option.
 
-  ![new_select_custom](https://user-images.githubusercontent.com/51316279/70775386-42731580-1da1-11ea-9523-41c83443ee5b.png)
+  ![new_select_custom](https://user-images.githubusercontent.com/51316279/72131434-31b6bf00-33a2-11ea-99da-a479dd76f826.png)
 
 2) Enter your commands in the **MATLAB command** field. If you specify more than one MATLAB command, use a comma or semicolon to separate the commands. The build will fail if the execution of any command results in an error.
 
@@ -69,33 +71,35 @@ To configure the plugin for a matrix build where tests run automatically in mult
 
 1) Add a user-defined axis in the **Configuration Matrix** to represent the MATLAB versions in the build. Specify the name of the axis in the **Name** field and its values in the **Values** field. Separate the elements in the **Values** field with a space. In this example, four MATLAB versions are specified, which will be used to run the same set of tests.
 
-![image](https://user-images.githubusercontent.com/47204011/62603081-c2c8cc00-b912-11e9-83a4-c5462f58f607.png)
+![image](https://user-images.githubusercontent.com/51316279/72132482-139e8e00-33a5-11ea-9d94-b8381c5dceca.png)
 
 2) In the **Run MATLAB Tests** section of the project, include the user-defined axis name in the **MATLAB root** field to specify the locations where MATLAB is installed. In this example, **$VERSION** will be replaced by one axis value per build step.
 
-![image](https://user-images.githubusercontent.com/51316279/70773424-72b8b500-1d9d-11ea-97ca-807ecd6c4e47.png)
+![image](https://user-images.githubusercontent.com/51316279/72131571-aee23400-33a2-11ea-8e76-4b36274b9d5b.png)
 
 You can select the test artifact check boxes when tests run automatically. Once you have made your selections, save your settings and run the build.
 
+**Note:** Producing a PDF test report is not currently supported on MacOS platforms.
+
 ### Option 2: Multi-Configuration Project with Custom Test Mode
 
 To configure the matrix build where you can customize the MATLAB test execution, create a multi-configuration project and follow these steps.
 
 1) Add a user-defined axis in the **Configuration Matrix** to represent the MATLAB versions in the build. 
 
-![image](https://user-images.githubusercontent.com/47204011/62603081-c2c8cc00-b912-11e9-83a4-c5462f58f607.png)
+![image](https://user-images.githubusercontent.com/51316279/72132482-139e8e00-33a5-11ea-9d94-b8381c5dceca.png)
 
 2) Add another user-defined axis using the **Add axis** button. In this example, the **TEST_TAG** axis specifies the possible test tags for a group of test elements.
 
-![image](https://user-images.githubusercontent.com/47204011/62517774-b6c30880-b845-11e9-86a0-8344a281fb27.png)
+![image](https://user-images.githubusercontent.com/51316279/72131623-d6d19780-33a2-11ea-9205-577d6634a1a6.png)
 
 3) In the **Run MATLAB Tests** section of the project, use the **VERSION** axis to specify the locations where MATLAB is installed.
 
-![image](https://user-images.githubusercontent.com/51316279/70773424-72b8b500-1d9d-11ea-97ca-807ecd6c4e47.png)
+![image](https://user-images.githubusercontent.com/51316279/72131668-ef41b200-33a2-11ea-9f97-d9476ae0431a.png)
 
 4) From the **Test mode** drop-down list, select the **Custom** option. Use the second user-defined axis to create your commands and enter them in the **MATLAB command** field. Then, save your settings and run the build. 
 
-![image](https://user-images.githubusercontent.com/47204011/62686681-cd529680-b9e2-11e9-82c1-c211f1740be4.png)
+![image](https://user-images.githubusercontent.com/51316279/72131696-04b6dc00-33a3-11ea-876f-5cdff3a027b4.png)
 
 **Notes:**
 1) For a user-defined axis named **VAR**,  **$VAR** and **${VAR}** are both valid formats for accessing the values.

README.md

@@ -1,6 +1,6 @@
 # Jenkins MATLAB Plugin
 
-The Jenkins plugin for MATLAB® enables you to easily run your MATLAB and Simulink® tests and generate artifacts such as *JUnit* and *TAP* test results, and *Cobertura* code or model coverage reports. You can also export your Simulink Test™ Manager results. This guide shows you examples of how to configure the plugin for different objectives, considering a Microsoft® Windows® platform.
+The Jenkins plugin for MATLAB® enables you to easily run your MATLAB and Simulink® tests and generate artifacts such as *PDF* test reports, *JUnit* and *TAP* test results, and *Cobertura* code or model coverage reports. You can also export your Simulink Test™ Manager results. This guide shows you examples of how to configure the plugin for different objectives, considering a Microsoft® Windows® platform.
 
 ## Documentation 
 

src/main/java/com/mathworks/ci/MatlabBuilder.java

@@ -1,7 +1,7 @@
 package com.mathworks.ci;
 
 /*
- * Copyright 2019 The MathWorks, Inc.
+ * Copyright 2019-2020 The MathWorks, Inc.
  * 
  * This is Matlab Builder class which describes the build step and its components. Builder displays
  * Build step As "Run MATLAB Tests" under Build steps. Author : Nikhil Bhoski email :
@@ -347,6 +347,7 @@ public static class RunTestsAutomaticallyOption extends TestRunTypeList {
         private boolean taCoberturaChkBx;
         private boolean taSTMResultsChkBx;
         private boolean taModelCoverageChkBx;
+        private boolean taPDFReportChkBx;
 
         @DataBoundConstructor
         public RunTestsAutomaticallyOption() {
@@ -378,6 +379,11 @@ public void setTaModelCoverageChkBx(boolean taModelCoverageChkBx) {
             this.taModelCoverageChkBx = taModelCoverageChkBx;
         }
 
+        @DataBoundSetter
+        public void setTaPDFReportChkBx(boolean taPDFReportChkBx) {
+            this.taPDFReportChkBx = taPDFReportChkBx;
+        }
+                
         public boolean getTatapChkBx() {
             return tatapChkBx;
         }
@@ -398,6 +404,10 @@ public boolean getTaModelCoverageChkBx() {
             return taModelCoverageChkBx;
         }
 
+        public boolean getTaPDFReportChkBx() {
+            return taPDFReportChkBx;
+        }
+                
         @Extension
         public static final class DescriptorImpl extends TestRunTypeDescriptor {
             @Override
@@ -419,6 +429,8 @@ public boolean getBooleanByName(String memberName) {
                     return this.getTaSTMResultsChkBx();
                 case "taModelCoverageChkBx":
                     return this.getTaModelCoverageChkBx();
+                case "taPDFReportChkBx":
+                    return this.getTaPDFReportChkBx();
                 default:
                     return false;
             }
@@ -611,13 +623,14 @@ private String getNodeSpecificFileSeperator(Launcher launcher) {
 
     // Concatenate the input arguments
     private String getInputArguments() {
+        String pdfReport = MatlabBuilderConstants.PDF_REPORT + "," + getTestRunTypeList().getBooleanByName("taPDFReportChkBx");
     	String tapResults = MatlabBuilderConstants.TAP_RESULTS + "," + getTestRunTypeList().getBooleanByName("tatapChkBx");
     	String junitResults = MatlabBuilderConstants.JUNIT_RESULTS + "," + getTestRunTypeList().getBooleanByName("taJunitChkBx");
     	String stmResults = MatlabBuilderConstants.STM_RESULTS + "," + getTestRunTypeList().getBooleanByName("taSTMResultsChkBx");
     	String coberturaCodeCoverage = MatlabBuilderConstants.COBERTURA_CODE_COVERAGE + "," + getTestRunTypeList().getBooleanByName("taCoberturaChkBx");
     	String coberturaModelCoverage = MatlabBuilderConstants.COBERTURA_MODEL_COVERAGE + "," + getTestRunTypeList().getBooleanByName("taModelCoverageChkBx");
 
-    	String inputArgsToMatlabFcn = tapResults + "," + junitResults + ","
+    	String inputArgsToMatlabFcn = pdfReport + "," + tapResults + "," + junitResults + ","
     			+ stmResults + "," + coberturaCodeCoverage + "," + coberturaModelCoverage;
 
         return inputArgsToMatlabFcn;

src/main/java/com/mathworks/ci/MatlabBuilderConstants.java

@@ -1,6 +1,6 @@
 package com.mathworks.ci;
 /*
- * Copyright 2019 The MathWorks, Inc.
+ * Copyright 2019-2020 The MathWorks, Inc.
  */
 
 public class MatlabBuilderConstants {
@@ -16,6 +16,7 @@ public class MatlabBuilderConstants {
     static final String AUTOMATIC_OPTION = "RunTestsAutomaticallyOption";
 
     // Input parameter names (Passed to runMatlabTests.m as name-value pair arguments)
+    static final String PDF_REPORT = "'PDFReport'";
     static final String TAP_RESULTS = "'TAPResults'";
     static final String JUNIT_RESULTS = "'JUnitResults'";
     static final String STM_RESULTS = "'SimulinkTestResults'";

src/main/resources/com/mathworks/ci/MatlabBuilder/RunTestsAutomaticallyOption/config.jelly

@@ -2,6 +2,10 @@
 <j:jelly xmlns:j="jelly:core" xmlns:st="jelly:stapler" xmlns:d="jelly:define" xmlns:l="/lib/layout" xmlns:t="/lib/hudson" xmlns:f="/lib/form">
 
 <f:section title="Generate Test Artifacts">
+  	 <f:entry field="taPDFReportChkBx">	
+    	<f:checkbox title="PDF test report " name="taPDFReportChkBx" checked="${instance.taPDFReportChkBx}"/>	
+     </f:entry>
+
 	 <f:entry field="tatapChkBx">
     	<f:checkbox title="TAP test results" name="tatapChkBx" checked="${instance.tatapChkBx}"/>
      </f:entry>

src/main/resources/com/mathworks/ci/MatlabBuilder/RunTestsAutomaticallyOption/help-taPDFReportChkBx.html

@@ -1,10 +1,10 @@
 <div>
 <br>
 Generate a test report in PDF format, <b>testreport.pdf</b>, and save it in the <b>matlabTestArtifacts</b> folder of the Jenkins workspace.
-<br>
-<br>
-Note: Due to Jenkins Content Security Policy rules, the generated report might not open properly from within the Jenkins workspace. 
+<br><br>
+Due to Jenkins Content Security Policy rules, the generated report might not open properly from within the Jenkins workspace. 
 Consider copying the report to a location outside the workspace and opening it from there. For more information, 
 see <a href="http://wiki.jenkins.io/display/JENKINS/Configuring+Content+Security+Policy">Configuring Jenkins Content Security Policy</a>.
-<br>    
+<br><br>
+<b>Note:</b>This feature is not currently supported on MacOS platforms.
 </div>

src/main/resources/com/mathworks/ci/MatlabBuilder/runMatlabTests.m

@@ -1,4 +1,4 @@
-%Copyright 2019 The MathWorks, Inc.
+%Copyright 2019-2020 The MathWorks, Inc.
 
 function failed = runMatlabTests(varargin)
 
@@ -73,7 +73,7 @@
     BASE_VERSION_COBERTURA_SUPPORT = '9.3';
 
     if verLessThan('matlab',BASE_VERSION_COBERTURA_SUPPORT)
-         warning('MATLAB:testArtifact:coberturaReportNotSupported', 'Producing Cobertura results is not supported in this release.');
+         warning('MATLAB:testArtifact:coberturaReportNotSupported', 'Producing Cobertura code coverage results is not supported in this release.');
     else 
         import('matlab.unittest.plugins.CodeCoveragePlugin');
         mkdirIfNeeded(resultsDir)
@@ -111,9 +111,12 @@
     end
 end
 
-% Produce PDF test report (Not supported below R2016b)
+% Produce PDF test report (Not supported on MacOS platforms and below R2017a)
 if producePDFReport
-    if ~testReportPluginPresent
+    if ismac
+        warning('MATLAB:testArtifact:unSupportedPlatform', ...
+            'Producing a PDF test report is not currently supported on MacOS platforms.');
+    elseif ~testReportPluginPresent
         issuePDFReportUnsupportedWarning;
     else
         mkdirIfNeeded(resultsDir);
@@ -129,6 +132,7 @@
 results = runner.run(suite);
 failed = any([results.Failed]);
 
+
 function tapToFile = getTapResultFile(resultsDir)
 import('matlab.unittest.plugins.ToFile');
 mkdirIfNeeded(resultsDir)
@@ -164,7 +168,7 @@ function mkdirIfNeeded(dir)
 filePath = fullfile(resultsDir, 'simulinktestresults.mldatx');
 
 function tf = testReportPluginPresent
-BASE_VERSION_REPORTPLUGIN_SUPPORT = '9.1'; % R2016b
+BASE_VERSION_REPORTPLUGIN_SUPPORT = '9.2'; % R2017a 
 
 tf = ~verLessThan('matlab',BASE_VERSION_REPORTPLUGIN_SUPPORT);
 

src/test/java/com/mathworks/ci/MatlabBuilderTest.java

@@ -32,7 +32,7 @@
 
 
 /*
- * Copyright 2018-2019 The MathWorks, Inc.
+ * Copyright 2018-2020 The MathWorks, Inc.
  * 
  * Test class for MatlabBuilder
  * 
@@ -300,7 +300,7 @@ public void verifyRunTestAutomaticallyIsDefault() throws Exception {
         this.matlabBuilder.setMatlabRoot(getMatlabroot("R2018b"));
         FreeStyleBuild build = getBuildforRunTestAutomatically();
         jenkins.assertLogContains("-batch", build);
-        jenkins.assertLogContains("\'TAPResults\',true," +
+        jenkins.assertLogContains("\'PDFReport\',true,\'TAPResults\',true," +
                                   "\'JUnitResults\',true,\'SimulinkTestResults\',true," +
                                   "\'CoberturaCodeCoverage\',true,\'CoberturaModelCoverage\',true", build);
     }
@@ -572,6 +572,7 @@ private FreeStyleBuild getBuildforRunTestAutomatically() throws InterruptedExcep
         runOption.setTatapChkBx(true);
         runOption.setTaModelCoverageChkBx(true);
         runOption.setTaSTMResultsChkBx(true);
+        runOption.setTaPDFReportChkBx(true);
         this.matlabBuilder.setTestRunTypeList(runOption);
         project.getBuildersList().add(this.matlabBuilder);
         FreeStyleBuild build = project.scheduleBuild2(0).get();