Update branch production branch Main with release v1.5.0 content (#756)

* Bug fix to correct job log returining and other edge cases (#688)

* Update zos_job_submit to handle some edge cases and return the job log nearly always

Signed-off-by: ddimatos <[email protected]>

* Linting correction

Signed-off-by: ddimatos <[email protected]>

* Add changelog fragement for pr 683

Signed-off-by: ddimatos <[email protected]>

* Clean up comments in code

Signed-off-by: ddimatos <[email protected]>

* Update restructured text for modules and changelog fragment

Signed-off-by: ddimatos <[email protected]>

* Add support to check for security exception to job submit module

Signed-off-by: ddimatos <[email protected]>

* Add additonal logic for jobs that use typerun=scan

Signed-off-by: ddimatos <[email protected]>

* Update chnagelog fragment

Signed-off-by: ddimatos <[email protected]>

* Correct run on sentence

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>

* Added additional `D SYMBOLS` example to  zos_operator documentation (#729)

* Update zos_operator with an additional example

Signed-off-by: ddimatos <[email protected]>

* Added change fragment

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>

* Ported bugfixes for #664 and #678

* Updated file normalization for sequential datasets

* Fixed encoding when a remote source is used

* Added encoding normalization for record length (#664)

This is a port and tweak from the normalization found in the L2 support branch. Now the normalization applies for both local and remote source files. There is still work left to validate how this behaves when copying to partitioned datasets, and see if the normalization needs to handle directories as well.

* Refactored normalization for USS files

* Updated line endings normalization in PDSEs

* Updated create_temp_with_lf_endings

* Added tests for line endings normalization

* Added changelog fragment

* Add copy lib member test case

* Corrected changelog fragment

* Add conflict resolution

* Update 641-copy-loadlib-member.yml

* zos_copy forward port a test case for symbols on a volume (#739)

* Add a test case for symbols using an existing volume on our test system

Signed-off-by: ddimatos <[email protected]>

* Add changelog fragment

Signed-off-by: ddimatos <[email protected]>

* Update 739-zos_copy-volume-symbol-test.yml

Update description to use symbols overs special chars.

* Update test case with added comments

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>

* bring in updates to zos_gather_facts RETURN docstring (#744)

Signed-off-by: Ketan Kelkar <[email protected]>

* Solve problem 619 about set mode on dest file (#746)

* Solve problem 619 about set mode on dest file

* Add fragment of 746 PR

---------

Co-authored-by: Andre Marcel Gutierrez Benitez <[email protected]>

* Merge release tasks for v1.5.0 into release branch (#752)

* Updated metadata

* Updated copyright information

* Updated copyright in README

* Generated changelog

* Added previous versions to changelog

* Changed a minor_change fragment to bugfix

* Updated release notes

* Update dependency finder to allow for all tests minus skip tests

Signed-off-by: ddimatos <[email protected]>

* Update release notes and changelog with full date formats

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>
Co-authored-by: ddimatos <[email protected]>

* remove changelog fragments no longer needed (#754)

Signed-off-by: ddimatos <[email protected]>

---------

Signed-off-by: ddimatos <[email protected]>
Signed-off-by: Ketan Kelkar <[email protected]>
Co-authored-by: Ivan Moreno <[email protected]>
Co-authored-by: Fernando Flores <[email protected]>
Co-authored-by: Rich Parker <[email protected]>
Co-authored-by: ketankelkar <[email protected]>
Co-authored-by: André Marcel Gutiérrez Benítez <[email protected]>
Co-authored-by: Andre Marcel Gutierrez Benitez <[email protected]>

Author: 7 people

CHANGELOG.rst

@@ -49,11 +49,11 @@ and ansible-doc to automate tasks on z/OS.
 
 Ansible version compatibility
 =============================
-This collection has been tested against the following Ansible versions: >=2.9,<=2.14.1.
+This collection has been tested against the following Ansible versions: >=2.9,<2.15.
 
 Copyright
 =========
-© Copyright IBM Corporation 2020-2021.
+© Copyright IBM Corporation 2020-2023.
 
 License
 =======

README.md

@@ -109,4 +109,4 @@ plugins:
   shell: {}
   strategy: {}
   vars: {}
-version: 1.5.0-beta.1
+version: 1.5.0

changelogs/.plugin-cache.yaml

@@ -232,7 +232,7 @@ releases:
     - v1.3.3_summary.yml
     - v1.3.3_summary_bugs.yml
     release_date: '2022-06-07'
-  1.3.4:
+  1.3.5:
     changes:
       bugfixes:
       - "zos_ssh - connection plugin was updated to correct a bug in Ansible that\n
@@ -251,6 +251,79 @@ releases:
     - 328-rename-retries-to-reconnection_retries.yml
     - v1.3.4_summary.yml
     release_date: '2022-06-07'
+  1.3.6:
+    changes:
+      bugfixes:
+      - jobs.py - fixes a utility used by module `zos_job_output` that would truncate
+        the DD content. (https://github.com/ansible-collections/ibm_zos_core/pull/462)
+      - zos_copy - fixes a bug that when a directory is copied from the controller
+        to the managed node and a mode is set, the mode is now applied to the directory
+        on the controller. If the directory being copied contains files and mode is
+        set, mode will only be applied to the files being copied not the pre-existing
+        files.(https://github.com/ansible-collections/ibm_zos_core/pull/462)
+      - zos_copy - fixes a bug where options were not defined in the module argument
+        spec that will result in error when running `ansible-core` 2.11 and using
+        options `force` or `mode`. (https://github.com/ansible-collections/ibm_zos_core/pull/462)
+      - zos_fetch - fixes a bug where an option was not defined in the module argument
+        spec that will result in error when running `ansible-core` 2.11 and using
+        option `encoding`. (https://github.com/ansible-collections/ibm_zos_core/pull/462)
+      - zos_job_submit - fixes a bug where an option was not defined in the module
+        argument spec that will result in error when running `ansible-core` 2.11 and
+        using option `encoding`. (https://github.com/ansible-collections/ibm_zos_core/pull/462)
+      - zos_ssh - fixes connection plugin which will error when using `ansible-core`
+        2.11 with an `AttributeError module 'ansible.constants' has no attribute 'ANSIBLE_SSH_CONTROL_PATH_DIR'`.
+        (https://github.com/ansible-collections/ibm_zos_core/pull/462)
+      - zos_ssh - fixes connection plugin which will error when using `ansible-core`
+        2.11 with an `AttributeError module 'ansible.constants' has no attribute 'ANSIBLE_SSH_CONTROL_PATH_DIR'`.
+        (https://github.com/ansible-collections/ibm_zos_core/pull/513)
+      minor_changes:
+      - zos_copy - was enhanced for when `src` is a directory and ends with "/", the
+        contents of it will be copied into the root of `dest`. If it doesn't end with
+        "/", the directory itself will be copied. (https://github.com/ansible-collections/ibm_zos_core/pull/515)
+      release_summary: "Release Date: '2022-10-07'\nThis changelog describes all changes
+        made to the modules and plugins included\nin this collection. The release
+        date is the date the changelog is created.\nFor additional details such as
+        required dependencies and availability review\nthe collections `release notes
+        <https://ibm.github.io/z_ansible_collections_doc/ibm_zos_core/docs/source/release_notes.html>`__
+        \n"
+    fragments:
+    - 462-copy-fetch-submit-utils.yml
+    - 513-zos_ssh-support-ansible-2.11.yml
+    - 515-copy-support-directories.yml
+    - v1.3.6_summary.yml
+    release_date: '2022-10-07'
+  1.4.0:
+    changes:
+      minor_changes:
+      - zos_copy - enhanced to optimize how it captures the permission bits state
+        for the `dest`. This change now reviews the source files instead of traversing
+        the entire `dest` path. (https://github.com/ansible-collections/ibm_zos_core/pull/561)
+      - zos_copy - enhanced to support creating a parent directory when it does not
+        exist in the `dest` path. Prior to this change, if a parent directory anywhere
+        in the path did not exist the task would fail as it was stated in documentation.
+        (https://github.com/ansible-collections/ibm_zos_core/pull/561)
+      - "zos_copy - enhanced to support system symbols in PARMLIB. System symbols
+        are elements that allow different z/OS\xAE systems to share PARMLIB definitions
+        while retaining unique values in those definitions. This was fixed in a future
+        release through the use of one of the ZOAU dependency but this version of
+        `ibm_zos_core` does not support that dependency version so this support was
+        added. (https://github.com/ansible-collections/ibm_zos_core/pull/566)"
+      release_summary: 'Release Date: ''2022-12-07''
+
+        This changelog describes all changes made to the modules and plugins included
+
+        in this collection. The release date is the date the changelog is created.
+
+        For additional details such as required dependencies and availability review
+
+        the collections `release notes <https://ibm.github.io/z_ansible_collections_doc/ibm_zos_core/docs/source/release_notes.html>`__
+
+        '
+    fragments:
+    - 561-update-directory-create.yml
+    - 566-update-with-symbol-support.yml
+    - v1.4.0_summary.yml
+    release_date: '2022-12-07'
   1.4.0-beta.1:
     changes:
       bugfixes:
@@ -378,22 +451,118 @@ releases:
     - 507-display-specific-ddname.yml
     - v1.4.0-beta.2_summary.yml
     release_date: '2022-10-13'
+  1.4.1:
+    changes:
+      bugfixes:
+      - zos_copy - Copy failed from a loadlib member to another loadlib member. Fix
+        now looks for error in stdout in the if statement to use -X option. (https://github.com/ansible-collections/ibm_zos_core/pull/640)
+      - zos_copy - Fixed a bug where the module would change the mode for a directory
+        when copying into it the contents of another. (https://github.com/ansible-collections/ibm_zos_core/pull/742)
+      - zos_copy - Fixes a bug where files not encoded in IBM-1047 would trigger an
+        error while computing the record length for a new destination dataset. Issue
+        664. (https://github.com/ansible-collections/ibm_zos_core/pull/732)
+      - zos_copy - Fixes a bug where the code for fixing an issue with newlines in
+        files (issue 599) would use the wrong encoding for normalization. Issue 678.
+        (https://github.com/ansible-collections/ibm_zos_core/pull/732)
+      - zos_copy - fixed wrongful creation of destination backups when module option
+        `force` is true, creating emergency backups meant to restore the system to
+        its initial state in case of a module failure only when force is false. (https://github.com/ansible-collections/ibm_zos_core/pull/590)
+      - zos_copy - fixes a bug where the computed record length for a new destination
+        dataset would include newline characters. (https://github.com/ansible-collections/ibm_zos_core/pull/620)
+      - zos_job_query - fixes a bug where a boolean was not being properly compared.
+        (https://github.com/ansible-collections/ibm_zos_core/pull/379)
+      release_summary: 'Release Date: ''2023-04-18''
+
+        This changelog describes all changes made to the modules and plugins included
+
+        in this collection. The release date is the date the changelog is created.
+
+        For additional details such as required dependencies and availability review
+
+        the collections `release notes <https://ibm.github.io/z_ansible_collections_doc/ibm_zos_core/docs/source/release_notes.html>`__
+
+        '
+    fragments:
+    - 579-zos-query-boolean-correction.yml
+    - 588-update-emergency-backup.yml
+    - 599-copy-carriage-return.yml
+    - 601-copy-loadlib-member.yml
+    - 728-zos_operator-example-updates.yml
+    - 732-zos_copy-encoding-bugs.yml
+    - 742_zos_copy-mode-is-applied-to-the-destination-directory-a-deviation-from-the-communtiy-module-behavior.yaml
+    - v1.4.1_summary.yml
+    release_date: '2023-04-18'
+  1.5.0:
+    changes:
+      bugfixes:
+      - zos_copy - Copy failed from a loadlib member to another loadlib member. Fix
+        now looks for error in stdout in the if statement to use -X option. (https://github.com/ansible-collections/ibm_zos_core/pull/641)
+      - zos_copy - Fixed a bug where the module would change the mode for a directory
+        when copying into it the contents of another. (https://github.com/ansible-collections/ibm_zos_core/pull/746)
+      - zos_copy - Fixes a bug where files not encoded in IBM-1047 would trigger an
+        error while computing the record length for a new destination dataset. Issue
+        664. (https://github.com/ansible-collections/ibm_zos_core/pull/725)
+      - zos_copy - Fixes a bug where the code for fixing an issue with newlines in
+        files (issue 599) would use the wrong encoding for normalization. Issue 678.
+        (https://github.com/ansible-collections/ibm_zos_core/pull/725)
+      - zos_job_submit - Fixes the issue when `wait_time_s` was set to 0 that would
+        result in a `type` error that a stack trace would result in the response,
+        issue 670. (https://github.com/ansible-collections/ibm_zos_core/pull/683)
+      - zos_job_submit - Fixes the issue when a job encounters a security exception
+        no job log would would result in the response, issue 684. (https://github.com/ansible-collections/ibm_zos_core/pull/683)
+      - zos_job_submit - Fixes the issue when a job is configured for a syntax check
+        using TYPRUN=SCAN that it would wait the full duration set by `wait_time_s`
+        to return a response, issue 685. (https://github.com/ansible-collections/ibm_zos_core/pull/683)
+      - zos_job_submit - Fixes the issue when a job is configured for a syntax check
+        using TYPRUN=SCAN that no job log would result in the response, issue 685.
+        (https://github.com/ansible-collections/ibm_zos_core/pull/683)
+      - zos_job_submit - Fixes the issue when a job is purged by the system that a
+        stack trace would result in the response, issue 681. (https://github.com/ansible-collections/ibm_zos_core/pull/683)
+      - zos_job_submit - Fixes the issue when invalid JCL syntax is submitted that
+        a stack trace would result in the response, issue 623. (https://github.com/ansible-collections/ibm_zos_core/pull/683)
+      - zos_job_submit - Fixes the issue when resources (data sets) identified in
+        JCL did not exist such that a stack trace would result in the response, issue
+        624. (https://github.com/ansible-collections/ibm_zos_core/pull/683)
+      - zos_job_submit - Fixes the issue where the response did not include the job
+        log when a non-zero return code would occur, issue 655. (https://github.com/ansible-collections/ibm_zos_core/pull/683)
+      - zos_operator - fixed incorrect example descriptions and updated the doc to
+        highlight the deprecated option `wait`. (https://github.com/ansible-collections/ibm_zos_core/pull/648)
+      release_summary: 'Release Date: ''2023-04-21''
+
+        This changelog describes all changes made to the modules and plugins included
+
+        in this collection. The release date is the date the changelog is created.
+
+        For additional details such as required dependencies and availability review
+
+        the collections `release notes <https://ibm.github.io/z_ansible_collections_doc/ibm_zos_core/docs/source/release_notes.html>`__'
+    fragments:
+    - 641-copy-loadlib-member.yml
+    - 648-zos_operator-examples.yml
+    - 663-zos_gather_facts-update-docstring.yml
+    - 683-zos_job_submit-bugs.yml
+    - 725-zos_copy-encoding-bugs.yml
+    - 729-zos_operator-example-added.yml
+    - 739-zos_copy-volume-symbol-test.yml
+    - 746--Mode-set-for-files-is-applied-to-destination-directory.yml
+    - v1.5.0_summary.yml
+    release_date: '2023-04-21'
   1.5.0-beta.1:
     changes:
       bugfixes:
       - zos_copy - Fixes a bug such that the module fails when copying files from
         a directory needing also to be encoded. The failure would also delete the
         `src` which was not desirable behavior. Fixes deletion of src on encoding
         error. (https://github.com/ansible-collections/ibm_zos_core/pull/321).
-      - zos_copy - Fixes wrongful creation of destination backups when module option
-        `force` is true, creating emergency backups meant to restore the system to
-        its initial state in case of a module failure only when force is false. (https://github.com/ansible-collections/ibm_zos_core/pull/590)
       - zos_copy - Fixes a bug where copying a member from a loadlib to another loadlib
         fails. (https://github.com/ansible-collections/ibm_zos_core/pull/640)
       - zos_copy - Fixes a bug where if a destination has accented characters in its
         content, the module would fail when trying to determine if it is empty. (https://github.com/ansible-collections/ibm_zos_core/pull/634)
       - zos_copy - Fixes a bug where the computed record length for a new destination
         dataset would include newline characters. (https://github.com/ansible-collections/ibm_zos_core/pull/620)
+      - zos_copy - Fixes wrongful creation of destination backups when module option
+        `force` is true, creating emergency backups meant to restore the system to
+        its initial state in case of a module failure only when force is false. (https://github.com/ansible-collections/ibm_zos_core/pull/590)
       - zos_copy - module was updated to correct a bug in the case when the destination
         (dest) is a PDSE and the source (src) is a Unix Systems File (USS). The module
         would fail in determining if the PDSE actually existed and try to create it
@@ -406,8 +575,8 @@ releases:
         the module should look for the data set, whether or not it is cataloged. (https://github.com/ansible-collections/ibm_zos_core/pull/325).
       - zos_data_set - Fixes a bug where the default record format FB was actually
         never enforced and when enforced it would cause VSAM creation to fail with
-        a Dynalloc failure. Also cleans up some of the options that are set by
-        default when they have no bearing for batch. (https://github.com/ansible-collections/ibm_zos_core/pull/647)
+        a Dynalloc failure. Also cleans up some of the options that are set by default
+        when they have no bearing for batch. (https://github.com/ansible-collections/ibm_zos_core/pull/647)
       - zos_fetch - Updates the modules behavior when fetching VSAM data sets such
         that the maximum record length is now determined when creating a temporary
         data set to copy the VSAM data into and a variable-length (VB) data set is
@@ -515,7 +684,7 @@ releases:
         behaviors and reduces the possibility to encounter a permissions issue. (https://github.com/ansible-collections/ibm_zos_core/issues/389).
       - zos_job_submit - was updated to include an additional error code condition
         JCLERR. (https://github.com/ansible-collections/ibm_zos_core/pull/312)
-      - zos_lineinfile- updates the module with a new option named tmp_hlq. This allows
+      - zos_lineinfile - updates the module with a new option named tmp_hlq. This allows
         for a user to specify the data set high level qualifier (HLQ) used in any
         temporary data set created by the module. Often, the defaults are not permitted
         on systems, this provides a way to override the defaults. (https://github.com/ansible-collections/ibm_zos_core/pull/341).