Release v1.4.0 beta.1 (#293)

* Update to support job id up to 7 numeric positions (#259)

Signed-off-by: ddimatos <[email protected]>

* Bugfix/3687 (#251)

* 3687: looking for 'JCL ERROR' in return code message, since that will not
contain a "CC" value to return.  Temporarily assigning that a CC=16

* Expanded changes for JCL error to 'fail fast'
  + added "JCL ERROR" to job_completion_messages list
  + corrected grammar in error message@615 (was 'slow to response')

* rebuilt timing delay loops for zos_job_submit

* fixed variable usage, added more meaningful error for JCL ERROR and ABEND detectors

* simplified jcl error search string
corrected logic in test (testing for failure)

* added debug call/conv strings to return from job_submit, to understand blank returns

* argh...typo in the tracking string

* continuing to track non-response issue

* corrected issue with temp_file.name (may need to add comment that it is a string, not an object)

* eliminated duplicate temp file 2 removal

* adding badjcl indicator, since the glitch moved

* added another fail_json to job_submit

* removed failed=true as an assertion

* removed tracking statements.

* minor corrections on comments... still need to discuss return code and no-wait action changes

* changed job to return msg_code=None on JCL error
updated documentation of return to note this

* Changed documentation to help clarify the wait/wait_time_s issue
Fixed another all-caps error message
Added JobId info to long-running meessage note.

* forgot to switch back to black editor...autopep8 moved the import statements
argh!

* Added discovered customer issue into description for the 'cmd' parame… (#265)

* Added discovered customer issue into description for the 'cmd' parameter.

* Update zos_operator.py

Minor text edits

* removed colon from 'for example', which tripped a sanity test in documentation

Co-authored-by: RadhaV2020 <[email protected]>

* Bugfix/5950 (#260)

* bugfix 5950 (exception thrown when a job info didn't find a matching job)
New behavior: job.py returns a single record populated with "NOTFOUND"
output, query, and submit functions had a small change to document that a single notfound could be returned
test feature added to look for job_name=NOTAJOB to see if the notfound entry is returned

* minor change removing a print statement from job.py

* added missing blank line between functions (pep8)

* corrected two outdated comments, and verified 'not found' code is in place.

* standardized on NOTFOUND, tweaked stepname.
fixed the 'no jobs found' reaction.

* correction on adding a 'not found' message...needed to pass blanks in one case.

* corrected issue with job value assignment in 'jobs not found'
corrected grammatical typo in error message in submit

* changes to get documentation more consistent,
and not-found response more system complient

* corrected test_job_output to look for the new 'not found' text

* Changed the not found response to give None responses on system/subsystem
Documentation was general enough to work with this
Changed unit test to look for the ret_code/msg value

* job.py didn't save, so I had to re-save and re-push

* typo in tester: typed "Jobs" instead of "jobs".  argh!

* Changed return code to JOB NOT FOUND on empty record, to match other response messages.
Changed JCLERROR return code to mixed case, for readability.
Corrected documentation entries that needed to match code changes.
Changed output test to match code changes.

* Update zos_mvs_raw.py (#279)

Removed the duplicate "name" entry from the examples. Defect fix for https://jsw.ibm.com/browse/NAZARE-6112.

* Enhancement/3723 (#256)

* added '_parse_stepwise' to job_output
.. this should add job["ret_code"]["stepwise"][<step_name>] = <last field of CC string> to the job entries
.. assuming that the full response is available when the data is pulled

* whoopsie: pep8 issue on a regex string

* .. expanded documentation to show 'steps' return in the 3 impacted functions
.. added/expanded a test on zos_job_output to make sure steps were being returned

* Adjusting return definition for yaml compliance
Correcting typo in 'sample' return, that wasn't encapsulated
Added 1 more layer to output function test

* added pretty print to uss_file_to_mvs_pds_member to see what is happening there

* added more output around new sibmit result test

* issue moved, so more output messages

* added directory list to exception message...temporary

* listdir not listdirs... argh

* pep8 issue

* correction because black formatter couldn't handle [0:

* removing tmp literal

* simplified vars() to print(result)

* changed a dictionary (typo) back to a list, so I can append to it.

* more test tweaking, on test zos_job_output

* corrected test to look for steps in ret_code, not in main return object

* corrected path import

* changed a test in zos_data_set, which doesn't account for the response on a pre-existing file.

* added tracking output to test_uss_encoding_conversion_mvs_ps_to_mvs_vsam
(looks like a case where submit may not return a job list?)

* copied 5950 changes to (job.py) into this branch.
changed 'stepid' to 'step_name' and 'stepcc' to 'step_name' (snake?).
updated documentation in submit and output commands.

* turns out ansible sanity checks flake-8 as well as pep-8.
Space before semicolon line 305 of encode.

* Changed unit test to use new 'step_name' instead of 'stepid'.

* Merged job.py 5950 into this ticket.
Added 'JOB NOT FOUND' into doc message in job.py.
Corrected description of 'abend' example in all 3 zos_job modules.  (be instead of better).

* Corrected documentation to match field names from job.py.
Swapped "The" to replace "Actual" because of implications.

* Copy6113 (#282)

* Initial proposal to have copy create data set when copying uss file to non-existant data set.
Code is there, documentation is there, but I haven't done sanity run yet.

* Removal of model_ds from copy function and sub-functions
Now through ansible-sanity.
This should address Jira 6183 as well as 6113
Note: 2 minor changes to zos_fetch based on a sanity test issue there with 128/130 strings

* Changed documentation to set force default to true, which is implemented in action

* Changes based on demetri's testing:
1 - Case correction for backup_name member
2 - explicitly generating member name, instead of depending on dcp to generate a value

* tried to remove extra blank line on 1905

* Changes based on pull request comments.

* Pull request updates to doc

Signed-off-by: ddimatos <[email protected]>

* partial change moving options to suboptions - not stable

* Local *3.8) sanity passing on changed indents and size changes.
Need to do playbook testing.

* corrected zos_copy to auto-populate destination_dataset if it is not provided

* removed a find_test that looked for files it had not created.

* corrected suboptions in dictionary definition

* bugfixes from playbook:
copy to pds was allowing invalid filenames (period in member, member>char, lower member)
rebuild workbook tests to make sure these tested okay

Co-authored-by: ddimatos <[email protected]>

* Feature zos_mount  module added (#280)

* Initial work on zos_mount.  This is NOT complete.
Making sure push goes to the right place.

* Initial commit of git_mount: needs unit tests

* Basic mount/unmount testing passed manually

* made interface more ansible-like
cleaned up formatting
saving before new test runs

* passed PEP8 testing in ansible-sanity... still working on documentation

* All p3p8 resolved.  Down to 6 validte-module issues.

* final series of ansible-test issues resolved locally
moving to server-level testing

* added mount to correct directory

* removed zos_mount from wrong folder

* GPL3 header correction

* Update zos_mount.py

Added second line between metadata and documentation, to see if that addresses the license issue

* new parameters for mount, aliases for mount, updated ignore files

* minor tweak to ignore file

* corrected string search and use of tmp directory

* corrected code for pep8 provisions

* pulled dev back in for code updates.  Touched ignore files.

* Responded to Blake and Nehnam's recommendations, except:
1 - Still not sure if cp->zos_copy change will help
2 - Need to add unit tests
3 - Need to run my test playbooks again

* responding to Blake and Behnam's ssuggestions

* code changes per Blake and Behnam's suggestions
Still looking at zos_copy
Still need to build unit tests

* simplified df to search with python for safe shell run...
re-tested mount idempotence
Need to test multiline comments for bpxprm modifier

* corrections for multiline comments.  All manually tested.
first pass at function test entries.

* removed zos_mount functional test until I can get it to pass locally.

* forcing rebuild

* corrected pep8 errors introduced during recommit

* re-pulled dev branch, added basic mount tests component

* added functional tests for mount, double mount, and permanent configs

* pep8 corrections from ansible-sanity

* return value edits

* removed 'persistent' from returns because it was tripping up sanity

* corrected documentation on comments I missed previously.

* pep8 fixes...found vscode was adding spaces when moving lines around.

* Reponse changes to December review.  No new tests yet.

* corrected bad merge.

* remerge

* Added newer licensing info at top.

* updated zos_mount to use 2021 header
minor test tweak
merged current dev into mount

* changing includes and renaming persistent_ds to handle sub-vars

* working through pep 8 and reducing imports

* Copied backupOper routine because of import issues.

* Figured out how to return a sub-dictionary in RETURN doc section

* added two detail lines in the routine that is closest to the JSON error

* pep8 issues in the test module

* changed zos_test to use the upper of the first component of sysname
...interesting that it returns differently when called manually

* added to test import funnctionality to see if DataSet class impacts data_set implementation

* Moving imports around to get data_set functions working

* commenting out one of the routines firing data_set entry, to see if the error moves

* change a few os calls to hosts.all.os
removed to assertions that couldn't complete because of scope

* increased mount space allocation slightly
really, previous run was in a bad initial condition

* Added missing spaces in zos_mount/unmount function
Changed test cleanups from hosts.all.os to hosts.all.shell

* argh: black formatter adds a space before a colon (zos_mount: 618)
content_lines[startidx : endidx + 1] will trigger a sanity fault

* in zos_mount, a discription in the doc was confusing
in test_zos_mount_func, 2 values were not passed during an unmount that were mandatory

* changed from shell/cp to zos_copy, so that pds members could be created

* corrected over-indent in test

* test routine was using 2 different filenames

* Changed test to fix a bad removal step, and to comment out the tmp_file removal

* changed implementation of populate_tmpfile to see if I can get it to stick around

* changing tempfile to open 'w' instead of 'w+b'

* hard-coding tmpfile creation to see if that lets the file exist

* changed copy to shell/cp

* adding trace/track messages (stonehenge style), and escaping a DS/member name's parentheses

* resolving pep8 issues

* zos_data_set call had extransous character in it

* correcting print statement

* changed testfile generator to be target-local

* bad block comment

* switching back to zos_copy, now that the tmp file is on the target

* fixed block comment

* changed comments in test to tabcomment, which is what it is defined as in zos_mount

* added tracking on comment tester for last of test chain

* space after paren in a print statement

* corrected issue where backup_name was not being loaded from persistent data section

* changed backup_name mechanism to use mt_backupOper convention

* added remote_src=True for copy-back, so the file should get copied (250 of mount test)

* changed zos_copy calls to include is_binary=True...trying to reconcile encoding

* changing to using outbound temp file on ansible server, instead of target

* adding cat of original test_tmp_file_filename to check encoding.

* back up step to check copy origin

* tryping to use copy_p2uss and uss2mvs instead of zos_copy for testing

* back to cp commands: zos_copy adds a 'F', and the copy functions won't import to test

* added missing spaces to the cp commands

* trying hosts.all.copy to move the tmp file to the target systems

* propagated 2-step copy to comment routine
un-commented the file deleters

* added more output in bcb

* space after paren in new print statements

* trying copy...content instead of copy...src

* removed an extra blank line

* changed the file's flag after it landed
chtag -t -c ISO8859-1 (destfile)

* it looks like chtag is making the file readable... trying for the third display

* commented out the dataset assertion until I find the setting to make data_set work

* re-introducing zos_copy to see if the sole issue was the file flag

* propagated change back to zos_copy + chtag to the other cases.
Retesting before moving to the data_set issue

* added remote_src=True to the zos_copy

* trying remote_src=true for copy in bcb, since it worked in bnn

* Added data set removal at the finally stage of tests.
Removed comment about zos_copy in the mount module.

* In test, removed the -volumes argument

* put volumes option back in

* changed test aggregate name because of catalog glitch.

* 2-10: removed redundant entry for zos_mount
2-11 & 2-12: added an entry for zos_mount

* Changed hard-coded "IMSCN1" to "222222", which should usually be a temp volume.

* Reponse to latest code reviews.
Corrected documentation, variable naming.
Added remount test.

* added second blank link @141 for p3p8 requirement in test

* Updates to address comments in git PR

Signed-off-by: ddimatos <[email protected]>

* Addressed PR comments regarding error messages needing context.
Changed from run_command(tsocommand) to mvs_cmd.ikjeft01 calls for compatibility.

* corrections to 2 issues that failed sanity test

* corrected a bad default condition.  We changed case of mount_opts, but I forgot to uppercase the default.

* restored comment about mounting a dataset that is already mounted.

* corrected a test looking in old variable.

* corrected issue with stderr being blanked out after it was populated.

* changed commands to authorized=True

* reverted calls to (rc, stdout, stderr) = module.run_command(
                        fullumcmd, use_unsafe_shell=False
                    )
because of error messages in mvs_cmd

* changed behavior so if rc=0, any text returned goes to stdout.

* needed to handle None return in stdout

* modified test (double mount) to check stderr and stdout

* modified tests to use persistant/data_store i/s/o data_set_name

* corrected parameter error (dataa instead of data)

* changed 'tabcomment' to 'comment' in test

* updates to zos_mount: moved 'comments' to under persistant grouping
corrected test_zos_mount_func to use the new comment format
fixed text processor for comments to remove nulls embedded in file

* Pushed these changes as a commit for @RadhaV2020

Signed-off-by: ddimatos <[email protected]>

* Update copyright to 2021 for files in role samples

Signed-off-by: ddimatos <[email protected]>

* Updated copyright year to 2021 for files changed in the respective year for GID  RadhaV2020

Signed-off-by: ddimatos <[email protected]>

* changed test_zos_find_func, removing a test based on pre-existance of files.

Co-authored-by: ddimatos <[email protected]>

* Update common code used to obtain job content such it is not trucated by the unrelated record count (#283)

* Update common code used to obtain job content such it is not trucated by the unrelated record count

Signed-off-by: ddimatos <[email protected]>

* Removed unused linecount variable

Signed-off-by: ddimatos <[email protected]>

* Deprecate zos_ssh connection plugin and update zos_ping (#287)

* Update zos_ping module to support Ansible 2_9 - 2_11

Signed-off-by: ddimatos <[email protected]>

* Add copyright year update

Signed-off-by: ddimatos <[email protected]>

* Remove doc referring to the deprecated zos_ssh connection plugin

Signed-off-by: ddimatos <[email protected]>

* Remove the unsued connectionp plugin since the logic is now in zos_ping

Signed-off-by: ddimatos <[email protected]>

* Fixes PEP8 line to long error

Signed-off-by: ddimatos <[email protected]>

* Fixes PEP8 continuation line under-indented

Signed-off-by: ddimatos <[email protected]>

* Fixes PEP8 continuation line under-indented

Signed-off-by: ddimatos <[email protected]>

* Remove zos_ssh plugin from test configuration to keep from failing tests

Signed-off-by: ddimatos <[email protected]>

* Remove zos_ssh plugin references from docs since the plugin is deprecated

Signed-off-by: ddimatos <[email protected]>

* Fix closing quote

Signed-off-by: ddimatos <[email protected]>

* Updated the ping module code with some logic change on variable use and how changes are restored

Signed-off-by: ddimatos <[email protected]>

* Correct indentation

Signed-off-by: ddimatos <[email protected]>

* Temporary work around for failing test test_find_data_sets_in_volume where its expecting 5 DS yet only 1 is cataloged

Signed-off-by: ddimatos <[email protected]>

* Update zos_copy and zos_fetch to use the SSH conneciton plugin APIs, eliminate spawned SFTP process.  (#286)

* Update zos_copy module changes to support use of ssh connection plugin

Signed-off-by: ddimatos <[email protected]>

* Update zos_fetch module changes to support use of ssh connection plugin

Signed-off-by: ddimatos <[email protected]>

* Add force to module spec to correct module lint error during runtime and fix changed returning null

Signed-off-by: ddimatos <[email protected]>

* Update copy module per arg spec and sanit test ignore files

Signed-off-by: ddimatos <[email protected]>

* Update test case to correctly compare changed files

Signed-off-by: ddimatos <[email protected]>

* Update test case to correctly compare changed files

Signed-off-by: ddimatos <[email protected]>

* Add stderr to the error case

Signed-off-by: ddimatos <[email protected]>

* Update test with more verbosity

Signed-off-by: ddimatos <[email protected]>

* Update test with more verbosity

Signed-off-by: ddimatos <[email protected]>

* Update test with more verbosity

Signed-off-by: ddimatos <[email protected]>

* Print out steps of a failing test case

* Syntax error in test

* Erase the ssh_args

Signed-off-by: ddimatos <[email protected]>

* Null out ssh_args

* Add debug to config

* Adding verbosity to debug docker

* Adding verbosity to debug docker

* Reduce verbosity to avoid ssh connection setting up verbosity which causes stderr to be output

* Updates to modules verbosity messages and cleanup

Signed-off-by: ddimatos <[email protected]>

* Correct syntax for default port

Signed-off-by: ddimatos <[email protected]>

* Add displaying of the playcontext verbosity

Signed-off-by: ddimatos <[email protected]>

* Added additional logic to handle the failure than can occur when users enabled verbosity > 3

Signed-off-by: ddimatos <[email protected]>

* Updated copyright year to include 2021

Signed-off-by: ddimatos <[email protected]>

* Set the ignore_stderr=True now that testing has completed

Signed-off-by: ddimatos <[email protected]>

* Update test case to remove uneeded logic

Signed-off-by: ddimatos <[email protected]>

* test case assertion back to True

Signed-off-by: ddimatos <[email protected]>

* Add documentation about co:z for copy and fetch

Signed-off-by: ddimatos <[email protected]>

* Update failing test case test_find_data_sets_in_volume to not rely on a hard coded number of data sets

Signed-off-by: ddimatos <[email protected]>

* Update plugin module doc with missing plugins and deprecated content (#288)

* Update plugin modudle doc with missing plugins and deprecated content

Signed-off-by: ddimatos <[email protected]>

* Correct Unknown target name: zos_ping

Signed-off-by: ddimatos <[email protected]>

* Bug fix not propagating non zero return codes and under certain cases job_name is null for operator action query (#290)

* Added on command failure return details , added fix for when no job_name is part of a query

Signed-off-by: ddimatos <[email protected]>

* Added comments and cleaned up some others

Signed-off-by: ddimatos <[email protected]>

* Update the failure command, typo and changed what is returned in the message field to stdout only

Signed-off-by: ddimatos <[email protected]>

* Update failing test case test_find_data_sets_in_volume so that it does not depend on hard coded value

Signed-off-by: ddimatos <[email protected]>

* Update galaxy yml to reflect the beta and other minor changes

Signed-off-by: ddimatos <[email protected]>

* Corrected doc that had random back slashes

Signed-off-by: ddimatos <[email protected]>

* Shorted short description to avoid navigation roll over

Signed-off-by: ddimatos <[email protected]>

* Update the release notes for version 1.4.0-beta.1

Signed-off-by: ddimatos <[email protected]>

* Update ignore_sftp_stderr to simplify the explanation

Signed-off-by: ddimatos <[email protected]>

* Correct URL RST syntax

Signed-off-by: ddimatos <[email protected]>

* Fix character set URLs to the updated URL

Signed-off-by: ddimatos <[email protected]>

* Update modules to use L for URLs

Signed-off-by: ddimatos <[email protected]>

* Updated module docs to correct the sphinx improper interpretation of VASM followed by an open paren

Signed-off-by: ddimatos <[email protected]>

* New generated RST from the new zos_mount module

Signed-off-by: ddimatos <[email protected]>

* Add the updated generated RST to the release for use by unified docs and ansible-doc

Signed-off-by: ddimatos <[email protected]>

* Update sphinx copyright year to include 2021

Signed-off-by: ddimatos <[email protected]>

* Update zos_mount module description to better reflect it does more than just mount

Signed-off-by: ddimatos <[email protected]>

* Add generated RST that has been updated

Signed-off-by: ddimatos <[email protected]>

* Module updates to fix linting issues

Signed-off-by: ddimatos <[email protected]>

* Correct typo in doc

Signed-off-by: ddimatos <[email protected]>

* commit generated RST for zos_job_query.rst

Signed-off-by: ddimatos <[email protected]>

* Update release notes with more enhancements

Signed-off-by: ddimatos <[email protected]>

* Update meta to match the beta release

Signed-off-by: ddimatos <[email protected]>

* Update release notes

Signed-off-by: ddimatos <[email protected]>

* Author updates to modules

Signed-off-by: ddimatos <[email protected]>

* Add the required runtime.yml

Signed-off-by: ddimatos <[email protected]>

Co-authored-by: Rich Parker <[email protected]>
Co-authored-by: RadhaV2020 <[email protected]>

Author: 3 people

ansible.cfg

@@ -1,5 +1,5 @@
 ################################################################################
-# Copyright (c) IBM Corporation 2020
+# Copyright (c) IBM Corporation 2020, 2021
 ################################################################################
 
 ################################################################################
@@ -10,18 +10,28 @@
 #   https://github.com/ansible/ansible/blob/devel/examples/ansible.cfg
 #
 # Note:
-#   Some of the common options configured are `remote_temp` and `ansible_port`.
+#   Examples of some options often used:
 #   remote_temp - The temporary directory Ansible uses to transfer files onto
 #                  the controller. Default is `/.ansible/tmp`
 #   ansible_port - The connection port number Ansible uses to connect to the
 #                  target; configure the target if is not using default SSH
 #                  port 22
+#   debug - Toggles debug output in Ansible. This is very verbose and can
+#           hinder multiprocessing. Debug output can also include secret
+#           information despite no_log settings being enabled, which means debug
+#           mode should not be used in production. Optionally, ad-hoc you can
+#           use ANSIBLE_DEBUG=1
+#  verbosity - Sets the default verbosity, equivalent to the number of -v's
+#              passed in the command line.
+#              i.e. 0|1|2|3|4 == None|-v|-vv|-vvv|-vvvv
 ################################################################################
 
 [defaults]
 forks = 25
 # remote_tmp = /u/ansible/tmp
-# remote_port = 2022
+# remote_port = 22
+# debug = True
+# verbosity = 1
 
 [ssh_connection]
 pipelining = True

docs/files/role_sample/doc_role_sample

@@ -1,4 +1,4 @@
-# Copyright (c) IBM Corporation 2020
+# Copyright (c) IBM Corporation 2020, 2021
 
 DOCUMENTATION = r"""
 role: role_sample

docs/files/role_sample/role-doc-readme.txt

@@ -1,4 +1,4 @@
-# Copyright (c) IBM Corporation 2020
+# Copyright (c) IBM Corporation 2020, 2021
 
 ################################################################################
 # Role metadata documentation readme
@@ -25,7 +25,7 @@
 # Role metadata documentation should:
 #
 # 1) Include a copyright in the metadata at the top.
-#    Copyright (c) IBM Corporation 2020
+#    Copyright (c) IBM Corporation 2020, 2021
 #
 # 2) Include DOCUMENTATION block (See example doc_role_sample).
 #    DOCUMENTATION = """ properties """ where properties supported:

docs/files/role_sample/roles.rst

@@ -1,5 +1,5 @@
 .. ...........................................................................
-.. © Copyright IBM Corporation 2020                                          .
+.. © Copyright IBM Corporation 2020, 2021                                          .
 .. ...........................................................................
 
 Roles

docs/source/conf.py

@@ -1,5 +1,5 @@
 ##############################################################################
-# © Copyright IBM Corporation 2020                                           #
+# © Copyright IBM Corporation 2020, 2021                                           #
 ##############################################################################
 
 ##############################################################################
@@ -16,7 +16,7 @@
 ##############################################################################
 
 project = 'IBM z/OS core collection'
-copyright = '2020, IBM'
+copyright = '2020, 2021, IBM'
 author = 'IBM'
 
 # The full version, including alpha/beta/rc tags

docs/source/index.rst

@@ -1,12 +1,12 @@
 .. ...........................................................................
-.. © Copyright IBM Corporation 2020                                          .
+.. © Copyright IBM Corporation 2020, 2021                                          .
 .. ...........................................................................
 
 ========================
 IBM z/OS core collection
 ========================
 
-The **IBM z/OS core collection**, also represented as **ibm\_zos\_core**
+The **IBM z/OS core collection**, also represented as **ibm_zos_core**
 in this document, is part of the broader offering **Red Hat® Ansible
 Certified Content for IBM Z**. IBM z/OS core collection supports tasks
 such as creating data sets, submitting jobs, querying jobs,

docs/source/modules.rst

@@ -1,5 +1,5 @@
 .. ...........................................................................
-.. © Copyright IBM Corporation 2020                                          .
+.. © Copyright IBM Corporation 2020, 2021                                          .
 .. ...........................................................................
 
 Modules

docs/source/modules/zos_blockinfile.rst

@@ -16,7 +16,7 @@ zos_blockinfile -- Manage block of multi-line textual data on z/OS
 
 Synopsis
 --------
-- Manage block of multi-lines in z/OS UNIX System Services (USS) files, PS(sequential data set), PDS, PDSE, or member of a PDS or PDSE.
+- Manage block of multi-lines in z/OS UNIX System Services (USS) files, PS (sequential data set), PDS, PDSE, or member of a PDS or PDSE.
 - This module ensures a particular block of multi-line text surrounded by customizable marker lines is present in a USS file or data set, or replaces an existing block identified by the markers.
 - This is primarily useful when you want to change a block of multi-line text in a USS file or data set.
 
@@ -29,7 +29,7 @@ Parameters
 
 
 src
-  The location can be a UNIX System Services (USS) file, PS(sequential data set), member of a PDS or PDSE, PDS, PDSE.
+  The location can be a UNIX System Services (USS) file, PS (sequential data set), member of a PDS or PDSE, PDS, PDSE.
 
   The USS file must be an absolute pathname.
 
@@ -217,7 +217,7 @@ Notes
 
    All data sets are always assumed to be cataloged. If an uncataloged data set needs to be encoded, it should be cataloged first. The :ref:`zos_data_set <zos_data_set_module>` module can be used to catalog uncataloged data sets.
 
-   For supported character sets used to encode data, refer to https://ibm.github.io/z_ansible_collections_doc/reference/supplementary_content/encode.html
+   For supported character sets used to encode data, refer to the `documentation <https://ibm.github.io/z_ansible_collections_doc/ibm_zos_core/docs/source/resources/character_set.html>`_.
 
    When using 'with_*' loops be aware that if you do not set a unique mark the block will be overwritten on each iteration.
 

docs/source/modules/zos_copy.rst

@@ -60,7 +60,7 @@ content
 
   Works only when ``dest`` is a USS file, sequential data set, or a partitioned data set member.
 
-  This is for simple values; for anything complex or with formatting, use https://docs.ansible.com/ansible/latest/modules/copy_module.html
+  This is for simple values; for anything complex or with formatting, use `ansible.builtin.copy <https://docs.ansible.com/ansible/latest/modules/copy_module.html>`_
 
   If ``dest`` is a directory, then content will be copied to ``/path/to/dest/inline_copy``.
 
@@ -79,13 +79,13 @@ dest
 
   If ``src`` and ``dest`` are files and if the parent directory of ``dest`` does not exist, then the task will fail.
 
-  When the ``dest`` is an existing VSA:ref:`KSDS <KSDS_module>` or VSA:ref:`ESDS <ESDS_module>`, then source can be ESDS, KSDS or RRDS. The ``dest`` will be deleted and storage management rules will be used to determine the volume where ``dest`` will be allocated.
+  When the ``dest`` is an existing VSAM (KSDS) or VSAM (ESDS), then source can be ESDS, KSDS or RRDS. The ``dest`` will be deleted and storage management rules will be used to determine the volume where ``dest`` will be allocated.
 
-  When the ``dest`` is an existing VSA:ref:`RRDS <RRDS_module>`, then the source must be RRDS. The ``dest`` will be deleted and storage management rules will be used to determine the volume where ``dest`` will be allocated.
+  When the ``dest`` is an existing VSAM (RRDS), then the source must be RRDS. The ``dest`` will be deleted and storage management rules will be used to determine the volume where ``dest`` will be allocated.
 
-  When ``dest`` is and existing VSA:ref:`LDS <LDS_module>`, then source must be LDS. The ``dest`` will be deleted and storage management rules will be used to determine the volume where ``dest`` will be allocated.
+  When ``dest`` is and existing VSAM (LDS), then source must be LDS. The ``dest`` will be deleted and storage management rules will be used to determine the volume where ``dest`` will be allocated.
 
-  When ``dest`` is a data set, you can override storage management rules by specifying both ``volume`` and ``model_ds``.
+  When ``dest`` is a data set, you can override storage management rules by specifying both ``volume`` and other optional DS specs (type, space, record size, etc).
 
   | **required**: True
   | **type**: str
@@ -131,7 +131,9 @@ force
 
 
 ignore_sftp_stderr
-  During data transfer through sftp, the module fails if the sftp command directs any content to stderr. The user is able to override this behavior by setting this parameter to ``true``. By doing so, the module would essentially ignore the stderr stream produced by sftp and continue execution.
+  During data transfer through SFTP, the module fails if the SFTP command directs any content to stderr. The user is able to override this behavior by setting this parameter to ``true``. By doing so, the module would essentially ignore the stderr stream produced by SFTP and continue execution.
+
+  When Ansible verbosity is set to greater than 3, either through the command line interface (CLI) using **-vvvv** or through environment variables such as **verbosity = 4**, then this parameter will automatically be set to ``true``.
 
   | **required**: False
   | **type**: bool
@@ -167,17 +169,6 @@ mode
   | **type**: str
 
 
-model_ds
-  When copying a local file/directory to a non-existing PDS, PDSE or PS, specify a model data set to allocate the destination after.
-
-  If this parameter is not provided, the destination data set will be allocated based on the size of the local file/directory.
-
-  Only valid if ``src`` is a local file or directory and ``dest`` does not exist.
-
-  | **required**: False
-  | **type**: str
-
-
 remote_src
   If set to ``false``, the module searches for ``src`` at the local machine.
 
@@ -188,9 +179,11 @@ remote_src
 
 
 sftp_port
-  Indicates which port should be used to connect to the remote z/OS system to perform data transfer.
+  Configuring the SFTP port used by the :ref:`zos_copy <zos_copy_module>` module has been deprecated and will be removed in ibm.ibm_zos_core collection version 1.5.0.
+
+  Configuring the SFTP port with *sftp_port* will no longer have any effect on which port is used by this module.
 
-  If this parameter is not specified, ``ansible_port`` will be used.
+  To configure the SFTP port used for module :ref:`zos_copy <zos_copy_module>`, refer to topic `using connection plugins <https://docs.ansible.com/ansible/latest/plugins/connection.html#using-connection-plugins>`_
 
   If ``ansible_port`` is not specified, port 22 will be used.
 
@@ -231,8 +224,6 @@ validate
 volume
   If ``dest`` does not exist, specify which volume ``dest`` should be allocated to.
 
-  ``volume`` must be used with ``model_ds``, otherwise the ``volume`` value is ignored.
-
   Only valid when the destination is an MVS data set.
 
   The volume must already be present on the device.
@@ -245,6 +236,84 @@ volume
   | **type**: str
 
 
+destination_dataset
+  These are settings to use when creating the destination data set
+
+  | **required**: False
+  | **type**: dict
+
+
+  dd_type
+    Organization of the destination
+
+    | **required**: False
+    | **type**: str
+    | **default**: BASIC
+    | **choices**: KSDS, ESDS, RRDS, LDS, SEQ, PDS, PDSE, MEMBER, BASIC
+
+
+  space_primary
+    If the destination *dest* data set does not exist , this sets the primary space allocated for the data set.
+
+    The unit of space used is set using *space_type*.
+
+    | **required**: False
+    | **type**: str
+    | **default**: 5
+
+
+  space_secondary
+    If the destination *dest* data set does not exist , this sets the secondary space allocated for the data set.
+
+    The unit of space used is set using *space_type*.
+
+    | **required**: False
+    | **type**: str
+    | **default**: 3
+
+
+  space_type
+    If the destination data set does not exist, this sets the unit of measurement to use when defining primary and secondary space.
+
+    Valid units of size are ``K``, ``M``, ``G``, ``CYL``, and ``TRK``.
+
+    | **required**: False
+    | **type**: str
+    | **default**: M
+    | **choices**: K, M, G, CYL, TRK
+
+
+  record_format
+    If the destination data set does not exist, this sets the format of the data set. (e.g ``FB``)
+
+    Choices are case-insensitive.
+
+    | **required**: False
+    | **type**: str
+    | **default**: FB
+    | **choices**: FB, VB, FBA, VBA, U
+
+
+  record_length
+    The length of each record in the data set, in bytes.
+
+    For variable data sets, the length must include the 4-byte prefix area.
+
+    Defaults vary depending on format: If FB/FBA 80, if VB/VBA 137, if U 0.
+
+    | **required**: False
+    | **type**: int
+    | **default**: 80
+
+
+  block_size
+    The block size to use for the data set.
+
+    | **required**: False
+    | **type**: int
+
+
+
 
 
 Examples
@@ -296,7 +365,7 @@ Examples
          from: UTF-8
          to: IBM-037
 
-   - name: Copy a VSAM(KSDS) to a VSAM(KSDS)
+   - name: Copy a VSAM (KSDS) to a VSAM (KSDS)
      zos_copy:
        src: SAMPLE.SRC.VSAM
        dest: SAMPLE.DEST.VSAM
@@ -414,9 +483,9 @@ Notes
 
    VSAM data sets can only be copied to other VSAM data sets.
 
-   For supported character sets used to encode data, refer to https://ansible-collections.github.io/ibm_zos_core/supplementary.html#encode
+   For supported character sets used to encode data, refer to the `documentation <https://ibm.github.io/z_ansible_collections_doc/ibm_zos_core/docs/source/resources/character_set.html>`_.
 
-   :ref:`zos_copy <zos_copy_module>` uses SFTP (Secure File Transfer Protocol) for the underlying transfer protocol; Co:Z SFTP is not supported. In the case of Co:z SFTP, you can exempt the Ansible userid on ZOS from using Co:Z thus falling back to using standard SFTP.
+   :ref:`zos_copy <zos_copy_module>` uses SFTP (Secure File Transfer Protocol) for the underlying transfer protocol; Co:Z SFTP is not supported. In the case of Co:z SFTP, you can exempt the Ansible userid on z/OS from using Co:Z thus falling back to using standard SFTP.
 
 
 

docs/source/modules/zos_encode.rst

@@ -16,8 +16,8 @@ zos_encode -- Perform encoding operations.
 
 Synopsis
 --------
-- Converts the encoding of characters that are read from a UNIX System Services (USS) file or path, PS(sequential data set), PDS, PDSE, or KSDS(VSAM data set).
-- Writes the data to a UNIX System Services (USS) file or path, PS(sequential data set), PDS, PDSE, or KSDS(VSAM data set).
+- Converts the encoding of characters that are read from a UNIX System Services (USS) file or path, PS (sequential data set), PDS, PDSE, or KSDS (VSAM data set).
+- Writes the data to a UNIX System Services (USS) file or path, PS (sequential data set), PDS, PDSE, or KSDS (VSAM data set).
 
 
 
@@ -48,7 +48,7 @@ to_encoding
 
 
 src
-  The location can be a UNIX System Services (USS) file or path, PS(sequential data set), PDS, PDSE, member of a PDS or PDSE, or KSDS(VSAM data set).
+  The location can be a UNIX System Services (USS) file or path, PS (sequential data set), PDS, PDSE, member of a PDS or PDSE, or KSDS (VSAM data set).
 
   The USS path or file must be an absolute pathname.
 
@@ -61,7 +61,7 @@ src
 dest
   The location where the converted characters are output.
 
-  The destination *dest* can be a UNIX System Services (USS) file or path, PS(sequential data set), PDS, PDSE, member of a PDS or PDSE, or KSDS(VSAM data set).
+  The destination *dest* can be a UNIX System Services (USS) file or path, PS (sequential data set), PDS, PDSE, member of a PDS or PDSE, or KSDS (VSAM data set).
 
   If the length of the PDSE member name used in *dest* is greater than 8 characters, the member name will be truncated when written out.
 
@@ -235,7 +235,7 @@ Notes
 
    All data sets are always assumed to be cataloged. If an uncataloged data set needs to be encoded, it should be cataloged first.
 
-   For supported character sets used to encode data, refer to https://ansible-collections.github.io/ibm_zos_core/supplementary.html#encode
+   For supported character sets used to encode data, refer to the `documentation <https://ibm.github.io/z_ansible_collections_doc/ibm_zos_core/docs/source/resources/character_set.html>`_.