|
| 1 | +.. _working_with_zos: |
| 2 | + |
| 3 | + |
| 4 | +Managing z/OS UNIX hosts with Ansible |
| 5 | +===================================== |
| 6 | + |
| 7 | + |
| 8 | +Ansible can connect to `IBM z/OS UNIX System Services <https://www.ibm.com/docs/en/zos/latest?topic=descriptions-zos-unix-system-services>`_ to bring your Ansible Automation strategy to IBM z/OS. |
| 9 | +This enables development and operations automation on IBM Z through a seamless, unified workflow orchestration with |
| 10 | +configuration management, provisioning, and application deployment with Ansible. |
| 11 | + |
| 12 | + |
| 13 | +Ansible and z/OS UNIX System Services |
| 14 | +------------------------------------- |
| 15 | +UNIX System Services can support the required dependencies for an Ansible managed node including running Python and |
| 16 | +spawning interactive shell processes through SSH connections. |
| 17 | +Ansible can target UNIX System Services nodes to modify files, directories, and so on, through ``ansible.builtin`` modules. |
| 18 | +Further, anything that one can do by typing command(s) into the UNIX System Services shell can be captured |
| 19 | +and automated in an Ansible playbook. |
| 20 | + |
| 21 | + |
| 22 | +The z/OS landscape |
| 23 | +------------------ |
| 24 | +While most systems process files in two modes - binary or text encoded in UTF-8, |
| 25 | +IBM z/OS including UNIX System Services features an additional third mode - text encoded in EBCDIC. |
| 26 | +Ansible has provisions to handle binary data and UTF-8 encoded textual data, but not EBCDIC encoded data. |
| 27 | +This is not necessarily a limitation, it simply requires additional tasks that convert files to and from their original encodings. |
| 28 | +It is up to the Ansible user managing z/OS UNIX nodes to understand the nature of the files in their automation. |
| 29 | + |
| 30 | +The type (binary or text) and encoding of files can be stored in file "tags". |
| 31 | +File tags is a z/OS UNIX System Services concept (part of Enhanced ASCII) designed to distinguish binary |
| 32 | +files from UTF-8 encoded text files and EBCDIC-encoded text files. |
| 33 | + |
| 34 | +Default behavior for an un-tagged file or stream is determined by the program, for example, |
| 35 | +`IBM Open Enterprise SDK for Python <https://www.ibm.com/products/open-enterprise-python-zos>`__ defaults to the UTF-8 encoding. |
| 36 | + |
| 37 | +Ansible modules will not read or recognize file tags. It is up to the user to determine the nature of remote data and tag it appropriately. |
| 38 | +Data sent to remote z/OS UNIX hosts through Ansible is, by default, encoded in UTF-8 and not tagged. |
| 39 | +Tagging a file is achievable with an additional task using the :ansplugin:`ansible.builtin.command#module` module. |
| 40 | + |
| 41 | +.. code-block:: yaml |
| 42 | +
|
| 43 | + - name: Tag my_file.txt as UTF-8. |
| 44 | + ansible.builtin.command: chtag -tc iso8859-1 my_file.txt |
| 45 | +
|
| 46 | +
|
| 47 | +The `z/OS shell <https://www.ibm.com/docs/en/zos/latest?topic=shells-introduction-zos>`_ available on |
| 48 | +z/OS UNIX System Services defaults to an EBCDIC encoding for un-tagged data streams. |
| 49 | +Ansible sends untagged UTF-8 encoded textual data to the z/OS shell which expects untagged data to be encoded in EBCDIC. |
| 50 | +This mismatch in data encodings can be resolved by setting the ``PYTHONSTDINENCODING`` environment variable, |
| 51 | +which causes the pipe used by Python to be tagged with the specified encoding. |
| 52 | +File and pipe tags can be used with automatic conversion between ASCII and EBCDIC, but only programs on |
| 53 | +z/OS UNIX which are aware of tags will use them. |
| 54 | + |
| 55 | + |
| 56 | +Using ``ansible.builtin`` modules with z/OS UNIX |
| 57 | +------------------------------------------------ |
| 58 | + |
| 59 | +The ``ansible.builtin`` modules operate under the assumption that all textual data (files and pipes/streams) is UTF-8 encoded. |
| 60 | +On z/OS, since textual data (file or stream) is sometimes encoded in EBCDIC and sometimes in UTF-8, special care must be taken to identify the correct encoding of target data. |
| 61 | + |
| 62 | +Here are some notes / pro-tips when using the ``ansible.builtin`` modules with z/OS UNIX. This is by no means a comprehensive list. |
| 63 | +Before using any Ansible modules, you must first :ref:`configure_zos_remote_environment`. |
| 64 | + |
| 65 | +* :ansplugin:`ansible.builtin.command#module` / :ansplugin:`ansible.builtin.shell#module` |
| 66 | + The command and shell modules are excellent for automating tasks for which command line solutions already exist. |
| 67 | + The thing to keep in mind when using these modules is that depending on the system configuration, the z/OS shell (``/bin/sh``) may return output in EBCDIC. |
| 68 | + The LE environment variable configurations will correctly convert streams if they are tagged and return readable output to Ansible. |
| 69 | + However, some command line programs may return output in UTF-8 and not tag the pipe. |
| 70 | + In this case, the autoconversion may incorrectly assume output is in EBCDIC and attempt to convert it and yield unreadable data. |
| 71 | + If the source encoding is known, you can use the :ansplugin:`ansible.builtin.shell#module` module's capability to chain commands together through pipes, |
| 72 | + and pipe the output to ``iconv``. In this example, you may need to select other encodings for the 'to' and 'from' that represent your file encodings. |
| 73 | + |
| 74 | + .. code-block:: yaml |
| 75 | +
|
| 76 | + ansible.builtin.shell: "some_pgm | iconv -f ibm-1047 -t iso8859-1" |
| 77 | +
|
| 78 | +
|
| 79 | +* :ansplugin:`ansible.builtin.raw#module` |
| 80 | + The raw module, by design, ignores all remote environment settings. However, z/OS UNIX System Services managed nodes require some base configurations. |
| 81 | + To use this module with UNIX System Services, configure the minimum environment variables as a chain of export statements before the desired command. |
| 82 | + |
| 83 | + .. code-block:: yaml |
| 84 | +
|
| 85 | + ansible.builtin.raw: | |
| 86 | + export _BPXK_AUTOCVT: "ON" ; |
| 87 | + export _CEE_RUNOPTS: "FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)" ; |
| 88 | + export _TAG_REDIR_ERR: "txt" ; |
| 89 | + export _TAG_REDIR_IN: "txt" ; |
| 90 | + export _TAG_REDIR_OUT: "txt" ; |
| 91 | + echo "hello world!" |
| 92 | +
|
| 93 | + Alternatively, consider using the :ansplugin:`ansible.builtin.command#module` or :ansplugin:`ansible.builtin.shell#module` modules mentioned above, |
| 94 | + which set up the configured remote environment for each task. |
| 95 | + |
| 96 | + |
| 97 | +* :ansplugin:`ansible.builtin.copy#module` / :ansplugin:`ansible.builtin.fetch#module` |
| 98 | + The ``ansible.builtin`` modules will NOT automatically tag files, nor will existing file tags be honored nor preserved. |
| 99 | + You can treat files as binaries when running copy/fetch operations, there is no issue in terms of data integrity, |
| 100 | + but remember to restore the file tag once the file is returned to z/OS UNIX, as tags are not preserved. Use the command module |
| 101 | + to set the file tag: |
| 102 | + |
| 103 | + .. code-block:: yaml |
| 104 | +
|
| 105 | + - name: Tag my_file.txt as UTF-8. |
| 106 | + ansible.builtin.command: chtag -tc iso8859-1 my_file.txt |
| 107 | +
|
| 108 | +* :ansplugin:`ansible.builtin.blockinfile#module` / :ansplugin:`ansible.builtin.lineinfile#module` |
| 109 | + These modules process all data in UTF-8. Ensure target files are UTF-8 encoded beforehand and re-tag the files afterwards. |
| 110 | + |
| 111 | +* :ansplugin:`ansible.builtin.script#module` |
| 112 | + The built in script module copies a local script file to a temp file on the remote target and runs it. |
| 113 | + The issue that z/OS UNIX System Services targets run into is that when the underlying z/OS shell attempts to read |
| 114 | + the script file, since the file does not get tagged as UTF-8 text, the shell assumes that the file is encoded in EBCDIC, |
| 115 | + and fails to correctly read or run the script. |
| 116 | + One work-around is to manually copy local files to managed nodes (:ansplugin:`ansible.builtin.copy#module` ) and convert or tag files (with the :ansplugin:`ansible.builtin.command#module` module). |
| 117 | + With this work-around, some of the conveniences of the script module are lost, such as automatically cleaning up the script file once it is run, |
| 118 | + but it is trivial to perform those steps as additional playbook tasks. |
| 119 | + |
| 120 | + .. code-block:: yaml |
| 121 | +
|
| 122 | + - name: Copy local script file to remote node. |
| 123 | + ansible.builtin.copy: |
| 124 | + src: "{{ playbook_dir }}/local/scripts/sample.sh" |
| 125 | + dest: /u/ibmuser/scripts/ |
| 126 | +
|
| 127 | + - name: Tag remote script file. |
| 128 | + ansible.builtin.command: "chtag -tc ISO8859-1 /u/ibmuser/scripts/sample.sh" |
| 129 | +
|
| 130 | + - name: Run script. |
| 131 | + ansible.builtin.command: "/u/ibmuser/scripts/sample.sh" |
| 132 | +
|
| 133 | + Another work-around is to store local script files in EBCDIC. |
| 134 | + They may be unreadable on the ansible control node, but they will copy correctly to z/OS UNIX System Services targets in EBCDIC, |
| 135 | + and the script will run. This approach takes advantage of the built-in conveniences of the script module, |
| 136 | + but managing unreadable EBCDIC files locally makes maintaining those script files more difficult. |
| 137 | + |
| 138 | +.. _configure_zos_remote_environment: |
| 139 | + |
| 140 | +Configure the remote environment |
| 141 | +-------------------------------- |
| 142 | + |
| 143 | +Certain Language Environment (LE) configurations enable automatic encoding conversion and automatic file tagging functionality |
| 144 | +required by Python on z/OS UNIX systems (`IBM Open Enterprise SDK for Python <https://www.ibm.com/products/open-enterprise-python-zos>`_ ). |
| 145 | + |
| 146 | +Include the following configurations when setting the remote environment for any z/OS UNIX managed nodes: |
| 147 | + |
| 148 | +.. code-block:: yaml |
| 149 | +
|
| 150 | + _BPXK_AUTOCVT: "ON" |
| 151 | + _CEE_RUNOPTS: "FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)" |
| 152 | +
|
| 153 | + _TAG_REDIR_ERR: "txt" |
| 154 | + _TAG_REDIR_IN: "txt" |
| 155 | + _TAG_REDIR_OUT: "txt" |
| 156 | +
|
| 157 | +
|
| 158 | +Ansible can be configured with remote environment variables in these options: |
| 159 | + |
| 160 | + |
| 161 | + * inventory - inventory.yml, group_vars/all.yml, or host_vars/all.yml |
| 162 | + * playbook - ``environment`` variable at top of playbook. |
| 163 | + * block or task - ``environment`` key word. |
| 164 | + |
| 165 | +For more details, see :ref:`playbooks_environment`. |
| 166 | + |
| 167 | +Configure the remote Python interpreter |
| 168 | +--------------------------------------- |
| 169 | + |
| 170 | +Ansible requires a Python interpreter to run most modules on the remote host, and it checks for Python at the 'default' path ``/usr/bin/python``. |
| 171 | + |
| 172 | +On z/OS UNIX, the Python3 interpreter (from `IBM Open Enterprise SDK for Python <https://www.ibm.com/products/open-enterprise-python-zos>`_) |
| 173 | +is often installed to a different path, typically something like: ``/usr/lpp/cyp/v3r12/pyz``. |
| 174 | + |
| 175 | +The path to the Python interpreter can be configured with the Ansible inventory variable ``ansible_python_interpreter``. |
| 176 | +For example: |
| 177 | + |
| 178 | +.. code-block:: ini |
| 179 | +
|
| 180 | + zos1 ansible_python_interpreter:/usr/lpp/cyp/v3r12/pyz |
| 181 | +
|
| 182 | +When the path to the Python interpreter is not found in the default location on the target host, |
| 183 | +an error containing the following message may result: ``/usr/bin/python: FSUM7351 not found`` |
| 184 | + |
| 185 | +For more details, see: :ref:`python_interpreters`. |
| 186 | + |
| 187 | +Configure the remote shell |
| 188 | +-------------------------- |
| 189 | +The z/OS UNIX System Services managed node includes several shells. |
| 190 | +Currently the only supported shell is the z/OS Shell located in path ``/bin/sh``. |
| 191 | +To configure which shell the Ansible control node uses on the target node, set inventory variable |
| 192 | +:ref:`ansible_shell_executable<ansible_shell_executable>`. For example: |
| 193 | + |
| 194 | +.. code-block:: ini |
| 195 | +
|
| 196 | + zos1 ansible_shell_executable=/bin/sh |
| 197 | +
|
| 198 | +Enable Ansible pipelining |
| 199 | +------------------------- |
| 200 | +Enable :ref:`ANSIBLE_PIPELINING` in the ansible.cfg file. |
| 201 | + |
| 202 | +When Ansible pipelining is enabled, Ansible passes any module code to the remote target node |
| 203 | +through Python's stdin pipe and runs it in all in a single call rather than copying data to temporary files first and then reading from those files. |
| 204 | +For more details on pipelining, see: :ref:`flow_pipelining`. |
| 205 | + |
| 206 | +Enabling this behavior is encouraged because Python will tag its pipes with the proper encoding, so there is less chance of encountering encoding errors. |
| 207 | +Further, using Python stdin pipes is more performant than file I/O. |
| 208 | + |
| 209 | + |
| 210 | +Include the following in the environment for any tasks performed on z/OS UNIX managed nodes. |
| 211 | + |
| 212 | +.. code-block:: yaml |
| 213 | +
|
| 214 | + PYTHONSTDINENCODING: "cp1047" |
| 215 | +
|
| 216 | +When Ansible pipelining is enabled but the ``PYTHONSTDINENCODING`` property is not correctly set, the following error may result. |
| 217 | +Note, the hex ``'\x81'`` below may vary depending on the source causing the error: |
| 218 | + |
| 219 | +.. code-block:: text |
| 220 | +
|
| 221 | + SyntaxError: Non-UTF-8 code starting with '\\x81' in file <stdin> on line 1, but no encoding declared; see https://peps.python.org/pep-0263/ for details |
| 222 | +
|
| 223 | +
|
| 224 | +Unreadable characters |
| 225 | +--------------------- |
| 226 | + |
| 227 | +Seeing unreadable characters in playbook output is most typically an EBCDIC encoding mix up. |
| 228 | +Double check that the remote environment is set up properly. |
| 229 | +Also check the expected file encodings, both on the remote node and the control node. |
| 230 | +``ansible.builtin`` modules will assume all textual data is UTF-8 encoded, while z/OS UNIX may be using EBCDIC. |
| 231 | +On many z/OS UNIX systems, the default encoding for untagged files is EBCDIC. |
| 232 | +This variation in default settings can easily lead to data being misinterpreted with the wrong encoding, |
| 233 | +whether that is failing to auto convert EBCDIC to UTF-8 or erroneously attempting to convert data that is already in UTF-8. |
| 234 | + |
| 235 | +.. _zos_as_control_node: |
| 236 | + |
| 237 | +Using z/OS as a control node |
| 238 | +---------------------------- |
| 239 | + |
| 240 | +The z/OS operating system currently cannot be configured to run as an Ansible control node. |
| 241 | +z/OS UNIX System Services interface also cannot be configured to run as an Ansible control node, despite being POSIX-compliant. |
| 242 | + |
| 243 | +There are options available on the IBM Z platform to use it as a control node: |
| 244 | + |
| 245 | +* IBM z/OS Container Extensions (zCX) |
| 246 | +* Red Hat OpenShift on IBM zSystems and LinuxONE |
| 247 | +* Linux on IBM Z |
0 commit comments