How to keep the proper format (text) of the output of a file while using zos_fetch module?

[rishinz]

Hi Team,

I have a playbook which -

1. Uses zos_job_output to filter the JESMSGLOG for a job.
2. Stores the output from zos_job_output module to a USS path in Mainframe using the COPY module.
3. Use zos_fetch to retrieve the file.
4. Print the contents of the file using a small python code.

OUTPUT OF THE BELOW MODULE IS STORED IN USS PATH USING ANSIBLE MODULE "COPY"- u/userid/files/cics_log.txt

• name: JES Job output with ddname
  zos_job_output:
  job_name: "jobname"
  owner: "owner"
  job_id: "STC*"
  ddname: "JESMSGLG"
  register: result

PYTHON CODE TO PRINT THE CONTENTS OF THE FILE:
try:
with open(file_path, 'r', encoding=' ISO-8859-1', errors='replace') as file:
contents = file.read()
print(contents)
except FileNotFoundError:
print(f"The file '{file_path}' was not found.")
except Exception as e:
print(f"An error occurred: {e}")

Below is the FETCH module:
- name: "Fetch file from USS and store in /tmp"
zos_fetch:
src: "./files/cics_log.txt"
dest: "/tmp/cics_log"
flat: true
encoding:
from: IBM-1047
to: ISO8859-1

Though I am not receiving any errors, but the file contents are displayed in ASCII (I guess) NOT TEXT as I am hoping it to be. I have attached the output from the print file-contents step.

Do you have any suggestions on how I can achieve this? get the file contents in clear text?

Thanks for you help and please let me know if you need any other information.

Thanks!
Playbook Output.txt

[rishinz]

I have tried the below code as well for zos_fetch and python read

• name: "Fetch file from USS and store in /tmp"
  zos_fetch:
  src: "./files/cics_log.txt"
  dest: "/tmp/cics_log"
  flat: true
  encoding:
  from: IBM-1047
  to: UTF-8

try:
with open(file_path, 'r', encoding='utf-8', errors='replace') as file:
contents = file.read()
print(contents)
except FileNotFoundError:
print(f"The file '{file_path}' was not found.")
except Exception as e:
print(f"An error occurred: {e}")

BUT NO LUCK..

[ketankelkar]

Would you be able to share the actual playbook/tasks? And also the environment variables/configurations you're running with (check group_vars/host_vars dirs)? Some of the steps you've mentioned seem a bit off to me (not necessarily wrong) and I'd like to understand what it is you're after.

For example, you pasted this task:

name: JES Job output with ddname
  zos_job_output:
    job_name: "KXDCUF55"
    owner: "KXDCUF"
    job_id: "STC*"
    ddname: "JESMSGLG"
  register: result

The line register: result in the task saves the job output to an ansible variable result. From there, you could leverage the ansible.builtin.debug module (docs) to print the results variable. Something like this:

- name: Print job output.
  debug:
    msg: "{{ result }}"

This will of course print the results on the controller where the playbook is run from. See the the submit_query_retrieve sample playbook (here). The sample playbook copies some JCL over to z/OS, submits the JCL, queries the job and prints the output. I've linked to the line where the job output is printed based on the job id identified earlier in the playbook.

But I see you've mentioned copying and fetching a USS file. This is the part I'm a little confused on, and where I think seeing your work flow (playbook, python) would help me get a clearer picture.

If you are trying to keep the job_output on the z/OS USS side, it may be helpful to figure out which encoding the file is in at each step in your flow. In my experience, ansible.builtin modules write to USS in UTF-8 only and leave the files untagged. On my z/VM dev system, re-tagging the file (chtag -tc ISO8859-1 file-name) allows me to cat it whereas if I don't re-tag, it prints out garbage characters similar to what you've got in that Playbook Output.txt file.

I'm suspecting that the COPY step you mentioned has copied over the file in UTF-8, and then the zos_fetch treats that UTF-8 as IBM-1047 (per your zos_fetch task's from/to encoding setting) and it's winding up treating the UTF-8 encoded file as IBM-1047 and double-converting it into the garbage chars you're getting as output.

[rishinz]

Thanks for looking into this.

The idea is to store the output in a file so it could be played with. For example, search for a particular message-id in the JESMSGLOG of a job output. Since, the output is large I am unable to just pass it as a variable to my python code so need it to be stored somewhere.

The playbook is being run as a template in AWX on Linux and the file is stored in my test zos LPAR.

How do I get over the last part you mentioned - "I'm suspecting that the COPY step you mentioned has copied over the file in UTF-8, and then the zos_fetch treats that UTF-8 as IBM-1047 (per your zos_fetch task's from/to encoding setting) and it's winding up treating the UTF-8 encoded file as IBM-1047 and double-converting it into the garbage chars you're getting as output."

Below my environment vars -
################################################################################

Configure dependency installations

################################################################################
PYZ: "/usr/lpp/IBM/cyp/v3r9/pyz"
ZOAU: "/usr/lpp/IBM/zoautil"
################################################################################

Playbook enviroment variables

################################################################################

environment_vars:
_BPXK_AUTOCVT: "ON"
ZOAU_HOME: "{{ ZOAU }}"

LIBPATH: "{{ ZOAU }}/lib:{{ PYZ }}/lib:/lib:/usr/lib:."
PATH: "{{ ZOAU }}/bin:{{ PYZ }}/bin:/bin:/usr/sbin:/var/bin"
_CEE_RUNOPTS: "FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)"
_TAG_REDIR_ERR: "txt"
_TAG_REDIR_IN: "txt"
_TAG_REDIR_OUT: "txt"
LANG: "C"

Playbook yml.txt

[ketankelkar]

Ah yes, your use case makes perfect sense, I have a tendency to overlook cases with large output, in a dev setting, I tend to primarily play with very tiny output, mainly to confirm things are working as expected.

Anyways, your environment variables look right to me, so that's not the issue.

From your playbook, it looks like you want to have the file locally on your AWX control node and that's also where the python script is run. If that's the case, might I suggest copying to the file using an ansible local_action or its equivalent delegate_to: localhost (link to docs).
The idea would be to execute the copy step locally, similar to how the python script is run.

Here are playbook tasks for getting zos facts and adding them to a local file using local_action and then delegate_to: localhost:

  tasks:
    - name: Gather zOS facts.
      zos_gather_facts:
      register: facts_output

    - name: Store facts_out variable to local file 1.
      local_action:
        module: ansible.builtin.copy
        content: "{{ facts_out }}"
        dest: "{{ playbook_dir }}/facts_out_1.txt"

    - name: Store facts_out variable to local file 2.
      ansible.builtin.copy:
        content: "{{ facts_out }}"
        dest: "{{ playbook_dir }}/facts_out_2.txt"
      delegate_to: localhost

I didn't have any issues with the copy module, but the following note does appear in the ansible.builtin.copy module docs:

If you need variable interpolation in copied files, use the ansible.builtin.template module. Using a variable in the content field will result in unpredictable output.

In that case, you'll have to create a template file, which is just a jinja formatted file. You could get pretty fancy with it and leverage any jinja features but a very simple one-liner like the following works too, just remember to match the variable name from the playbook:

"{{ facts_out }}"

The corresponding playbook task to the above file (I named the template file template_file) would be:

 - name: Store facts_out variable to local file 3.
    local_action:
      module: ansible.builtin.template
      src: "{{ playbook_dir }}/template_file"
      dest: "{{ playbook_dir }}/facts_out_3.txt"

As far as the encoding stuff goes, I'd be happy to discuss it more if you like. From what I see in your playbook, since you called copy (that is ansible.builtin.copy) and not zos_copy to add the variable to your file, the text would have been written to file in UTF-8. Anything from the ansible core community gets interpreted (read/write) in UTF-8. I'd expect changing the from-encoding portion of the zos_fetch task from IBM-1047 to UTF-8 would resolve the issue. My next idea would be to leverage zos_copy instead of the builtin.

[rishinz]

Thanks for your suggestion. Changing the encoding from IBM-1047 to UTF-8 did resolve the issue.

Now, trying out your other suggestions. Will keep you posted on the results.

Thanks again!!