Unable to create STG with ISE 3.1 Patch 0 or 3.1 with Patch 3

[RyanMBess]

I'm unable to create an STG with ansible. I'm running as superadmin. Below is the error. Others work just fine but the STG for me doesn't work. Thoughtgs?

ryan@ANSIBLE-ISE:~/ansible_ise/create_stg$ ansible-playbook -i inventory.txt playbook_createstg.yml -vvv
ansible-playbook [core 2.13.3]
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/ryan/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/ryan/.local/lib/python3.10/site-packages/ansible
ansible collection location = /home/ryan/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/bin/ansible-playbook
python version = 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0]
jinja version = 3.1.2
libyaml = True
Using /etc/ansible/ansible.cfg as config file
host_list declined parsing /home/ryan/ansible_ise/create_stg/inventory.txt as it did not pass its verify_file() method
script declined parsing /home/ryan/ansible_ise/create_stg/inventory.txt as it did not pass its verify_file() method
auto declined parsing /home/ryan/ansible_ise/create_stg/inventory.txt as it did not pass its verify_file() method
yaml declined parsing /home/ryan/ansible_ise/create_stg/inventory.txt as it did not pass its verify_file() method
Parsed /home/ryan/ansible_ise/create_stg/inventory.txt inventory source with ini plugin
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: playbook_createstg.yml *******************************************************************************************************************************************************************************************************************
1 plays in playbook_createstg.yml
Read vars_file 'credentials.yml'
Read vars_file 'credentials.yml'
Read vars_file 'credentials.yml'

PLAY [Create STG] **********************************************************************************************************************************************************************************************************************************
Read vars_file 'credentials.yml'
META: ran handlers
Read vars_file 'credentials.yml'

TASK [Create STG] **********************************************************************************************************************************************************************************************************************************
task path: /home/ryan/ansible_ise/create_stg/playbook_createstg.yml:8
The full traceback is:
Traceback (most recent call last):
File "/home/ryan/.local/lib/python3.10/site-packages/ciscoisesdk/restsession.py", line 523, in request
check_response_code(response, erc,
File "/home/ryan/.local/lib/python3.10/site-packages/ciscoisesdk/misc.py", line 62, in check_response_code
raise ApiError(response, **kwargs)
ciscoisesdk.exceptions.ApiError: [403] - The request is understood, but it has been refused or access is not allowed.
Check ApiError.additional_data for more info.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/ryan/.ansible/collections/ansible_collections/cisco/ise/plugins/plugin_utils/ise.py", line 207, in exec
response = func(**params)
File "/home/ryan/.local/lib/python3.10/site-packages/ciscoisesdk/api/v3_1_patch_1/security_groups.py", line 856, in create_security_group
_api_response = self._session.post(endpoint_full_url, params=_params,
File "/home/ryan/.local/lib/python3.10/site-packages/ciscoisesdk/restsession.py", line 643, in post
response = self.request('POST', url, erc, 0, params=params,
File "/home/ryan/.local/lib/python3.10/site-packages/ciscoisesdk/restsession.py", line 546, in request
return self.request(method, url, erc, 1, **kwargs)
File "/home/ryan/.local/lib/python3.10/site-packages/ciscoisesdk/restsession.py", line 523, in request
check_response_code(response, erc,
File "/home/ryan/.local/lib/python3.10/site-packages/ciscoisesdk/misc.py", line 62, in check_response_code
raise ApiError(response, **kwargs)
ciscoisesdk.exceptions.ApiError: [403] - The request is understood, but it has been refused or access is not allowed.
Check ApiError.additional_data for more info.

fatal: [10.10.10.20]: FAILED! => {
"changed": false,
"msg": "An error occured when executing operation. The error was: [403] - The request is understood, but it has been refused or access is not allowed.\nCheck ApiError.additional_data for more info.\n"
}

PLAY RECAP *****************************************************************************************************************************************************************************************************************************************
10.10.10.20 : ok=0 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0

[bvargasre]

Hi @RyanMBess
could you add the variable ise_debug and set it to true, and send me the api Request and Response provided by ise_debug.

- name: Create SGT
  cisco.ise.sgt:
    ise_hostname: "{{ise_hostname}}"
    ise_username: "{{ise_username}}"
    ise_password: "{{ise_password}}"
    ise_verify: "{{ise_verify}}"
    ise_debug: "{{ise_debug}}" #true
    ...

also send me the playbook you used for the creation of the sgt

[RyanMBess]

Hey @bvargasre
Will need to reset my lab as i was able to figured out the issue (will get you what you're asking for tomorrow). I have 2 matrix's in my home lab. One called Production and one call FailBack. Once i removed the Failbox matrix AND set ISE to not allow multiple Matrix's it worked. I had been communicating to another user over github on this topic and once i mentioned multiple matrix he seemed to know there's some issue that is around multiple matrix. Are you aware of this matrix issue?

[RyanMBess]

Before i reset my lab i went in and attempted to re-run the same stg playbook. This should result in no changes being made as ansbile is idempotent. However this resulted in an error. This seems to only happen with the STG playbook. I had previously thought this idenpotetcy issue effected all ISE playbooks but as best i can tell only exists in the STG playbook. I will now reset my lab to get you the data you requested but wanted to let you know there is still an issue with the STG playbook not being idenpotent. Attached is the playbook being ran.

ryan@ANSIBLE-ISE:~/ansible_ise/create_stg$ ansible-playbook -i inventory.txt playbook_createstg.yaml -vvv
ansible-playbook [core 2.13.3]
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/ryan/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/ryan/.local/lib/python3.10/site-packages/ansible
ansible collection location = /home/ryan/.ansible/collections:/usr/share/ansible/collections
executable location = /home/ryan/.local/bin/ansible-playbook
python version = 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0]
jinja version = 3.1.2
libyaml = True
Using /etc/ansible/ansible.cfg as config file
host_list declined parsing /home/ryan/ansible_ise/create_stg/inventory.txt as it did not pass its verify_file() method
script declined parsing /home/ryan/ansible_ise/create_stg/inventory.txt as it did not pass its verify_file() method
auto declined parsing /home/ryan/ansible_ise/create_stg/inventory.txt as it did not pass its verify_file() method
yaml declined parsing /home/ryan/ansible_ise/create_stg/inventory.txt as it did not pass its verify_file() method
Parsed /home/ryan/ansible_ise/create_stg/inventory.txt inventory source with ini plugin
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: playbook_createstg.yaml **************************************************
1 plays in playbook_createstg.yaml
Read vars_file 'credentials.yml'
Read vars_file 'credentials.yml'
Read vars_file 'credentials.yml'

PLAY [Create STG] ******************************************************************
Read vars_file 'credentials.yml'
META: ran handlers
Read vars_file 'credentials.yml'

TASK [Create STG] ******************************************************************
task path: /home/ryan/ansible_ise/create_stg/playbook_createstg.yaml:8
The full traceback is:
Traceback (most recent call last):
File "/home/ryan/.ansible/collections/ansible_collections/cisco/ise/plugins/plugin_utils/ise.py", line 207, in exec
response = func(**params)
File "/home/ryan/.local/lib/python3.10/site-packages/ciscoisesdk/api/v3_1_patch_1/security_groups.py", line 334, in update_security_group_by_id
_api_response = self._session.put(endpoint_full_url, params=_params,
File "/home/ryan/.local/lib/python3.10/site-packages/ciscoisesdk/restsession.py", line 678, in put
response = self.request('PUT', url, erc, 0, params=params,
File "/home/ryan/.local/lib/python3.10/site-packages/ciscoisesdk/restsession.py", line 523, in request
check_response_code(response, erc,
File "/home/ryan/.local/lib/python3.10/site-packages/ciscoisesdk/misc.py", line 62, in check_response_code
raise ApiError(response, **kwargs)
ciscoisesdk.exceptions.ApiError: [400] - Validation Error - Illegal values: [illeagal sgt value: sgt value cannot be changed in 'auto-generated' mode]
{
"ERSResponse" : {
"operation" : "PUT-update-sgt",
"messages" : [ {
"title" : "Validation Error - Illegal values: [illeagal sgt value: sgt value cannot be changed in 'auto-generated' mode]",
"type" : "ERROR",
"code" : "Application resource validation exception"
} ],
"link" : {
"rel" : "related",
"href" : "https://10.10.10.20/ers/config/sgt/450e29c5-3c3e-4ddc-bc10-7eb759442da4",
"type" : "application/xml"
}
}
}
fatal: [10.10.10.20]: FAILED! => {
"changed": false,
"msg": "An error occured when executing operation. The error was: [400] - Validation Error - Illegal values: [illeagal sgt value: sgt value cannot be changed in 'auto-generated' mode]\n{\n "ERSResponse" : {\n "operation" : "PUT-update-sgt",\n "messages" : [ {\n "title" : "Validation Error - Illegal values: [illeagal sgt value: sgt value cannot be changed in 'auto-generated' mode]",\n "type" : "ERROR",\n "code" : "Application resource validation exception"\n } ],\n "link" : {\n "rel" : "related",\n "href" : "https://10.10.10.20/ers/config/sgt/450e29c5-3c3e-4ddc-bc10-7eb759442da4\",\n "type" : "application/xml"\n }\n }\n}"
}
playbook_createstg.txt

PLAY RECAP *************************************************************************
10.10.10.20 : ok=0 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0

ryan@ANSIBLE-ISE:~/ansible_ise/create_stg$

[RyanMBess]

I set ise_debug set to "True" in the credential file then re-enabled "Multiple Matrices" and ran the attached playbook_createstg_error file. This resulted in the ISEError.txt file. Please let me know if you would like any additional info.

ISEError.txt
playbook_createstg_error.txt

[bvargasre]

Hi @RyanMBess could you pass the following info

ansible --version
pip list | grep ciscoisesdk
ansible-galaxy collection list  | grep "cisco.ise"

[RyanMBess]

Data requested attached. Are you saying that we should be able to create SGT's via the API when you have multiple matrices?

[bvargasre]

Hi @RyanMBess , ansible-ise has been updated to version 2.5.3, so try this new version and tell me if it works.

With respect to the idempotency problem this is because a value is being used to be auto generated value: -1 then when the playbook is re-ran it will detect changes since the value of the value field is not -1 but it will be another one that is auto generated, and when trying to update it to -1 it generates this problem.

The error was: [400] - Validation Error - Illegal values: [illeagal sgt value: sgt value cannot be changed in 'auto-generated' mode].

To avoid this behavior in v2.5.3 the value field will be ignored when sgt exists and value has the value of -1 so it does not detect changes.
Or simply after running the playbook for the first time check what was the value given to value and change it in the playbook to the new value.

- name: Create STG
    cisco.ise.sgt:
        ise_hostname: "{{ ise_hostname }}"
        ise_username: "{{ ise_username }}"
        ise_password: "{{ ise_password }}"
        ise_verify: "{{ ise_verify }}"
        state: present
        name: teststg
        value: -1
        description: teststg description
    register: result

Regarding the 403 error and the API behavior when you have multiple matrices. A flag will be raised to cisco, because we only provide support for ISE libraries for python, ansible, go and terraform.
And this seems to be more of an API problem

[RyanMBess]

I can confirm that with 2.5.3 I'm now able to run a single playbook that has all my SGT's in it, and should it need to re-create one it will. My issue now is that should it recreate a previously created SGT is that the ID is now new along with TAG value. I'm not sure what downstream impact this would have. It's like ansible is not storing this data when the SGT was first created so it can ensure that should it need to recreate the SGT, it would be 100% identical.

[RyanMBess]

Or if you update the name of an SGT, it creates a new one. When the object is created Ansible should store the ID of the SGT so it doesn't create a new SGT but rather update the name of the existing SGT.

[1homas]

Ryan,

I'm glad you got it working

I don't understand this latest issue:

My issue now is that should it recreate a previously created SGT is that the ID is now new along with TAG value. I'm not sure what downstream impact this would have. It's like ansible is not storing this data when the SGT was first created so it can ensure that should it need to recreate the SGT, it would be 100% identical.

When I run your playbook below multiple time it is idempotent.

Are you saying that if you create and delete and create an SGT with the same name or value via Ansible (or REST API) that the id (which is a UUID format like 06e068cd-028f-44cb-bd2c-c1c3dc528764) of the resource may be different? Yes, it can and probably will be. There is never a guarantee that id will be the same so you should not hardcode the id. The ID is an internal object reference created by ISE and completely independent of the SGT name or value.

Or if you update the name of an SGT, it creates a new one. When the object is created Ansible should store the ID of the SGT so it doesn't create a new SGT but rather update the name of the existing SGT.

I don't know how you are updating the name but when I do it, the ID does not change:

  - name: Create SGT
    cisco.ise.sgt:
      ise_hostname: "{{ ansible_host }}"
      ise_username: "{{ ise_username }}"
      ise_password: "{{ ise_password }}"
      ise_verify:   "{{ ise_verify }}"
      ise_debug:    "{{ ise_debug }}"
      state: present
      name: teststg3
      value: 100
      description: teststg3 description
    register: sgt_create

  - name: Rename SGT
    cisco.ise.sgt:
      ise_hostname: "{{ ansible_host }}"
      ise_username: "{{ ise_username }}"
      ise_password: "{{ ise_password }}"
      ise_verify:   "{{ ise_verify }}"
      ise_debug:    "{{ ise_debug }}"
      state: present
      id: "{{ sgt_create.ise_response.id }}"
      name: teststg3_renamed
      value: "{{ sgt_create.ise_response.value }}"
    register: sgt_rename

  - ansible.builtin.debug:
      msg:
      - "SGT ID Before: {{ sgt_create.ise_response.id }}"
      - "SGT ID After:  {{ sgt_rename.ise_response.id }}"

  - name: Assert SGT ID did not change during rename
    ansible.builtin.assert:
      that: sgt_create.ise_response.id == sgt_rename.ise_response.id

Here is the output:

TASK [Create SGT] *********************************************************************************************
changed: [ise]

TASK [Rename SGT] *********************************************************************************************
changed: [ise]

TASK [ansible.builtin.debug] **********************************************************************************
ok: [ise] =>
  msg:
  - 'SGT ID Before: 400f8aa3-3da4-4fe7-81ef-6208ae1bc099'
  - 'SGT ID After:  400f8aa3-3da4-4fe7-81ef-6208ae1bc099'

TASK [Assert SGT ID did not change during rename] *************************************************************
ok: [ise] => changed=false
  msg: All assertions passed

[RyanMBess]

Hi Thomas,
We are embarking on our infrastructure as code journey for our on-premises gear. As part of this we are looking to have single documents that define the state of the the infrastructure (ISE in this case) and should any config in ISE deviate from the document the tool (Ansible in this case) would set the infrastructure back to match the document. Should i want to rename an SGT, i simply go into the document, update the name, and the tool should be smart enough to know I'm not telling it to create a NEW SGT but should update the existing one. When we created the SGT's originally, ISE spits back the ID. Ansible should capture that ID and store it for future use (such as this example). It should do this for all things it creates in ISE. Having to have multiple playbooks to define the state of the infrastructure doesn't scale.

Example: I have the playbook to create all my SGT's. Now if i want to update them i have to have another playbook to get the SGT ID and then another playbook to update the config (sure both of these tasks can be in a single playbook). Then i have to go back into my original playbook that created all the SGT's and update it with the new info so it doesn't create NEW SGT's the next time it runs. Some may ask why would you rerun the playbook that created the SGTs. The reason is we want our documents as the source of truth so if any human went into ISE and changed anything about the SGT, it would be corrected the next time the playbook ran.

Ansible with this module is very good at creating items THE FIRST TIME. Maintaining that item throughout it's life cycle via infrastructure as Code, I can't see this happening with this tool/modules. Thus, all were doing is using scripts to create items in ISE. I'm struggling to understand how this is more efficient than manually creating them via the GUI. Sure, if I had a use case to make 1000 SGT's you bet I'd use this tool. That's the only use case i see. Is this the use case for this module?

I believe that Ryan S. @ cisco is working with you to get a meeting between you and I so we can discuss this in depth. I'm looking forward to that conversation. Hope this post helps shed some light on what we're trying to accomplish with Infrastructure as Code.