Webhook body template generates wrong json

[Dees7]

Hello.
For posting vars to AWX I need to add "extra_vars" before webhook body.
by default it generate correct json. But If use custom template with {{data}} it do not resolve OrderedDict

NetBox version

v2.11.2

Python version

3.8

Steps to Reproduce

Create webhook

Content types: IPAM>IP Address
URL: http://192.168.1.100:10000/post
HTTP method: POST
Body template: {"extra_vars":{{data}}}

On 192.168.1.100:
while true; do nc -l 10000 < /dev/null ; printf '\n\n\n'; done

when I change any IP in Netbox I see:

{"extra_vars":{
   'id': 19212, 
   'url': '/api/ipam/ip-addresses/19212/', 
   'display': '10.0.7.7/24', 
   'family': OrderedDict([('value', 4), ('label', 'IPv4')]), 
   'address': '10.0.7.7/24', 
   'vrf': None, 
   'tenant': None, 
   'status': OrderedDict([('value', 'reserved'), ('label', 'Reserved')]) 
   }
}

Expected Behavior

Expected json

{"extra_vars":{
   'id': 19212, 
   'url': '/api/ipam/ip-addresses/19212/', 
   'display': '10.0.7.7/24', 
   'family': {
        'value': 4, 
        'label': 'IPv4'
   }, 
   'address': '10.0.7.7/24', 
   'vrf': None, 
   'tenant': None, 
   'status': {
        'value': 'active', 
        'label': 'Active'
     }
   }
}

Observed Behavior

It is not valid json and AWX failed.

[candlerb]

{{data}} interpolates the object as a python string.

A quick websearch shows that jinja2 has a tojson filter.

>>> from jinja2 import Template
>>> t = Template('{"extra_vars":{{data | tojson}}}')
>>> t.render(data={"bar":"baz"})
'{"extra_vars":{"bar": "baz"}}'
>>> 

That is: try {"extra_vars":{{data | tojson}}} as your body template.

[Kani999]

Using {"extra_vars":{{data | tojson}}} as Netbox Webhook Body Template does not work.
It does not execute the hook. I suppose the method tojson fails, but I cannot find any logs about it.

[candlerb]

Hmm, if data is a Netbox model instance, then it isn't directly encodable using the tojson filter:

root@netbox:~# cd /opt/netbox
root@netbox:/opt/netbox# . venv/bin/activate
(venv) root@netbox:/opt/netbox# python3 netbox/manage.py nbshell
### NetBox interactive shell (netbox)
### Python 3.6.9 | Django 3.2.5 | NetBox 2.11.9
### lsmodels() will show available models. Use help(<model>) for more info.
>>> dev = Device.objects.get(id=1)
>>> from jinja2 import Template
>>> t = Template('{"extra_vars":{{data | tojson}}}')
>>> t.render(data=dev)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/opt/netbox/venv/lib/python3.6/site-packages/jinja2/environment.py", line 1304, in render
    self.environment.handle_exception()
  File "/opt/netbox/venv/lib/python3.6/site-packages/jinja2/environment.py", line 925, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
  File "/opt/netbox/venv/lib/python3.6/site-packages/jinja2/filters.py", line 1673, in do_tojson
    return htmlsafe_json_dumps(value, dumps=dumps, **kwargs)
  File "/opt/netbox/venv/lib/python3.6/site-packages/jinja2/utils.py", line 736, in htmlsafe_json_dumps
    dumps(obj, **kwargs)
  File "/usr/lib/python3.6/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python3.6/json/encoder.py", line 180, in default
    o.__class__.__name__)
TypeError: Object of type 'Device' is not JSON serializable
>>>

Looking at the source, what it actually does is here:

    def render_body(self, context):
        """
        Render the body template, if defined. Otherwise, jump the context as a JSON object.
        """
        if self.body_template:
            return render_jinja2(self.body_template, context)
        else:
            return json.dumps(context, cls=JSONEncoder)

• which is called from here.

Possibly Netbox needs to update the jinja2 environment to give a suitable JSON dumping function. However I can't see right now what needs to be done, because even passing cls=JSONEncoder still doesn't work:

>>> from rest_framework.utils.encoders import JSONEncoder
>>> import json
>>> json.dumps({"data":dev}, cls=JSONEncoder)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/lib/python3.6/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/lib/python3.6/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.6/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/opt/netbox/venv/lib/python3.6/site-packages/rest_framework/utils/encoders.py", line 67, in default
    return super().default(obj)
  File "/usr/lib/python3.6/json/encoder.py", line 180, in default
    o.__class__.__name__)
TypeError: Object of type 'Device' is not JSON serializable

However this works:

>>> from extras.webhooks import serialize_for_webhook
>>> serialize_for_webhook(dev)

and so does this:

>>> from jinja2 import Template
>>> t = Template('{"extra_vars":{{data | tojson}}}')
>>> t.render(data=serialize_for_webhook(dev))

It's a plain dict which serializes fine. So really, we need to get to grips with exactly what the type of object "data" is that's being provided to the template, to work out what's going on.

[candlerb]

I just tested it, and it all works fine (Netbox v2.11.9)

• Create webhook
  • Name "testhook"
  • Enabled; enabled for create/update/delete
  • Select DCIM > Device
  • URL http://localhost:9999/cgi-bin/test
  • Body template {"extra_vars":{{data | tojson}}}

And start a web server on port 9999 on the netbox host:

# python -m http.server --cgi 9999

When modifying a device, it generates a POST attempt. The CGI server responds with 404, but only because I haven't actually written a "test" CGI. tcpdump shows that the body is sent successfully:

# tcpdump -i lo -nn -s0 -A tcp port 9999
...
{"extra_vars":{"asset_tag": null, "cluster": {"display": ...

If you're not seeing any webhook request being generated, check for errors using `Admin > Background Tasks", look for a non-zero number under default / Failed, and click on that number.

[Kani999]

I found the problem. I had extra quotes in the body template.

• {"extra_vars": "{{data | tojson }}"} - does not work
• {"extra_vars": {{data | tojson}}} - works

Thanks for your help

[Dees7]

Indeed it works well! Thanks.
For my case:
{"extra_vars": {"event": "{{ event }}","timestamp": "{{ timestamp }}","model": "{{ model }}","username": "{{ username }}","request_id": "{{ request_id }}","data": {{ data | tojson }},"snapshots": {{ snapshots | tojson }}}}