urequests response has no headers

[GC-RnD]

esp32-20220618-v1.19.1.bin
Trying to return headers from w3schools...
https://www.w3schools.com/python/showpython.asp?filename=demo_requests_head

import urequests as requests
import ujson as json
r = requests.get('https://www.w3schools.com/python/demopage.php')
print('r = ', r)
print('r.content = ', r.content)
print('r.text = ', r.text)
print('r.status_code = ', r.status_code)
print('r.headers = ', r.headers)
r.close()

Responce....

r =  <Response object at 3ffe7040>
r.content =  b''
r.text =
r.status_code =  200
♦Traceback (most recent call last):
  File "<stdin>", line 71, in <module>
AttributeError: 'Response' object has no attribute 'headers'
♦>

This is a simple call, what am I missing??
got a r.status_code = 200

Also looked at mrequests with same outcome...

Using Thunder Client worked...

Add code... formatting hard to read.

(Edit by @jimmo to add code formatting)

[scruss]

urequests doesn't have that method:

MicroPython v1.19.1 on 2022-10-14; Raspberry Pi Pico W with RP2040
Type "help()" for more information.
>>> import urequests as requests
>>> r = requests.get('https://www.w3schools.com/python/demopage.php')
>>> help(r)
object <Response object at 20013c20> is of type Response
  text -- <property>
  __init__ -- <function __init__ at 0x2000ae50>
  __qualname__ -- Response
  content -- <property>
  close -- <function close at 0x2000ae40>
  json -- <function json at 0x2000aed0>
  __module__ -- urequests

Don't forget the micro in MicroPython … everything's a subset.

[jimmo]

urequests doesn't have that method:

@scruss Support for parsing headers was added since MicroPython v1.19.1 was released.

It doesn't show up in help() though -- same as status_code and reason it's assigned after the Response object is created.

@GC-RnD Probably your best bet is to update to a nightly/unstable build for ESP32 (which will include the latest version of urequests).

With the latest version, this works for me (on ESP32, Pico W and Linux):

>>> import urequests as requests
>>> r = requests.get('https://www.w3schools.com/python/demopage.php')
>>> r.headers
{'Server': 'ECS (agb/A43F)', 'Last-Modified': 'Mon, 24 Oct 2022 10:39:03 GMT', 'Connection': 'close', 'Cache-Control': 'public', 'Content-Length': '0', 'X-Powered-By': 'ASP.NET', 'X-Cache': 'HIT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Security-Policy': "frame-ancestors 'self' https://mycourses.w3schools.com;", 'Accept-Ranges': 'bytes', 'Date': 'Tue, 25 Oct 2022 00:35:48 GMT', 'X-Content-Security-Policy': "frame-ancestors 'self' https://mycourses.w3schools.com;", 'Age': '50205'}

[GM-Script-Writer-62850]

OP could just pull the urequests lib and put it in / to replace the stock one (if OP does not want to use the nightly build)
urequests.py
i am running my own version with my own patches, i have 2 versions of this lib myself

[GC-RnD]

urequests doesn't have that method:
could you please elaborate.
mrequest show a sample of working script...
https://github.com/SpotlightKid/mrequests

You have any ideas how I can get the header info?
Thanks

[scruss]

urequests doesn't have that method:
could you please elaborate.

It means you can't do it using urequests.

I tried mrequests with no luck either. I don't think the code works as expected

[GC-RnD]

using your help(r) on mrequest...
object <Response object at 3ffe75e0> is of type Response
text --
qualname -- Response
module -- mrequests
save -- <function save at 0x3ffe6e30>
add_header -- <function add_header at 0x3ffe6e00>
read -- <function read at 0x3ffe6e10>
_parse_header -- <function _parse_header at 0x3ffe6df0>
content --
close -- <function close at 0x3ffe6e50>
json -- <function json at 0x3ffe6fe0>
init -- <function init at 0x3ffe6dd0>
r.headers = None

again any ideas?

[karfas]

Even when urequests.py doesn't have a method to read the headers, it has a mechanism to get them.
From https://github.com/micropython/micropython-lib/blob/master/python-ecosys/urequests/urequests.py:

def request(method, url, ..., parse_headers=True)
...
            if parse_headers is False:
                pass
            elif parse_headers is True:
                l = str(l, "utf-8")
                k, v = l.split(":", 1)
                resp_d[k] = v.strip()
            else:
                parse_headers(l, resp_d)

Sometimes it's very useful to read some code, so ... Use the source, Luke!
Above means that you can pass a handler function for the headers, e.g. (untested)

def handle_headers(line, headers):
    for key,value in headers.items(): 
        do_something() 
urequests.request(...., parse_headers=handle_headers)

[GC-RnD]

So I have a choice of a modifyed urequest or mrequests

mrequests does work @scruss

Christopher Arndt of mrequests
From the readme:
save_headers (bool) - a boolean, which is passed to the constructor of the response class instance,
which determines whether it keeps a reference to the response headers in the instance.
This is set to False by default to save memory.
If set to True, the default response class will make the reponse header lines available
via its headers instance attribute as a list of unparsed bytes objects.

Trying to find a fast fix put me into sensory overload.
Thank You Christopher for the fast responce yesterday!

`# MREQUESTS EXAMPLE
import mrequests as requests

r = requests.head('https://www.w3schools.com/python/demopage.php', save_headers=True)
print('r.headers = ', r.headers)
r.close()

r.headers = [b'Accept-Ranges: bytes\r\n', b'Age: 88589\r\n', b'Cache-Control: public\r\n', b"Content-Security-Policy: frame-ancestors 'self' https://mycourses.w3schools.com;\r\n", b'Content-Type: text/html; charset=UTF-8\r\n', b'Date: Tue, 25 Oct 2022 11:15:32 GMT\r\n', b'Last-Modified: Mon, 24 Oct 2022 10:39:03 GMT\r\n', b'Server: ECS (nyb/1D1F)\r\n', b'X-Cache: HIT\r\n', b"X-Content-Security-Policy: frame-ancestors 'self' https://mycourses.w3schools.com;\r\n", b'X-Powered-By: PHP/8.0.7\r\n', b'X-Powered-By: ASP.NET\r\n', b'Content-Length: 0\r\n', b'Connection: close\r\n']
`

[karfas]

Note that @jimmo's reply implied two possible solutions:

• upgrade to latest/nightly build (this comes with current urequest)
• install the current urequest.py

I don't think an urequest update counts as "modified urequest", isn't it ?