ResourceWarning in urllib tests

[serhiy-storchaka]

Many urllib tests produce resource warnings. For example (there are much more cases):

$ ./python -We -m test -vuall test_urllib test_urllib2 test_urllib2_localnet test_urllib_response test_urllibnet
...
test_geturl (test.test_urllib.urlopen_DataTests.test_geturl) ... ok
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697425919168 whose fp = <_io.BytesIO object at 0x7ff6abc15400>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697425378528 whose fp = <_io.BytesIO object at 0x7ff6abc15390>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697425378176 whose fp = <_io.BytesIO object at 0x7ff6ac21be70>>
test_info (test.test_urllib.urlopen_DataTests.test_info) ... Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420160224 whose fp = <_io.BytesIO object at 0x7ff6abc154e0>>
ok
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420342752 whose fp = <_io.BytesIO object at 0x7ff6abc15400>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420343392 whose fp = <_io.BytesIO object at 0x7ff6abc15390>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697425919168 whose fp = <_io.BytesIO object at 0x7ff6ac21be70>>
test_interface (test.test_urllib.urlopen_DataTests.test_interface) ... ok
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697426455808 whose fp = <_io.BytesIO object at 0x7ff6abc15400>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697426462784 whose fp = <_io.BytesIO object at 0x7ff6abc15390>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697426467104 whose fp = <_io.BytesIO object at 0x7ff6ac21be70>>
test_invalid_base64_data (test.test_urllib.urlopen_DataTests.test_invalid_base64_data) ... ok
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697426847840 whose fp = <_io.BytesIO object at 0x7ff6abc15400>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697426847072 whose fp = <_io.BytesIO object at 0x7ff6abc15390>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697426455808 whose fp = <_io.BytesIO object at 0x7ff6ac21be70>>
test_missing_comma (test.test_urllib.urlopen_DataTests.test_missing_comma) ... ok
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697425844896 whose fp = <_io.BytesIO object at 0x7ff6abc15400>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697425844000 whose fp = <_io.BytesIO object at 0x7ff6abc15390>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697419659664 whose fp = <_io.BytesIO object at 0x7ff6ac21be70>>
test_read_image (test.test_urllib.urlopen_DataTests.test_read_image) ... ok
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420324640 whose fp = <_io.BytesIO object at 0x7ff6abc15400>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697419712848 whose fp = <_io.BytesIO object at 0x7ff6abc15390>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697419711184 whose fp = <_io.BytesIO object at 0x7ff6ac21be70>>
test_read_text (test.test_urllib.urlopen_DataTests.test_read_text) ... ok
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420324832 whose fp = <_io.BytesIO object at 0x7ff6abc15400>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420324448 whose fp = <_io.BytesIO object at 0x7ff6abc15390>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420323104 whose fp = <_io.BytesIO object at 0x7ff6ac21be70>>
test_read_text_base64 (test.test_urllib.urlopen_DataTests.test_read_text_base64) ... ok
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420324640 whose fp = <_io.BytesIO object at 0x7ff6abc15400>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420325216 whose fp = <_io.BytesIO object at 0x7ff6abc15390>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420323104 whose fp = <_io.BytesIO object at 0x7ff6ac21be70>>
...
test_invalid_redirect (test.test_urllib.urlopen_HttpTests.test_invalid_redirect) ... Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <HTTPError 302: "Found - Redirection to url 'file://guidocomputer.athome.com:/python/license' is not allowed">
ok
...
5 tests altered the execution environment (env changed):
    test_urllib test_urllib2 test_urllib2_localnet
    test_urllib_response test_urllibnet
...

The reason is that addinfourl() is a subclass of tempfile._TemporaryFileWrapper. It keeps a reference to an open file and emits a warning in its destructor if was not explicitly closed.

The solution: every result of urllib.request.open() and similar functions and any HTTPError exception should be explicitly closed.

See also #128731.

Linked PRs

• gh-128734: Fix ResourceWarning in urllib tests #128735
• [3.13] gh-128734: Explicitly close sockets in urllib tests (GH-128735) #128748
• [3.12] gh-128734: Explicitly close sockets in urllib tests (GH-128735) #128750

[serhiy-storchaka]

The warnings are only emitted in the main branch, but it is worth to backport the changes to prevent depending on the garbage collector. It may matter on PyPy and others implementations.