Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ResourceWarning in urllib tests #128734

Open
serhiy-storchaka opened this issue Jan 11, 2025 · 0 comments
Open

ResourceWarning in urllib tests #128734

serhiy-storchaka opened this issue Jan 11, 2025 · 0 comments
Labels
tests Tests in the Lib/test dir

Comments

@serhiy-storchaka
Copy link
Member

serhiy-storchaka commented Jan 11, 2025 •
edited by bedevere-app bot
Loading

Many urllib tests produce resource warnings. For example (there are much more cases):

$ ./python -We -m test -vuall test_urllib test_urllib2 test_urllib2_localnet test_urllib_response test_urllibnet
...
test_geturl (test.test_urllib.urlopen_DataTests.test_geturl) ... ok
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697425919168 whose fp = <_io.BytesIO object at 0x7ff6abc15400>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697425378528 whose fp = <_io.BytesIO object at 0x7ff6abc15390>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697425378176 whose fp = <_io.BytesIO object at 0x7ff6ac21be70>>
test_info (test.test_urllib.urlopen_DataTests.test_info) ... Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420160224 whose fp = <_io.BytesIO object at 0x7ff6abc154e0>>
ok
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420342752 whose fp = <_io.BytesIO object at 0x7ff6abc15400>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420343392 whose fp = <_io.BytesIO object at 0x7ff6abc15390>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697425919168 whose fp = <_io.BytesIO object at 0x7ff6ac21be70>>
test_interface (test.test_urllib.urlopen_DataTests.test_interface) ... ok
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697426455808 whose fp = <_io.BytesIO object at 0x7ff6abc15400>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697426462784 whose fp = <_io.BytesIO object at 0x7ff6abc15390>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697426467104 whose fp = <_io.BytesIO object at 0x7ff6ac21be70>>
test_invalid_base64_data (test.test_urllib.urlopen_DataTests.test_invalid_base64_data) ... ok
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697426847840 whose fp = <_io.BytesIO object at 0x7ff6abc15400>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697426847072 whose fp = <_io.BytesIO object at 0x7ff6abc15390>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697426455808 whose fp = <_io.BytesIO object at 0x7ff6ac21be70>>
test_missing_comma (test.test_urllib.urlopen_DataTests.test_missing_comma) ... ok
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697425844896 whose fp = <_io.BytesIO object at 0x7ff6abc15400>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697425844000 whose fp = <_io.BytesIO object at 0x7ff6abc15390>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697419659664 whose fp = <_io.BytesIO object at 0x7ff6ac21be70>>
test_read_image (test.test_urllib.urlopen_DataTests.test_read_image) ... ok
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420324640 whose fp = <_io.BytesIO object at 0x7ff6abc15400>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697419712848 whose fp = <_io.BytesIO object at 0x7ff6abc15390>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697419711184 whose fp = <_io.BytesIO object at 0x7ff6ac21be70>>
test_read_text (test.test_urllib.urlopen_DataTests.test_read_text) ... ok
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420324832 whose fp = <_io.BytesIO object at 0x7ff6abc15400>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420324448 whose fp = <_io.BytesIO object at 0x7ff6abc15390>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420323104 whose fp = <_io.BytesIO object at 0x7ff6ac21be70>>
test_read_text_base64 (test.test_urllib.urlopen_DataTests.test_read_text_base64) ... ok
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420324640 whose fp = <_io.BytesIO object at 0x7ff6abc15400>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420325216 whose fp = <_io.BytesIO object at 0x7ff6abc15390>>
Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <addinfourl at 140697420323104 whose fp = <_io.BytesIO object at 0x7ff6ac21be70>>
...
test_invalid_redirect (test.test_urllib.urlopen_HttpTests.test_invalid_redirect) ... Warning -- Unraisable exception
Exception ignored in: <function _TemporaryFileCloser.__del__ at 0x7ff6ac410dd0>
Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/tempfile.py", line 483, in __del__
    _warnings.warn(self.warn_message, ResourceWarning)
ResourceWarning: Implicitly cleaning up <HTTPError 302: "Found - Redirection to url 'file://guidocomputer.athome.com:/python/license' is not allowed">
ok
...
5 tests altered the execution environment (env changed):
    test_urllib test_urllib2 test_urllib2_localnet
    test_urllib_response test_urllibnet
...

The reason is that addinfourl() is a subclass of tempfile._TemporaryFileWrapper. It keeps a reference to an open file and emits a warning in its destructor if was not explicitly closed.

The solution: every result of urllib.request.open() and similar functions and any HTTPError exception should be explicitly closed.

See also #128731.

Linked PRs
@serhiy-storchaka serhiy-storchaka added the tests Tests in the Lib/test dir label Jan 11, 2025
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this issue Jan 11, 2025
@serhiy-storchaka
pythongh-128734: Fix ResourceWarning in urllib tests
d770062
@bedevere-app bedevere-app bot mentioned this issue Jan 11, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
tests Tests in the Lib/test dir
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@serhiy-storchaka