ERROR: Task was destroyed but it is pending! #188
Description
I sometimes get this error when i use scrapy-pilaywright
2023-03-31 09:33:35 [asyncio] ERROR: Task was destroyed but it is pending!
source_traceback: Object created at (most recent call last):
File "/Users/rccpony/PycharmProjects/bid-spiders/packages/universal_spiders/main.py", line 5, in <module>
cmdline.execute(
File "/Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/scrapy/cmdline.py", line 154, in execute
_run_print_help(parser, _run_command, cmd, args, opts)
File "/Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/scrapy/cmdline.py", line 109, in _run_print_help
func(*a, **kw)
File "/Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/scrapy/cmdline.py", line 162, in _run_command
cmd.run(args, opts)
File "/Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/scrapy/commands/crawl.py", line 27, in run
self.crawler_process.start()
File "/Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/scrapy/crawler.py", line 363, in start
reactor.run(installSignalHandlers=False) # blocking call
File "/Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/twisted/internet/asyncioreactor.py", line 257, in run
self._asyncioEventloop.run_forever()
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/base_events.py", line 603, in run_forever
self._run_once()
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/base_events.py", line 1901, in _run_once
handle._run()
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "/Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 249, in run
await self._transport.run()
File "/Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/playwright/_impl/_transport.py", line 155, in run
self.on_message(obj)
File "/Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 210, in <lambda>
self._transport.on_message = lambda msg: self.dispatch(msg)
File "/Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/playwright/_impl/_connection.py", line 356, in dispatch
object._channel.emit(
File "/Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/pyee/base.py", line 176, in emit
handled = self._call_handlers(event, args, kwargs)
File "/Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/pyee/base.py", line 154, in _call_handlers
self._emit_run(f, args, kwargs)
File "/Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/pyee/asyncio.py", line 48, in _emit_run
coro: Any = f(*args, **kwargs)
File "/Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/playwright/_impl/_browser_context.py", line 131, in <lambda>
lambda params: self._on_request(
File "/Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/playwright/_impl/_browser_context.py", line 485, in _on_request
page.emit(Page.Events.Request, request)
File "/Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/pyee/base.py", line 176, in emit
handled = self._call_handlers(event, args, kwargs)
File "/Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/pyee/base.py", line 154, in _call_handlers
self._emit_run(f, args, kwargs)
File "/Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/pyee/asyncio.py", line 57, in _emit_run
fut: Any = ensure_future(cast(Any, coro), loop=self._loop)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/tasks.py", line 615, in ensure_future
return _ensure_future(coro_or_future, loop=loop)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/tasks.py", line 636, in _ensure_future
return loop.create_task(coro_or_future)
task: <Task pending name='Task-386' coro=<_make_request_logger.<locals>._log_request() running at /Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/scrapy_playwright/handler.py:606> wait_for=<Future pending cb=[Task.task_wakeup()] created at /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/base_events.py:429> cb=[AsyncIOEventEmitter._emit_run.<locals>.callback() at /Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/pyee/asyncio.py:65, ProtocolCallback.__init__.<locals>.cb() at /Users/rccpony/Library/Caches/pypoetry/virtualenvs/bid-spiders-KPnmHDqb-py3.10/lib/python3.10/site-packages/playwright/_impl/_connection.py:168] created at /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/tasks.py:636>
[scrapy.utils.log] INFO: Scrapy 2.7.0 started (bot: universal_spiders)
[scrapy.utils.log] INFO: Versions: lxml 4.9.2.0, libxml2 2.9.13, cssselect 1.2.0, parsel 1.7.0, w3lib 2.1.1, Twisted 21.7.0, Python 3.10.10 (v3.10.10:aad5f6a891, Feb 7 2023, 08:47:40) [Clang 13.0.0 (clang-1300.0.29.30)], pyOpenSSL 22.1.0 (OpenSSL 3.0.7 1 Nov 2022), cryptography 38.0.4, Platform macOS-13.2.1-arm64-arm-64bi
(Edit) Reproducible example at #233