-
Notifications
You must be signed in to change notification settings - Fork 147
refactor: Asynchronous cog/extension loading. #1132
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from 41 commits
3c35a8c
d6877d6
c24dc14
124890b
8f85551
26c4663
213edba
e76ce01
fb8b1f2
f812d1c
c3988ff
2b30e94
ebea051
f37594e
1900902
1daebcd
3c1baa9
5566e17
416333e
f677880
8a873a5
abd0a1d
b68f114
1d740c3
94ceb0b
f38fed2
a078653
6cc4a6d
d8e80b0
0d34a70
45382d6
a0c5a8e
df499a7
c0ab613
01fc5f2
10c39cf
6328931
1f2dde0
ead0604
1e703e5
1dacea2
302f91f
442bad8
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Removed the ``loop`` and ``asyncio_debug`` parameters from :class:`Client`. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| The majority of the library now assumes that there is an :mod:`asyncio` event loop running. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Deprecate :attr:`Client.loop`. Use :func:`asyncio.get_running_loop` instead. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| Add :meth:`Client.setup_hook`. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| :meth:`Client.run` now uses :func:`asyncio.run` under-the-hood instead of custom runner logic. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| |commands| Make :meth:`Bot.load_extensions <ext.commands.Bot.load_extensions>`, :meth:`Bot.load_extension <ext.commands.Bot.load_extension>`, :meth:`Bot.unload_extension <ext.commands.Bot.unload_extension>`, :meth:`Bot.reload_extension <ext.commands.Bot.reload_extension>`, :meth:`Bot.add_cog <ext.commands.Bot.add_cog>`, and :meth:`Bot.remove_cog <ext.commands.Bot.remove_cog>` asynchronous. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| |commands| :meth:`Cog.cog_load <ext.commands.Cog.cog_load>` is now called *after* the cog has finished loading. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| |commands| :meth:`Cog.cog_load <ext.commands.Cog.cog_load>` and :meth:`Cog.cog_unload <ext.commands.Cog.cog_unload>` can now be either sync or async. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| |commands| The ``setup`` and ``teardown`` functions utilized by :ref:`ext_commands_extensions` can now be asynchronous. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -4,10 +4,10 @@ | |
|
|
||
| import asyncio | ||
| import logging | ||
| import signal | ||
| import sys | ||
| import traceback | ||
| import types | ||
| import warnings | ||
| from collections.abc import Callable, Coroutine | ||
| from datetime import timedelta | ||
| from errno import ECONNRESET | ||
|
|
@@ -117,41 +117,6 @@ | |
| _log = logging.getLogger(__name__) | ||
|
|
||
|
|
||
| def _cancel_tasks(loop: asyncio.AbstractEventLoop) -> None: | ||
| tasks = {t for t in asyncio.all_tasks(loop=loop) if not t.done()} | ||
|
|
||
| if not tasks: | ||
| return | ||
|
|
||
| _log.info("Cleaning up after %d tasks.", len(tasks)) | ||
| for task in tasks: | ||
| task.cancel() | ||
|
|
||
| loop.run_until_complete(asyncio.gather(*tasks, return_exceptions=True)) | ||
| _log.info("All tasks finished cancelling.") | ||
|
|
||
| for task in tasks: | ||
| if task.cancelled(): | ||
| continue | ||
| if task.exception() is not None: | ||
| loop.call_exception_handler( | ||
| { | ||
| "message": "Unhandled exception during Client.run shutdown.", | ||
| "exception": task.exception(), | ||
| "task": task, | ||
| } | ||
| ) | ||
|
|
||
|
|
||
| def _cleanup_loop(loop: asyncio.AbstractEventLoop) -> None: | ||
| try: | ||
| _cancel_tasks(loop) | ||
| loop.run_until_complete(loop.shutdown_asyncgens()) | ||
| finally: | ||
| _log.info("Closing the event loop.") | ||
| loop.close() | ||
|
|
||
|
|
||
| class SessionStartLimit: | ||
| """A class that contains information about the current session start limit, | ||
| at the time when the client connected for the first time. | ||
|
|
@@ -233,6 +198,9 @@ class Client: | |
|
|
||
| A number of options can be passed to the :class:`Client`. | ||
|
|
||
| .. versionchanged:: 3.0 | ||
| The ``asyncio_debug`` parameter has been removed. Use :meth:`asyncio.loop.set_debug` directly. | ||
|
|
||
| Parameters | ||
| ---------- | ||
| max_messages: :class:`int` | :data:`None` | ||
|
|
@@ -241,13 +209,6 @@ class Client: | |
|
|
||
| .. versionchanged:: 1.3 | ||
| Allow disabling the message cache and change the default size to ``1000``. | ||
| loop: :class:`asyncio.AbstractEventLoop` | :data:`None` | ||
| The :class:`asyncio.AbstractEventLoop` to use for asynchronous operations. | ||
| Defaults to :data:`None`, in which case the current event loop is | ||
| used, or a new loop is created if there is none. | ||
| asyncio_debug: :class:`bool` | ||
| Whether to enable asyncio debugging when the client starts. | ||
| Defaults to False. | ||
| connector: :class:`aiohttp.BaseConnector` | :data:`None` | ||
| The connector to use for connection pooling. | ||
| proxy: :class:`str` | :data:`None` | ||
|
|
@@ -365,8 +326,6 @@ class Client: | |
| ---------- | ||
| ws | ||
| The websocket gateway the client is currently connected to. Could be :data:`None`. | ||
| loop: :class:`asyncio.AbstractEventLoop` | ||
| The event loop that the client uses for asynchronous operations. | ||
| session_start_limit: :class:`SessionStartLimit` | :data:`None` | ||
| Information about the current session start limit. | ||
| Only available after initiating the connection. | ||
|
|
@@ -382,8 +341,6 @@ class Client: | |
| def __init__( | ||
| self, | ||
| *, | ||
| asyncio_debug: bool = False, | ||
| loop: asyncio.AbstractEventLoop | None = None, | ||
| shard_id: int | None = None, | ||
| shard_count: int | None = None, | ||
| enable_debug_events: bool = False, | ||
|
|
@@ -409,21 +366,26 @@ def __init__( | |
| # self.ws is set in the connect method | ||
| self.ws: DiscordWebSocket = None # pyright: ignore[reportAttributeAccessIssue] | ||
|
|
||
| if loop is None: | ||
| self.loop: asyncio.AbstractEventLoop = utils.get_event_loop() | ||
| else: | ||
| self.loop: asyncio.AbstractEventLoop = loop | ||
|
|
||
| self.loop.set_debug(asyncio_debug) | ||
| self._listeners: dict[str, list[tuple[asyncio.Future, Callable[..., bool]]]] = {} | ||
| self.session_start_limit: SessionStartLimit | None = None | ||
|
|
||
| if connector: | ||
| try: | ||
| asyncio.get_running_loop() | ||
| except RuntimeError: | ||
| msg = ( | ||
| "`connector` was created outside of an asyncio loop, which will likely cause" | ||
| "issues later down the line due to the client and `connector` running on" | ||
| "different asyncio loops; consider moving client instantiation to an '`async" | ||
| "main`' function and then manually asyncio.run it" | ||
| ) | ||
| raise RuntimeError(msg) from None | ||
|
Comment on lines
+372
to
+382
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I will remove this as this check only works when you create a dedicated loop for the connector but forget to set it as the global running loop, which is a very specific condition. I would rather not have this check at all (and document the requirement that was actually already there the whole time) or do a proper loop equivalency check, but the latter requires accessing private fields ( |
||
|
|
||
| self.http: HTTPClient = HTTPClient( | ||
| connector, | ||
| proxy=proxy, | ||
| proxy_auth=proxy_auth, | ||
| unsync_clock=assume_unsync_clock, | ||
| loop=self.loop, | ||
| ) | ||
|
|
||
| self._handlers: dict[str, Callable[..., Any]] = { | ||
|
|
@@ -510,7 +472,6 @@ def _get_state( | |
| handlers=self._handlers, | ||
| hooks=self._hooks, | ||
| http=self.http, | ||
| loop=self.loop, | ||
| max_messages=max_messages, | ||
| application_id=application_id, | ||
| heartbeat_timeout=heartbeat_timeout, | ||
|
|
@@ -531,6 +492,29 @@ def _handle_first_connect(self) -> None: | |
| return | ||
| self._first_connect.set() | ||
|
|
||
| @property | ||
| def loop(self): | ||
| """:class:`asyncio.AbstractEventLoop`: Same as :func:`asyncio.get_running_loop`. | ||
|
|
||
| .. deprecated:: 3.0 | ||
| Use :func:`asyncio.get_running_loop` directly. | ||
| """ | ||
| warnings.warn( | ||
| "Accessing `Client.loop` is deprecated. Use `asyncio.get_running_loop()` instead.", | ||
| category=DeprecationWarning, | ||
| stacklevel=2, | ||
| ) | ||
| return asyncio.get_running_loop() | ||
|
|
||
| @loop.setter | ||
| def loop(self, value: asyncio.AbstractEventLoop) -> None: | ||
| warnings.warn( | ||
| "Assigning to `Client.loop` is deprecated. Use `asyncio.set_event_loop()` instead.", | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This isn't communicated in the docstring of |
||
| category=DeprecationWarning, | ||
| stacklevel=2, | ||
| ) | ||
| asyncio.set_event_loop(value) | ||
|
|
||
| @property | ||
| def latency(self) -> float: | ||
| """:class:`float`: Measures latency between a HEARTBEAT and a HEARTBEAT_ACK in seconds. | ||
|
|
@@ -1015,12 +999,34 @@ async def before_identify_hook(self, shard_id: int | None, *, initial: bool = Fa | |
| if not initial: | ||
| await asyncio.sleep(5.0) | ||
|
|
||
| async def setup_hook(self) -> None: | ||
| """A hook that allows you to perform asynchronous setup like | ||
| initiating database connections or loading cogs/extensions after | ||
| the bot has logged in but before it has connected to the websocket. | ||
|
|
||
| This is only called once, in :meth:`.login`, before any events are | ||
| dispatched, making it a better solution than doing such setup in | ||
| the :func:`disnake.on_ready` event. | ||
|
|
||
| .. warning:: | ||
| Since this is called *before* the websocket connection is made, | ||
| anything that waits for the websocket will deadlock, which includes | ||
| methods like :meth:`.wait_for`, :meth:`.wait_until_ready` | ||
| and :meth:`.wait_until_first_connect`. | ||
|
|
||
| .. versionadded:: 3.0 | ||
| """ | ||
|
|
||
| # login state management | ||
|
|
||
| async def login(self, token: str) -> None: | ||
| """|coro| | ||
|
|
||
| Logs in the client with the specified credentials. | ||
| Logs in the client with the specified credentials and calls | ||
| :meth:`.setup_hook`. | ||
|
|
||
| .. versionchanged:: 3.0 | ||
| Now also calls :meth:`.setup_hook`. | ||
|
|
||
| Parameters | ||
| ---------- | ||
|
|
@@ -1045,6 +1051,8 @@ async def login(self, token: str) -> None: | |
| data = await self.http.static_login(token.strip()) | ||
| self._connection.user = ClientUser(state=self._connection, data=data) | ||
|
|
||
| await self.setup_hook() | ||
|
|
||
| async def connect( | ||
| self, *, reconnect: bool = True, ignore_session_start_limit: bool = False | ||
| ) -> None: | ||
|
|
@@ -1259,62 +1267,26 @@ def run(self, *args: Any, **kwargs: Any) -> None: | |
| function should not be used. Use :meth:`start` coroutine | ||
| or :meth:`connect` + :meth:`login`. | ||
|
|
||
| Roughly Equivalent to: :: | ||
| Equivalent to: :: | ||
|
|
||
| try: | ||
| loop.run_until_complete(start(*args, **kwargs)) | ||
| asyncio.run(start(*args, **kwargs)) | ||
| except KeyboardInterrupt: | ||
| loop.run_until_complete(close()) | ||
| # cancel all tasks lingering | ||
| finally: | ||
| loop.close() | ||
| return | ||
|
|
||
| .. warning:: | ||
|
|
||
| This function must be the last function to call due to the fact that it | ||
| is blocking. That means that registration of events or anything being | ||
| called after this function call will not execute until it returns | ||
| This function should be the last function to be called because it is blocking. | ||
| That means that registration of commands, events or any code after this function | ||
| call will not execute until it returns. | ||
|
|
||
| Parameters | ||
| ---------- | ||
| token: :class:`str` | ||
| The discord token of the bot that is being ran. | ||
| .. versionchanged:: 3.0 | ||
| Changed to use :func:`asyncio.run` instead of custom logic. | ||
| """ | ||
| loop = self.loop | ||
|
|
||
| try: | ||
| loop.add_signal_handler(signal.SIGINT, lambda: loop.stop()) | ||
| loop.add_signal_handler(signal.SIGTERM, lambda: loop.stop()) | ||
| except NotImplementedError: | ||
| pass | ||
|
|
||
| async def runner() -> None: | ||
| try: | ||
| await self.start(*args, **kwargs) | ||
| finally: | ||
| if not self.is_closed(): | ||
| await self.close() | ||
|
|
||
| def stop_loop_on_completion(f) -> None: | ||
| loop.stop() | ||
|
|
||
| future = asyncio.ensure_future(runner(), loop=loop) | ||
| future.add_done_callback(stop_loop_on_completion) | ||
| try: | ||
| loop.run_forever() | ||
| asyncio.run(self.start(*args, **kwargs)) | ||
| except KeyboardInterrupt: | ||
| _log.info("Received signal to terminate bot and event loop.") | ||
| finally: | ||
| future.remove_done_callback(stop_loop_on_completion) | ||
| _log.info("Cleaning up tasks.") | ||
| _cleanup_loop(loop) | ||
|
|
||
| if not future.cancelled(): | ||
| try: | ||
| future.result() | ||
| except KeyboardInterrupt: | ||
| # I am unsure why this gets raised here but suppress it anyway | ||
| pass | ||
| return | ||
|
|
||
| # properties | ||
|
|
||
|
|
@@ -1750,6 +1722,9 @@ def wait_for( | |
|
|
||
| This function returns the **first event that meets the requirements**. | ||
|
|
||
| .. important:: | ||
| Requires an :mod:`asyncio` loop to be running. | ||
|
|
||
| Examples | ||
| -------- | ||
| Waiting for a user reply: :: | ||
|
|
@@ -1813,6 +1788,8 @@ def check(reaction, user): | |
|
|
||
| Raises | ||
| ------ | ||
| RuntimeError | ||
| No running ``asyncio`` loop. | ||
| asyncio.TimeoutError | ||
| If a timeout is provided and it was reached. | ||
|
|
||
|
|
@@ -1823,7 +1800,7 @@ def check(reaction, user): | |
| arguments that mirrors the parameters passed in the | ||
| :ref:`event <disnake_api_events>`. | ||
| """ | ||
| future = self.loop.create_future() | ||
| future = asyncio.get_running_loop().create_future() | ||
| if check is None: | ||
|
|
||
| def _check(*args) -> bool: | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it be worth to add this back and only deprecate it?
Also, it might be better to upgrade to Sphinx 7.3 first and change the directive to
.. versionremoved::instead.