Merge branch 'release/0.8.0'

Author: s3rius

.python-version

@@ -0,0 +1,4 @@
+3.11.4
+3.10.12
+3.9.17
+3.8.17

docs/.vuepress/styles/index.scss

@@ -1,18 +1,14 @@
-.toggle-navbar-button {
+.vp-site-name {
     visibility: hidden;
 }
 
-.hero-info-wrapper {
+.vp-hero-info {
     img {
         max-width: 60% !important;
         padding: 1rem;
     }
 }
 
-.actions {
+.vp-actions {
     align-items: flex-end;
 }
-
-span.site-name {
-    visibility: hidden;
-}

docs/contrib.md

@@ -0,0 +1,80 @@
+---
+order: 5
+---
+
+# Contribution guide
+
+We love contributions. This guide is for all folks who want to make taskiq
+better together.
+
+We have several rules for contributors:
+* Please do not add malware.
+* Please make sure that your request solves problem.
+
+If you struggle with something or feel frustrating, you either create an issue, create a discussion on page or publish draft PR and ask your question in description.
+
+We have lots of tests in CI. But since runs from first-time contributors should be approved you better test locally, it just takes less time.
+
+We love contributions. This guide is for all folks who want to make taskiq better together.
+We have several rules for contributors:
+* Please do not add malware.
+* Please make sure that your request solves the problem.
+
+If you struggle with something or feel frustrated, you either create an issue, create a [discussions](https://github.com/orgs/taskiq-python/discussions).
+page or publish a draft PR and ask your question in the description.
+
+We have lots of tests in CI. But since CI runs from first-time contributors should be approved, you better test locally. It just takes less time to prepare PR for merging.
+
+## Setting up environment
+
+We use poetry for managing dependencies. To install it, please follow the official guide in [documentation](https://python-poetry.org/docs/).
+
+After you have cloned the taskiq repo, install dependencies using this command:
+
+```bash
+poetry install
+```
+
+It will install all required dependencies.
+If you want to run pytest against different python environments, please install `pyenv` using instructions from its [readme](https://github.com/pyenv/pyenv).
+
+After pyenv is ready, you can install all python versions using this command:
+
+```bash
+pyenv install
+```
+
+## Linting
+
+We have `pre-commit` configured with all our settings. We highly recommend you to install it as a git hook using `pre-commit install` command.
+
+But even without installation, you can run all lints manually:
+
+```bash
+pre-commit run -a
+```
+
+
+## Testing
+
+You can run `pytest` without any parameters and it will do the thing.
+
+```bash
+pytest
+```
+
+If you want to speedup testings, you can run it with `-n` option from [pytest-xdist](https://pypi.org/project/pytest-xdist/) to run tests in parallel.
+
+```bash
+pytest -n 2
+```
+
+Also we use `tox` to test against different environments. You can publish a PR to run pytest with different
+python versions, but if you want to do it locally, just run `tox` command.
+
+
+```bash
+tox
+```
+
+Tox assumes that you've installed python versions using pyenv with command above.

docs/examples/extending/middleware_async.py

@@ -5,6 +5,14 @@
 
 
 class MyMiddleware(TaskiqMiddleware):
+    async def startup(self) -> None:
+        print("RUN STARTUP")
+        await sleep(1)
+
+    async def shutdown(self) -> None:
+        print("RUN SHUTDOWN")
+        await sleep(1)
+
     async def pre_execute(self, message: "TaskiqMessage") -> TaskiqMessage:
         await sleep(1)
         return message

docs/examples/extending/middleware_sync.py

@@ -5,6 +5,14 @@
 
 
 class MyMiddleware(TaskiqMiddleware):
+    def startup(self) -> None:
+        print("RUN STARTUP")
+        sleep(1)
+
+    def shutdown(self) -> None:
+        print("RUN SHUTDOWN")
+        sleep(1)
+
     def pre_execute(self, message: "TaskiqMessage") -> TaskiqMessage:
         sleep(1)
         return message

docs/guide/architecture-overview.md

@@ -82,9 +82,60 @@ asyncio.run(main())
 
 ```
 
+## Messages
+
+Every message has labels. You can define labels
+using `task` decorator, or you can add them using kicker.
+
+For example:
+
+```python
+
+@broker.task(my_label=1, label2="something")
+async def my_async_task() -> None:
+    """My lovely task."""
+    await asyncio.sleep(1)
+    print("Hello")
+
+async def main():
+    await my_async_task.kiq()
+```
+
+It's equivalent to this
+
+```python
+
+@broker.task
+async def my_async_task() -> None:
+    """My lovely task."""
+    await asyncio.sleep(1)
+    print("Hello")
+
+async def main():
+    await my_async_task.kicker().with_labels(
+        my_label=1,
+        label2="something",
+    ).kiq()
+```
+
+Also you can assign custom task names using decorator.
+This is useful to be sure that task names are unique and resolved correctly.
+Also it may be useful to balance message routing in some brokers.
+
+for example:
+
+```python
+@broker.task(task_name="my_tasks.add_one", label1=1)
+async def my_async_task() -> None:
+    """My lovely task."""
+    await asyncio.sleep(1)
+    print("Hello")
+
+```
+
 ## Result backend
 
-This part is used to store and get results of the execution.
+Result backend is used to store and get results of the execution.
 Results have type `TaskiqResult` from [taskiq.result](https://github.com/taskiq-python/taskiq/blob/master/taskiq/result.py).
 
 Every ResultBackend must implement `AsyncResultBackend` from [taskiq.abc.result_backend](https://github.com/taskiq-python/taskiq/blob/master/taskiq/abc/result_backend.py). By default, brokers use `DummyResultBackend`. It doesn't do anything and cannot be used
@@ -131,6 +182,12 @@ taskiq worker test_project.broker:broker -fsd
 If you have uvloop installed, taskiq will automatically install new policies to event loop.
 You can get more info about the CLI in the [CLI](./cli.md) section.
 
+::: info Cool info
+
+By default we start two processes, if you want to change this value, please take a look at `--help`.
+
+:::
+
 ## Middlewares
 
 Middlewares are used to modify message, or take
@@ -182,88 +239,35 @@ Middlewares can store information in `message.labels` for
 later use. For example `SimpleRetryMiddleware` uses labels
 to remember number of failed attempts.
 
-## Messages
+## Context
 
-Every message has labels. You can define labels
-using `task` decorator, or you can add them using kicker.
+Context is a useful class with some additional functions.
+You can use context to get broker that runs this task, from inside of the task.
 
-For example:
+Or it has ability to control the flow of execution. Here's example of how to get
+the context.
 
 ```python
+from taskiq import Context, TaskiqDepends, ZeroMQBroker
 
-@broker.task(my_label=1, label2="something")
-async def my_async_task() -> None:
-    """My lovely task."""
-    await asyncio.sleep(1)
-    print("Hello")
+broker = ZeroMQBroker()
 
-async def main():
-    await my_async_task.kiq()
-```
-
-It's equivalent to this
-
-```python
 
 @broker.task
-async def my_async_task() -> None:
-    """My lovely task."""
-    await asyncio.sleep(1)
-    print("Hello")
-
-async def main():
-    await my_async_task.kicker().with_labels(
-        my_label=1,
-        label2="something",
-    ).kiq()
+async def my_task(context: Context = TaskiqDepends()):
+    ...
 ```
 
-Also you can assign custom task names using decorator.
-This is useful to be sure that task names are unique and resolved correctly.
-Also it may be useful to balance message routing in some brokers.
-
-for example:
+Also through contexts you can reject or requeue a task. It's easy as this:
 
 ```python
-@broker.task(task_name="my_tasks.add_one", label1=1)
-async def my_async_task() -> None:
-    """My lovely task."""
-    await asyncio.sleep(1)
-    print("Hello")
-
+@broker.task
+async def my_task(context: Context = TaskiqDepends()):
+   await context.requeue()
 ```
 
-## Context
-
-This section is useful for library developers. Who want to get current broker during shared task execution.
-
-For example, you've created shared_task and you want to send message in that task.
-This can be done with context.
-
-Context holds information about the current broker and current incoming message.
-To get it, simply add the context parameter with `type-hint`.
-
-::: danger Cool warning!
-Context injected only if you have a type hint.
-:::
-
-Example:
-
-```python
-from taskiq import async_shared_broker, BrokerMessage, Context
-
-
-@async_shared_broker.task
-async def my_shr_task(context: Context):
-    message = BrokerMessage(
-        task_id="123",
-        task_name="dummy_name",
-        message='{"one": "two"}',
-        labels={},
-    )
-    await context.broker.kick(message)
-
-```
+Calling `requeue` or `reject` stops task execution and either drops the message,
+or puts it back to the queue.
 
-This is useless example, but it's good as a demonstration.
-Pipelines are built using this magic.
+Also, with context you'll be able to get current message that was received by the broker
+or even instance of a broker who received a message. This may be useful for lib developers.

docs/guide/cli.md

@@ -71,6 +71,16 @@ To enable this option simply pass the `--reload` or `-r` option to worker taskiq
 Also this option supports `.gitignore` files. If you have such file in your directory, it won't reload worker
 when you modify ignored files. To disable this functionality pass `--do-not-use-gitignore` option.
 
+### Other parameters
+
+* `--no-configure-logging` - disables default logging configuration for workers.
+* `--max-async-tasks` - maximum number of simultaneously running async tasks.
+* `--max-prefetch` - number of tasks to be prefetched before execution. (Useful for systems with high message rates, but brokers should support acknowledgements).
+* `--max-threadpool-threads` - number of threads for sync function exection.
+* `--no-propagate-errors` - if this parameter is enabled, exceptions won't be thrown in generator dependencies.
+* `--receiver` - python path to custom receiver class.
+* `--receiver_arg` - custom args for receiver.
+
 ## Scheduler
 
 Scheduler is used to schedule tasks as described in [Scheduling tasks](./scheduling-tasks.md) section.

docs/guide/state-and-deps.md

@@ -165,29 +165,12 @@ If you want to do something asynchronously, convert this function to an asynchro
 
 @[code python](../examples/state/async_generator_deps.py)
 
-### Default dependencies
-
-By default taskiq has only two dependencies:
-
-- Context from `taskiq.context.Context`
-- TaskiqState from `taskiq.state.TaskiqState`
-
-
-### Adding first-level dependencies
-
-You can expand default list of available dependencies for you application.
-Taskiq have an ability to add new first-level dependencies using brokers.
-
-The AsyncBroker interface has a function called `add_dependency_context` and you can add
-more default dependencies to the taskiq. This may be useful for libraries if you want to
-add new dependencies to users.
-
 
-### Exception handling
+#### Exception handling
 
-Dependencies can handle exceptions that happen in tasks. This feature is handy if you want your system to be more atomic.
+Generator dependencies can handle exceptions that happen in tasks. This feature is handy if you want your system to be more atomic.
 
-For example, if you open a database transaction in your dependency and want to commit it only if the function is completed successfully.
+For example, if you open a database transaction in your dependency and want to commit it only if the function you execute is completed successfully.
 
 ```python
 async def get_transaction(db_driver: DBDriver = TaskiqDepends(get_driver)) -> AsyncGenerator[Transaction, None]:
@@ -211,3 +194,20 @@ taskiq worker my_file:broker --no-propagate-errors
 ```
 
 In this case, no exception will ever going to be propagated to any dependency.
+
+### Default dependencies
+
+By default taskiq has only two dependencies:
+
+- Context from `taskiq.context.Context`
+- TaskiqState from `taskiq.state.TaskiqState`
+
+
+### Adding first-level dependencies
+
+You can expand default list of available dependencies for you application.
+Taskiq have an ability to add new first-level dependencies using brokers.
+
+The AsyncBroker interface has a function called `add_dependency_context` and you can add
+more default dependencies to the taskiq. This may be useful for libraries if you want to
+add new dependencies to users.