Skip to content

Commit b03811f

Browse files
authored
Big docs update. (#140)
1 parent 7388c6a commit b03811f

File tree

10 files changed

+2231
-1826
lines changed

10 files changed

+2231
-1826
lines changed

docs/.vuepress/config.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ import { hopeTheme } from "vuepress-theme-hope";
55
export default defineUserConfig({
66
lang: "en-US",
77
title: "Taskiq",
8-
description: "Distributed task queue with full async support",
8+
description: "Async Distributed Task Manager",
99
head: [
1010
[
1111
"meta",
@@ -31,6 +31,7 @@ export default defineUserConfig({
3131
backToTop: false,
3232

3333
plugins: {
34+
readingTime: false,
3435
copyCode: {
3536
showInMobile: true,
3637
},

docs/README.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Home
2+
title: Task manager for asyncio
33
home: true
44
heroImage: logo.svg
55
heroAlt: logo
@@ -28,6 +28,15 @@ features:
2828
footer: MIT Licensed | Copyright© 2022-2023
2929
---
3030

31+
## What is taskiq in a nutshell
32+
33+
Consider taskiq as an asyncio celery implementation. It uses almost the same patterns, but it's more modern
34+
and flexible.
35+
36+
It's not a drop-in replacement for any other task manager. It has a different ecosystem of libraries and a different set of features.
37+
Also, it doesn't work for synchronous projects. You won't be able to send tasks synchronously.
38+
39+
3140
## Installation
3241

3342
You can install taskiq with pip or your favorite dependency manager:

docs/available-components/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
order: 1
33
dir:
4-
order: 3
4+
order: 4
55
---
66

77
# Available components

docs/extending-taskiq/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
dir:
3-
order: 2
3+
order: 3
44
---
55

66
# Extending taskiq

docs/framework_integrations/README.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
order: 1
3+
dir:
4+
order: 2
5+
---
6+
7+
# Framework integrations
8+
9+
Taskiq is meant to be simple and adaptive. That's why we try to add different integrations to make development with taskiq and your favorite framework easy and fun!
10+
11+
Integrations with frameworks add two things:
12+
1. Startup and Shutdown events;
13+
1. Dependencies to use in your handler.
Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
---
2+
order: 2
3+
---
4+
5+
# Taskiq + AioHTTP
6+
7+
AioHTTP is a framework for building robust applications. We created several libraries to make the experience with AioHTTP even better.
8+
9+
# Dependency inecjeciton for AioHTTP
10+
11+
We created a library [aiohttp-deps](https://pypi.org/project/aiohttp-deps/) to add FastAPI-like dependency injection in AioHTTP.
12+
13+
To install it, simply run:
14+
15+
```python
16+
pip install "aiohttp-deps"
17+
```
18+
19+
After the installation, please add startup event to your application to initialize dependencies context.
20+
21+
```python
22+
from aiohttp import web
23+
import aiohttp_deps
24+
25+
26+
app = web.Application()
27+
28+
# This startup event makes all the magic happen.
29+
# It parses current handlers and create dependency graphs for them.
30+
app.on_startup.append(aiohttp_deps.init)
31+
32+
web.run_app(app)
33+
```
34+
35+
You can read more about dependency injection and available dependencies in the project's [README.md](https://github.com/taskiq-python/aiohttp-deps).
36+
37+
38+
39+
## Adding taskiq integration
40+
41+
We highly recommend using aiohttp with aiohttp-deps because it allows us to reuse the same dependencies for your handlers and tasks. First of all, you should install the [taskiq-aiohttp](https://pypi.org/project/taskiq-aiohttp/) library.
42+
43+
```python
44+
pip install "taskiq-aiohttp"
45+
```
46+
47+
After the installation is complete, add an initialization function call to your broker's main file so it becomes something like this:
48+
49+
```python
50+
import taskiq_aiohttp
51+
52+
broker = MyBroker()
53+
54+
# The second argument is a path to web.Application variable.
55+
# Also you can provide here a factory function that takes no
56+
# arguments and returns an application. This function can be async.
57+
taskiq_aiohttp.init(broker, "my_project.main:app")
58+
```
59+
60+
From this point, you'll be able to reuse the same dependencies as with `aiohttp-deps`.
61+
Let's take a look at this function:
62+
63+
```python
64+
from aiohttp import web
65+
from taskiq import TaskiqDepends
66+
from my_project.tkq import broker
67+
68+
@broker.task
69+
async def my_task(app: web.Application = TaskiqDepends()):
70+
...
71+
72+
```
73+
74+
In this example, we depend on the current application. We can use its state in a current task or any other dependency. We can take db_pool from your application's state, which is the same pool, as the one you've created on AiohTTP's startup.
75+
But this application is only a mock of your application. It has correct types and all your variables that you filled on startup, but it doesn't handle any request.
76+
This integration adds two main dependencies:
77+
* web.Application - current application.
78+
* web.Request - mocked request. This request only exists to be able to use the same dependencies.
79+
80+
You can find more detailed examples in the [examples repo](https://github.com/taskiq-python/examples).
81+
82+
## Testing
83+
84+
Writing tests for AioHTTP with taskiq is as easy as writing tests for the aiohttp application. The only difference is that, if you want to use InMemoryBroker, then you need to add context for dependency injection. It's easier to call `populate_context` when creating a `test_client` fixture.
85+
86+
```python
87+
import taskiq_aiohttp
88+
89+
@pytest.fixture
90+
async def test_client(
91+
app: web.Application,
92+
) -> AsyncGenerator[TestClient, None]:
93+
"""
94+
Create a test client.
95+
96+
This function creates a TestServer
97+
and a test client for the application.
98+
99+
Also this fixture populates context
100+
with needed variables.
101+
102+
:param app: current application.
103+
:yield: ready to use client.
104+
"""
105+
loop = asyncio.get_running_loop()
106+
server = TestServer(app)
107+
client = TestClient(server, loop=loop)
108+
109+
await client.start_server()
110+
111+
# This is important part.
112+
# Since InMemoryBroker doesn't
113+
# run as a worker process, we have to populate
114+
# broker's context by hand.
115+
taskiq_aiohttp.populate_context(
116+
broker=broker,
117+
server=server.runner.server,
118+
app=app,
119+
loop=loop,
120+
)
121+
122+
yield client
123+
124+
broker.custom_dependency_context = {}
125+
await client.close()
126+
```
Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
---
2+
order: 1
3+
---
4+
5+
# Taskiq + FastAPI
6+
7+
FastAPI is a highly popular async web framework in Python. It has gained its popularity because of two things:
8+
1. It's easy to use;
9+
2. Cool dependency injection.
10+
11+
In taskiq, we try to make our libraries easy to use, and We have a dependency injection too. So we have created the library "[taskiq-fastapi](https://github.com/taskiq-python/taskiq-fastapi)" to make integration with FastAPI as smooth as possible.
12+
13+
Let's see what we got here. In this library, we provide users with only one public function called `init`. It takes a broker and a string path (as in uvicorn) to the fastapi application (or factory function). People should call this function in their main broker file.
14+
15+
```python
16+
from taskiq import ZeroMQBroker
17+
import taskiq_fastapi
18+
19+
broker = ZeroMQBroker()
20+
21+
taskiq_fastapi.init(broker, "my_package.application:app")
22+
23+
```
24+
25+
There are two rules to make everything work as you expect:
26+
1. Add `TaskiqDepends` as a default value for every parameter with `Request` or `HTTPConnection` types in base dependencies.
27+
2. Use only `TaskiqDepends` in tasks.
28+
29+
30+
::: tip Cool and important note!
31+
32+
The `Request` or `HTTPConnection` that you'll get injected in your task is not the same request or connection you have had in your handler when you were sending the task!
33+
34+
:::
35+
36+
Many fastapi dependency functions depend on `fastapi.Request`. We provide a mocked request to such dependencies. But taskiq cannot resolve dependencies until you explicitly specify that this parameter must be injected.
37+
38+
As an example. If you previously had a dependency like this:
39+
40+
```python
41+
from fastapi import Request
42+
from typing import Any
43+
44+
def get_redis_pool(request: Request) -> Any:
45+
return request.app.state.redis_pool
46+
47+
```
48+
49+
To make it resolvable in taskiq, people should add `TaskiqDepends` as a default value for each parameter. Like this:
50+
```python
51+
from fastapi import Request
52+
from taskiq import TaskiqDepends
53+
54+
55+
async def get_redis_pool(request: Request = TaskiqDepends()):
56+
return request.app.state.redis_pool
57+
58+
```
59+
60+
61+
Also you want to call startup of your brokers somewhere.
62+
63+
```python
64+
from fastapi import FastAPI
65+
from your_project.taskiq import broker
66+
67+
app = FastAPI()
68+
69+
70+
@app.on_event("startup")
71+
async def app_startup():
72+
if not broker.is_worker_process:
73+
await broker.startup()
74+
75+
76+
@app.on_event("shutdown")
77+
async def app_shutdown():
78+
if not broker.is_worker_process:
79+
await broker.shutdown()
80+
81+
```
82+
83+
And that's it. Now you can use your taskiq tasks with functions and classes that depend on FastAPI dependenices. You can find bigger examples in the [examples repo](https://github.com/taskiq-python/examples/).
84+
85+
86+
## Testing
87+
88+
Testing is no different from general testing advice from articles about [testing](../guide/testing-taskiq.md). But if you use `InMemoryBroker` in your tests, you need to provide it with a custom dependency context because it doesn't run as a worker process.
89+
90+
Let's imagine that you have a fixture of your application. It returns a new fastapi application to use in tests.
91+
```python
92+
93+
@pytest.fixture
94+
def fastapi_app() -> FastAPI:
95+
return get_app()
96+
97+
```
98+
99+
Right after this fixture, we define another one.
100+
101+
```python
102+
import taskiq_fastapi
103+
104+
105+
@pytest.fixture(autouse=True)
106+
def init_taskiq_deps(fastapi_app: FastAPI):
107+
# This is important part. Here we add dependency context,
108+
# this thing helps in resolving dependencies for tasks
109+
# for inmemory broker.
110+
taskiq_fastapi.populate_dependency_context(broker, fastapi_app)
111+
112+
yield
113+
114+
broker.custom_dependency_context = {}
115+
116+
```
117+
118+
This fixture has autouse flag, which means it would run on every test automatically.

0 commit comments

Comments
 (0)