Skip to content

Commit 5397858

Browse files
author
Anton
committed
Merge branch 'develop' into 'idle-tasks'
2 parents 03b309c + 1f332da commit 5397858

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

46 files changed

+3271
-1882
lines changed

.flake8

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,8 @@ ignore =
103103
F403,
104104
; Found wrong metadata variable
105105
WPS410,
106+
; Found commented out cod
107+
E800,
106108

107109
per-file-ignores =
108110
; all tests

.github/FUNDING.yml

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# These are supported funding model platforms
2+
3+
github: taskiq-python
4+
patreon: # Replace with a single Patreon username
5+
open_collective: # Replace with a single Open Collective username
6+
ko_fi: # Replace with a single Ko-fi username
7+
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
8+
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
9+
liberapay: # Replace with a single Liberapay username
10+
issuehunt: # Replace with a single IssueHunt username
11+
otechie: # Replace with a single Otechie username
12+
lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
13+
custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']

docs/.vuepress/config.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ import { hopeTheme } from "vuepress-theme-hope";
55
export default defineUserConfig({
66
lang: "en-US",
77
title: "Taskiq",
8-
description: "Distributed task queue with full async support",
8+
description: "Async Distributed Task Manager",
99
head: [
1010
[
1111
"meta",
@@ -31,6 +31,7 @@ export default defineUserConfig({
3131
backToTop: false,
3232

3333
plugins: {
34+
readingTime: false,
3435
copyCode: {
3536
showInMobile: true,
3637
},

docs/README.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Home
2+
title: Task manager for asyncio
33
home: true
44
heroImage: logo.svg
55
heroAlt: logo
@@ -28,6 +28,15 @@ features:
2828
footer: MIT Licensed | Copyright© 2022-2023
2929
---
3030

31+
## What is taskiq in a nutshell
32+
33+
Consider taskiq as an asyncio celery implementation. It uses almost the same patterns, but it's more modern
34+
and flexible.
35+
36+
It's not a drop-in replacement for any other task manager. It has a different ecosystem of libraries and a different set of features.
37+
Also, it doesn't work for synchronous projects. You won't be able to send tasks synchronously.
38+
39+
3140
## Installation
3241

3342
You can install taskiq with pip or your favorite dependency manager:

docs/available-components/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
order: 1
33
dir:
4-
order: 3
4+
order: 4
55
---
66

77
# Available components

docs/available-components/brokers.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ In this section we'll list officially supported brokers.
1010

1111
This is a special broker for local development. It uses the same functions to execute tasks,
1212
but all tasks are executed locally in the current thread.
13-
By default it uses `InMemoryResultBackend` but this can be overriden.
13+
By default it uses `InMemoryResultBackend` but this can be overridden.
1414

1515
## ZeroMQBroker
1616

docs/examples/extending/result_backend.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ async def is_result_ready(
5353
Check if result exists.
5454
5555
This function must check whether result
56-
is available in your resul backend
56+
is available in your result backend
5757
without fetching the result.
5858
5959
:param task_id: id of a task.

docs/extending-taskiq/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
dir:
3-
order: 2
3+
order: 3
44
---
55

66
# Extending taskiq

docs/framework_integrations/README.md

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
---
2+
order: 1
3+
dir:
4+
order: 2
5+
---
6+
7+
# Framework integrations
8+
9+
Taskiq is meant to be simple and adaptive. That's why we try to add different integrations to make development with taskiq and your favorite framework easy and fun!
10+
11+
Integrations with frameworks add two things:
12+
1. Startup and Shutdown events;
13+
1. Dependencies to use in your handler.
Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
---
2+
order: 2
3+
---
4+
5+
# Taskiq + AioHTTP
6+
7+
AioHTTP is a framework for building robust applications. We created several libraries to make the experience with AioHTTP even better.
8+
9+
# Dependency inecjeciton for AioHTTP
10+
11+
We created a library [aiohttp-deps](https://pypi.org/project/aiohttp-deps/) to add FastAPI-like dependency injection in AioHTTP.
12+
13+
To install it, simply run:
14+
15+
```python
16+
pip install "aiohttp-deps"
17+
```
18+
19+
After the installation, please add startup event to your application to initialize dependencies context.
20+
21+
```python
22+
from aiohttp import web
23+
import aiohttp_deps
24+
25+
26+
app = web.Application()
27+
28+
# This startup event makes all the magic happen.
29+
# It parses current handlers and create dependency graphs for them.
30+
app.on_startup.append(aiohttp_deps.init)
31+
32+
web.run_app(app)
33+
```
34+
35+
You can read more about dependency injection and available dependencies in the project's [README.md](https://github.com/taskiq-python/aiohttp-deps).
36+
37+
38+
39+
## Adding taskiq integration
40+
41+
We highly recommend using aiohttp with aiohttp-deps because it allows us to reuse the same dependencies for your handlers and tasks. First of all, you should install the [taskiq-aiohttp](https://pypi.org/project/taskiq-aiohttp/) library.
42+
43+
```python
44+
pip install "taskiq-aiohttp"
45+
```
46+
47+
After the installation is complete, add an initialization function call to your broker's main file so it becomes something like this:
48+
49+
```python
50+
import taskiq_aiohttp
51+
52+
broker = MyBroker()
53+
54+
# The second argument is a path to web.Application variable.
55+
# Also you can provide here a factory function that takes no
56+
# arguments and returns an application. This function can be async.
57+
taskiq_aiohttp.init(broker, "my_project.main:app")
58+
```
59+
60+
From this point, you'll be able to reuse the same dependencies as with `aiohttp-deps`.
61+
Let's take a look at this function:
62+
63+
```python
64+
from aiohttp import web
65+
from taskiq import TaskiqDepends
66+
from my_project.tkq import broker
67+
68+
@broker.task
69+
async def my_task(app: web.Application = TaskiqDepends()):
70+
...
71+
72+
```
73+
74+
In this example, we depend on the current application. We can use its state in a current task or any other dependency. We can take db_pool from your application's state, which is the same pool, as the one you've created on AiohTTP's startup.
75+
But this application is only a mock of your application. It has correct types and all your variables that you filled on startup, but it doesn't handle any request.
76+
This integration adds two main dependencies:
77+
* web.Application - current application.
78+
* web.Request - mocked request. This request only exists to be able to use the same dependencies.
79+
80+
You can find more detailed examples in the [examples repo](https://github.com/taskiq-python/examples).
81+
82+
## Testing
83+
84+
Writing tests for AioHTTP with taskiq is as easy as writing tests for the aiohttp application. The only difference is that, if you want to use InMemoryBroker, then you need to add context for dependency injection. It's easier to call `populate_context` when creating a `test_client` fixture.
85+
86+
```python
87+
import taskiq_aiohttp
88+
89+
@pytest.fixture
90+
async def test_client(
91+
app: web.Application,
92+
) -> AsyncGenerator[TestClient, None]:
93+
"""
94+
Create a test client.
95+
96+
This function creates a TestServer
97+
and a test client for the application.
98+
99+
Also this fixture populates context
100+
with needed variables.
101+
102+
:param app: current application.
103+
:yield: ready to use client.
104+
"""
105+
loop = asyncio.get_running_loop()
106+
server = TestServer(app)
107+
client = TestClient(server, loop=loop)
108+
109+
await client.start_server()
110+
111+
# This is important part.
112+
# Since InMemoryBroker doesn't
113+
# run as a worker process, we have to populate
114+
# broker's context by hand.
115+
taskiq_aiohttp.populate_context(
116+
broker=broker,
117+
server=server.runner.server,
118+
app=app,
119+
loop=loop,
120+
)
121+
122+
yield client
123+
124+
broker.custom_dependency_context = {}
125+
await client.close()
126+
```

0 commit comments

Comments
 (0)