Skip to content

Commit cb7c951

Browse files
authored
More documentation (#219)
* New Doc: docs/userguide/app-conf.rst * New Doc: docs/userguide/utils-module.rst * New Doc: docs/userguide/examples/myutils.rst * New Doc: docs/userguide/tasks.rst * New Doc: docs/userguide/signals.rst * New Doc: docs/faq.rst * Fixed readthedocs build failure
1 parent 4bb56fd commit cb7c951

File tree

16 files changed

+797
-48
lines changed

16 files changed

+797
-48
lines changed

.readthedocs.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ build:
1313
post_create_environment:
1414
# Install poetry
1515
# https://python-poetry.org/docs/#installing-manually
16-
- pip install poetry
16+
- pip install poetry==1.7.1
1717
# Tell poetry to not use a virtual environment
1818
- poetry config virtualenvs.create false
1919
post_install:

docs/conf.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@
3030
r"^http://localhost",
3131
r"https://github\.com/Jc2k/pytest-docker-tools\?tab=readme-ov-file#images",
3232
r"https://github\.com/Jc2k/pytest-docker-tools\?tab=readme-ov-file#containers",
33+
r"https://github\.com/Jc2k/pytest-docker-tools\?tab=readme-ov-file#fixture-wrappers",
3334
],
3435
autodoc_mock_imports=[],
3536
)

docs/faq.rst

Lines changed: 158 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -7,21 +7,168 @@
77
.. contents::
88
:local:
99

10-
.. _faq-general:
10+
Getting Started
11+
===============
1112

12-
General
13-
=======
13+
What are the prerequisites for installing pytest-celery?
14+
--------------------------------------------------------
1415

15-
.. _faq-tbd-1:
16+
**Answer:** The celery package and the required dependencies for the used :ref:`vendors`.
1617

17-
TBD 1?
18-
------
18+
How do I install pytest-celery?
19+
-------------------------------
1920

20-
**Answer:** Yes/No.
21+
**Answer:** See :ref:`installation`.
2122

22-
.. _faq-tbd-2:
23+
What initial configuration is required to start using pytest-celery?
24+
--------------------------------------------------------------------
2325

24-
TBD 2?
25-
------
26+
**Answer:** Generally speaking, everything is set up by default. However, you may need to tweak
27+
some settings to fit your specific use case.
2628

27-
**Answer:** Yes/No.
29+
For new projects, it is recommended to at least start with a ``pytest.ini`` file at the root of your project
30+
with the following content:
31+
32+
.. code-block:: ini
33+
34+
[pytest]
35+
log_cli = true
36+
log_cli_level = INFO
37+
log_cli_format = %(asctime)s [%(levelname)8s] %(message)s (%(filename)s:%(lineno)s)
38+
log_cli_date_format = %Y-%m-%d %H:%M:%S
39+
40+
This will enable plugin logging to the console, which can be helpful for debugging.
41+
42+
Are there any example projects or templates to help me get started?
43+
-------------------------------------------------------------------
44+
45+
**Answer:** Yes. See the :ref:`examples` section.
46+
47+
In addition, you may review the `official Celery smoke tests <https://github.com/celery/celery/tree/main/t/smoke>`_
48+
which are the defacto production environment for the plugin.
49+
50+
Configuration and Customization
51+
===============================
52+
53+
Can I use pytest-celery with different message brokers or backends?
54+
-------------------------------------------------------------------
55+
56+
**Answer:** Yes. The built-in :ref:`vendors` are supported out of the box, but you can also use custom ones,
57+
or reconfigure the built-in ones to fit your needs.
58+
59+
Vendors are different technologies that provides components to the environment.
60+
Such components may be brokers, backends or worker components, which are constructed by a node controller
61+
and a docker container combination.
62+
63+
If you provide your own component (broker/backend/worker) using your own docker image, you may inject
64+
the component into the environment as described in the :ref:`architecture-injection` section.
65+
66+
How can I manage worker concurrency settings in pytest-celery?
67+
--------------------------------------------------------------
68+
69+
**Answer:** Using the :func:`default_worker_app <pytest_celery.vendors.worker.fixtures.default_worker_app>` fixture.
70+
71+
.. code-block:: python
72+
73+
@pytest.fixture
74+
def default_worker_app(default_worker_app: Celery) -> Celery:
75+
app = default_worker_app
76+
app.conf.worker_concurrency = 42
77+
return app
78+
79+
For more details, see :ref:`worker-app-configuration`.
80+
81+
How do I simulate different environments using pytest-celery?
82+
-------------------------------------------------------------
83+
84+
**Answer:** See :ref:`manipulating-the-environment`.
85+
86+
Debugging and Troubleshooting
87+
=============================
88+
89+
Why doesn't the worker recognize my tasks?
90+
------------------------------------------
91+
92+
**Answer:** Because you don't use the :func:`default_worker_tasks <pytest_celery.vendors.worker.fixtures.default_worker_tasks>` fixture.
93+
94+
.. code-block:: python
95+
96+
@pytest.fixture
97+
def default_worker_tasks(default_worker_tasks: set) -> set:
98+
from tests import tasks
99+
100+
default_worker_tasks.add(tasks)
101+
return default_worker_tasks
102+
103+
For more details, see :ref:`injecting-tasks`.
104+
105+
Why aren't my consumer signal handlers triggering?
106+
--------------------------------------------------
107+
108+
**Answer:** Because you don't use the :func:`default_worker_signals <pytest_celery.vendors.worker.fixtures.default_worker_signals>` fixture.
109+
110+
.. code-block:: python
111+
112+
@pytest.fixture
113+
def default_worker_signals(default_worker_signals: set) -> set:
114+
from tests import signals
115+
116+
default_worker_signals.add(signals)
117+
yield default_worker_signals
118+
119+
For more details, see :ref:`injecting-signals-handlers`.
120+
121+
What should I do if the Celery worker doesn't start?
122+
----------------------------------------------------
123+
124+
**Answer:** This is most probably due to docker build failure.
125+
126+
1. Try to build your worker image manually to see if there are any errors. Make sure to use the same arguments as the plugin.
127+
2. If it does, then it is probably due to incorrect setup configuration. Review your fixtures usage.
128+
129+
How can I manually inspect my running setup during a test execution?
130+
--------------------------------------------------------------------
131+
132+
**Answer:** You may place a breakpoint in your test and inspect the environment using any standard tool.
133+
134+
If possible, the `Docker Desktop <https://www.docker.com/products/docker-desktop/>`_ is very helpful for inspecting
135+
the running containers during the test execution.
136+
137+
Integrating with Docker
138+
=======================
139+
140+
How do I get pytest-celery to work with Docker?
141+
-----------------------------------------------
142+
143+
**Answer:** The engine behind the plugin's docker integration is :pypi:`pytest-docker-tools`.
144+
145+
It does not interact with Docker directly.
146+
147+
The Docker environment should be install normally, regardless of the plugin.
148+
149+
How can I clean up Docker artifacts left after a test run?
150+
----------------------------------------------------------
151+
152+
**Answer:** You may use this snippet from the ``tox -e clean`` environment.
153+
154+
.. literalinclude:: ../tox.ini
155+
:language: ini
156+
:caption: tox.ini
157+
:start-after: make -C ./docs clean
158+
:end-before: [testenv:docs]
159+
160+
What are the common pitfalls when integrating pytest-celery with Docker, and how can I avoid them?
161+
--------------------------------------------------------------------------------------------------
162+
163+
**Answer:** The most common issues encountered so far are limited docker resources and network issues.
164+
165+
To avoid these, you may:
166+
167+
1. Increase the resources available to Docker.
168+
2. Use the :pypi:`pytest-rerunfailures` pytest plugin to retry failed tests with:
169+
170+
.. code-block:: bash
171+
172+
--reruns 5 --reruns-delay 60 --rerun-except AssertionError
173+
174+
Experiment with the values to find the best fit for your environment.

docs/getting-started/first-steps.rst

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -340,6 +340,8 @@ Before we can learn how to fit the environment to our needs, let's have a quick
340340
* - **Isoalted Environments**
341341
- Each test case has its own instances of the environment, allowing for parallelism and isolation of test cases.
342342

343+
.. _manipulating-the-environment:
344+
343345
Manipulating the Environment
344346
============================
345347

@@ -763,7 +765,7 @@ Tasks
763765
-----
764766

765767
This will be our ``tasks.py`` file. It adds a simple ``noop`` task
766-
`Using the @shared_task decorator <https://docs.celeryq.dev/en/main/django/first-steps-with-django.html#using-the-shared-task-decorator>`_.
768+
`Using the @shared_task decorator <https://docs.celeryq.dev/en/stable/django/first-steps-with-django.html#using-the-shared-task-decorator>`_.
767769

768770
.. code-block:: python
769771

docs/getting-started/next-steps.rst

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -154,8 +154,8 @@ For example, such ``signals.py`` module might look like this:
154154
Celery app
155155
~~~~~~~~~~
156156

157-
The provided :ref:`app.py <content-app>` uses the `config_from_object() <https://docs.celeryq.dev/en/main/userguide/application.html#config-from-object>`_
158-
and `app.conf.changes <https://docs.celeryq.dev/en/main/userguide/application.html#configuration>`_ to transmit the configuration
157+
The provided :ref:`app.py <content-app>` uses the `config_from_object() <https://docs.celeryq.dev/en/stable/userguide/application.html#config-from-object>`_
158+
and `app.conf.changes <https://docs.celeryq.dev/en/stable/userguide/application.html#configuration>`_ to transmit the configuration
159159
from the test environment to the worker container.
160160

161161
The ``app.conf`` is set like this when the worker is booting up:

0 commit comments

Comments
 (0)