Skip to content

Commit 888e1b6

Browse files
DarkaMaulwoodruffwmiketheman
authored
Documentation on how to implement a new service (#16595)
* Add documentation on how to add a service to warehouse. * Fix to expose documentation if built within docker * Apply suggestions from code review Co-authored-by: William Woodruff <[email protected]> * Update documentation to include service registration factories * Fix wrapping issue * Update with remarks. * Apply suggestions from code review Co-authored-by: Mike Fiedler <[email protected]> * Add a flag in the Makefile --------- Co-authored-by: William Woodruff <[email protected]> Co-authored-by: Mike Fiedler <[email protected]>
1 parent f3dcfcc commit 888e1b6

File tree

2 files changed

+109
-3
lines changed

2 files changed

+109
-3
lines changed

docker-compose.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -259,9 +259,12 @@ services:
259259
context: .
260260
target: docs
261261
image: warehouse:docker-compose-docs
262+
command: sphinx-autobuild --host 0.0.0.0 "docs/dev/" "docs/dev/_build"
262263
volumes:
263264
- ./bin:/opt/warehouse/src/bin:z
264265
- ./docs:/opt/warehouse/src/docs:z
266+
ports:
267+
- "10002:8000"
265268

266269
user-docs:
267270
image: warehouse:docker-compose-docs

docs/dev/development/patterns.rst

Lines changed: 106 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
Patterns
2-
========
2+
********
33

44
Dependency management
5-
---------------------
5+
=====================
66

77
Warehouse's approach to dependency management can be summarized as follows:
88

@@ -82,7 +82,7 @@ process for adding new dependencies:
8282
3. Commit the changes
8383

8484
Returning vs Raising HTTP Exceptions
85-
------------------------------------
85+
====================================
8686

8787
Pyramid allows the various HTTP Exceptions to be either returned or raised,
8888
and the difference between whether you return or raise them are subtle. The
@@ -107,6 +107,109 @@ Class Method
107107
``HTTPServerError`` (5xx) Raise
108108
========================= ==================================
109109

110+
Implementing new services
111+
=========================
112+
113+
Warehouse uses services to provide pluggable functionalities within the codebase. They are implemented using
114+
`pyramid-services`_. After being registered, services are accessible using the ``find_service`` method of the
115+
``request`` object.
116+
117+
When adding new services to ``warehouse``, the following checklist serves as a comprehensive guideline to ensure
118+
you stay on track.
119+
120+
Adding a new service
121+
~~~~~~~~~~~~~~~~~~~~~
122+
123+
1. Create an Interface for the service. The interface serves as the baseline of the new service (design by
124+
contract pattern) and details all methods and attributes shared by the different service implementations.
125+
126+
Warehouse uses `zope.interface`_ to define interfaces. The interfaces are usually declared in a file named
127+
``interfaces.py`` within the relevant component, such as ``packaging/interfaces.py``.
128+
129+
2. Create the new service. The service must define all methods and attributes declared in the interface.
130+
This implementation contains the core logic of the service features. Additionally, services may add
131+
further methods that are not required on all implementations of the interface.
132+
133+
3. (Optional) Create other implementations of the interface. For instance, many services in ``warehouse``
134+
also provide a ``NullService`` version used for development. These Null implementations only
135+
provide basic functionalities without verifications and reduce the need for stubs in tests.
136+
137+
Any new implementation must implement the complete interface, including all its methods and attributes.
138+
139+
4. Implement each service creation method. If the Service is simple enough, use a class method in
140+
your service implementation (usually named ``create_service``). For more complex cases, implement
141+
a ``ServiceFactory`` class, responsible to create the service instance.
142+
143+
5. Register the service. The new service(s) must be registered to be available in the request object.
144+
145+
- If you have multiple services, create a new setting (in ``warehouse/config.py``) to select
146+
which backend to use.
147+
148+
- Add a default value for the setting in ``dev/environment`` for the development environment.
149+
150+
- Use the setting value in the ``includeme`` function to instantiate the appropriate service.
151+
152+
- Register your service factory. This registration must be in the service module's ``includeme``
153+
function for Pyramid to detect it and use the service factory created at the previous step.
154+
155+
6. (Optional) Add the new module to the ``warehouse/config.py``. If the new service is defined in a
156+
new module, add the new module within the warehouse ``configure`` function. This enrollment
157+
ensures Pyramid can detect it.
158+
159+
Using the service
160+
~~~~~~~~~~~~~~~~~
161+
162+
To use a service, query it using ``request.find_service()`` with the service interface. This
163+
method will return an instance of the service correctly selected based on the context and environment.
164+
165+
Example (from `packaging/utils.py`_):
166+
167+
.. code-block:: python
168+
169+
storage = request.find_service(ISimpleStorage)
170+
171+
172+
Testing the service
173+
~~~~~~~~~~~~~~~~~~~
174+
175+
Like the rest of the ``warehouse`` codebase, the new service requires tests. Below are some
176+
recommended practices for performing appropriate tests.
177+
178+
Testing the service itself
179+
^^^^^^^^^^^^^^^^^^^^^^^^^^
180+
181+
1. Implement a ``test_includeme`` function to test the service registration.
182+
2. Test each service implementation individually to meet ``warehouse`` 100% test coverage.
183+
184+
- Write a ``Test<ServiceName>`` class and implement ``test_interface_matches`` function (the exact name is irrelevant) to verify that the service implementation matches the interface definition using the ``verifyClass`` function from zope.
185+
186+
- Write appropriate test functions for the different methods.
187+
188+
3. Register the new service using its interface in ``tests/conftests.py``.
189+
4. (Optional) Modify ``tests/unit/test_config.py`` to check:
190+
191+
- If you have multiple services, that the new setting exists.
192+
- That the module registration works if your service is part of a new module.
193+
194+
5. (Optional) Depending on the needs, create a pytest fixture that returns the NullService
195+
and register it in the pyramid_services fixture.
196+
197+
Testing the service usage
198+
^^^^^^^^^^^^^^^^^^^^^^^^^
199+
200+
Except in the service tests, avoid mocking the service behavior and use the ``NullService``
201+
instead.
202+
203+
Example
204+
~~~~~~~
205+
206+
The following `Pull Request`_ can serve as a baseline as it implements all these steps.
207+
208+
110209
.. |pip-tools| replace:: ``pip-tools``
111210
.. _pip-tools: https://pypi.org/project/pip-tools/
211+
.. _`packaging/utils.py`: https://github.com/pypi/warehouse/blob/a36ae299d043bb4a770d6fd0f4e73b8e99dd6461/warehouse/packaging/utils.py#L122
112212
.. _Dependabot pull requests: https://github.com/pypi/warehouse/pulls?q=is%3Apr+is%3Aopen+label%3Adependencies
213+
.. _`pyramid-services`: https://github.com/mmerickel/pyramid_services
214+
.. _`zope.interface`: https://zopeinterface.readthedocs.io/
215+
.. _pull request: https://github.com/pypi/warehouse/pull/16546

0 commit comments

Comments
 (0)