Skip to content

Commit 05d9ca9

Browse files
authored
Clarify OAProc configuration in docs (geopython#2127)
1 parent 20ddc0a commit 05d9ca9

File tree

3 files changed

+180
-160
lines changed

3 files changed

+180
-160
lines changed

docs/source/data-publishing/index.rst

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
.. _data-publishing:
22

3-
Data publishing
4-
===============
3+
Resource publishing
4+
===================
55

66
Let's start working on integrating your data into pygeoapi. pygeoapi provides the capability to
77
publish vector/coverage data, processes, catalogues, and exposing filesystems of geospatial data.
@@ -22,11 +22,24 @@ return back data to the pygeoapi API framework in a plug and play fashion.
2222
ogcapi-coverages
2323
ogcapi-maps
2424
ogcapi-tiles
25-
ogcapi-processes
2625
ogcapi-records
2726
ogcapi-edr
2827
stac
2928

3029

30+
Processes overview
31+
------------------
32+
33+
In addition to data publishing, pygeoapi also provides the capability to publish
34+
processes that can be executed via the OGC API - Processes standard.
35+
36+
.. toctree::
37+
:maxdepth: 3
38+
:caption: Process publishing
39+
:name: Process publishing
40+
41+
ogcapi-processes
42+
43+
3144
.. seealso::
3245
:ref:`configuration` for more information on publishing hidden resources.

docs/source/data-publishing/ogcapi-processes.rst

Lines changed: 93 additions & 157 deletions
Original file line numberDiff line numberDiff line change
@@ -13,152 +13,94 @@ fashion (inputs, outputs).
1313
pygeoapi implements OGC API - Processes functionality by providing a plugin architecture, thereby
1414
allowing developers to implement custom processing workflows in Python.
1515

16-
The pygeoapi offers two processes: a default ``hello-world`` process which allows you to quickly explore the capabilities of processes, and an optional ``shapely-functions`` process with more advanced features that leverages `Shapely`_ to expose various geometric processing functionality.
17-
1816
Configuration
1917
-------------
2018

21-
The below configuration is an example of a process defined within the pygeoapi internal plugin registry:
19+
Processes are configured in the resources section of the pygeoapi configuration file.
20+
pygeoapi offers two default processes:
21+
22+
Hello World
23+
^^^^^^^^^^^
24+
25+
``HelloWorld`` is a process with a simple input and output to demonstrate the process concept.
2226

2327
.. code-block:: yaml
2428
25-
processes:
26-
# enabled by default
27-
hello-world:
28-
processor:
29-
name: HelloWorld
29+
hello-world:
30+
type: process
31+
processor:
32+
name: HelloWorld
3033
31-
The below configuration is an example of a process defined as part of a custom Python process:
34+
Shapely Functions
35+
^^^^^^^^^^^^^^^^^
3236

33-
.. code-block:: yaml
37+
``ShapelyFunctionsProcessor`` is process with more advanced features that leverages `Shapely`_
38+
to expose various geometric processing functionality.
39+
The selection cut across different operations in Shapely.
40+
To avoid function collision, it uses the name of the function category as the namespace.
41+
E.g *union* operation under the *set* module is described as *set:union*.
3442

35-
processes:
36-
# enabled by default
37-
hello-world:
38-
processor:
39-
# refer to a process in the standard PYTHONPATH
40-
# e.g. my_package/my_module/my_file.py (class MyProcess)
41-
# the MyProcess class must subclass from pygeoapi.process.base.BaseProcessor
42-
name: my_package.my_module.my_file.MyProcess
43+
The process is configured to accept a list of geometry *inputs* (WKT and/or GeoJSON geometry),
44+
*operation* and an optional *output_format*.
45+
It performs the specified operation and returns the result in the specified *output_format*
46+
(If the operation does not return a geometry, then this is ignored).
4347

44-
Documenting process metadata
45-
----------------------------
46-
47-
When defining a process, various metadata must be supplied in order to enable discovery and description
48-
of inputs and outputs. The metadata is realized by a Python dictionary in a given process that is
49-
supplied to process initialization at runtime.
50-
51-
Below is a sample process definition as a Python dictionary:
52-
53-
.. code-block:: python
54-
55-
PROCESS_METADATA = {
56-
'version': '0.2.0', # the version of the process
57-
'id': 'hello-world', # process identifier
58-
'title': 'Hello World', # process title, can also be multilingual
59-
'description': 'An example process that takes a name as input, and echoes ' # process description, can also be multilingual
60-
'it back as output. Intended to demonstrate a simple '
61-
'process with a single literal input.',
62-
'jobControlOptions': ['sync-execute', 'async-execute'], # whether the process can be executed in sync or async mode
63-
'keywords': ['hello world', 'example', 'echo'], # keywords associated with the process
64-
'links': [{ # a list of 1..n # link objects relevant to the process
65-
'type': 'text/html',
66-
'rel': 'about',
67-
'title': 'information',
68-
'href': 'https://example.org/process',
69-
'hreflang': 'en-US'
70-
}],
71-
'inputs': { # process inputs (one key per input), structured as JSON Schema
72-
'name': {
73-
'title': 'Name',
74-
'description': 'The name of the person or entity that you wish to'
75-
'be echoed back as an output',
76-
'schema': {
77-
'type': 'string'
78-
},
79-
'minOccurs': 1,
80-
'maxOccurs': 1,
81-
'keywords': ['full name', 'personal']
82-
},
83-
'message': {
84-
'title': 'Message',
85-
'description': 'An optional message to echo as well',
86-
'schema': {
87-
'type': 'string'
88-
},
89-
'minOccurs': 0,
90-
'maxOccurs': 1,
91-
'keywords': ['message']
92-
}
93-
},
94-
'outputs': { # outputs
95-
'echo': { # an identifier for the output
96-
'title': 'Hello, world',
97-
'description': 'A "hello world" echo with the name and (optional)'
98-
' message submitted for processing',
99-
'schema': { # output definition, structured as JSON Schema
100-
'type': 'object',
101-
'contentMediaType': 'application/json'
102-
}
103-
}
104-
},
105-
'example': { # example request payload
106-
'inputs': {
107-
'name': 'World',
108-
'message': 'An optional message.',
109-
}
110-
}
111-
}
48+
.. code-block:: yaml
11249
113-
See :ref:`example-custom-pygeoapi-processing-plugin` for full processing plugin examples.
50+
shapely-functions:
51+
type: process
52+
processor:
53+
name: ShapelyFunctionsProcessor
11454
115-
Processing and response handling
116-
--------------------------------
55+
**Supported operations**
11756

118-
pygeoapi processing plugins must return a tuple of media type and native outputs. Multipart
119-
responses are not supported at this time, and it is up to the process plugin implementor to return a single
120-
payload defining multiple artifacts (or references to them).
57+
* `measurement:bounds` - Computes the bounds (extent) of a geometry.
58+
* `measurement:area` - Computes the area of a (multi)polygon.
59+
* `measurement:distance` - Computes the Cartesian distance between two geometries.
60+
* `predicates:covers` - Returns True if no point in geometry B is outside geometry A.
61+
* `predicates:within` - Returns True if geometry A is completely inside geometry B.
62+
* `set:difference` - Returns the part of geometry A that does not intersect with geometry B.
63+
* `set:union` - Merges geometries into one.
64+
* `constructive:buffer` - Computes the buffer of a geometry for positive and negative buffer distance.
65+
* `constructive:centroid` - Computes the geometric center (center-of-mass) of a geometry.
66+
67+
.. note::
12168

122-
By default (or via the OGC API - Processes ``response: raw`` execution parameter), pygeoapi provides
123-
processing responses in their native encoding and media type, as defined by a given
124-
plugin (which needs to set the response content type and payload accordingly).
69+
There is no support for passing optional function arguments yet.
70+
For example, when computing buffer on a geometry, there is no option to pass in the buffer distance.
12571

126-
pygeoapi also supports a JSON-based response type (via the OGC API - Processes ``response: document``
127-
execution parameter). When this mode is requested, the response will always be a JSON encoding, embedding
128-
the resulting payload (part of which may be Base64 encoded for binary data, for example).
72+
Custom processes
73+
^^^^^^^^^^^^^^^^
12974

75+
The below configuration is an example of a process defined as part of a custom Python process:
13076

131-
Asynchronous support
132-
--------------------
77+
.. code-block:: yaml
13378
134-
By default, pygeoapi implements process execution (jobs) as synchronous mode. That is, when
135-
jobs are submitted, the process is executed and returned in real-time. Certain processes
136-
that may take time to execute, or be delegated to a scheduler/queue, are better suited to
137-
an asynchronous design pattern. This means that when a job is submitted in asynchronous
138-
mode, the server responds immediately with a reference to the job, which allows the client
139-
to periodically poll the server for the processing status of a given job.
79+
my-process:
80+
type: process
81+
processor:
82+
name: HelloWorld
83+
# refer to a process in the standard PYTHONPATH
84+
# e.g. my_package/my_module/my_file.py (class MyProcess)
85+
# the MyProcess class must subclass from pygeoapi.process.base.BaseProcessor
86+
name: my_package.my_module.my_file.MyProcess
14087
141-
In keeping with the OGC API - Processes specification, asynchronous process execution
142-
can be requested by including the ``Prefer: respond-async`` HTTP header in the request.
14388
144-
Job management is required for asynchronous functionality.
89+
See :ref:`example-custom-pygeoapi-processing-plugin` for full processing plugin examples.
14590

146-
Job management
147-
--------------
91+
Job managers
92+
------------
14893

14994
pygeoapi provides job management by providing a 'manager' concept which, well,
15095
manages job execution. The manager concept is implemented as part of the pygeoapi
15196
:ref:`plugins` architecture. pygeoapi provides a default manager implementation
15297
based on `TinyDB`_ for simplicity. Custom manager plugins can be developed for more
15398
advanced job management capabilities (e.g. Kubernetes, databases, etc.).
15499

155-
Job managers
156-
------------
157-
158100
TinyDB
159101
^^^^^^
160102

161-
TinyDB is the default job manager for pygeoapi when enabled.
103+
TinyDB is a local file-system based job manager for pygeoapi when enabled.
162104

163105
.. code-block:: yaml
164106
@@ -207,20 +149,48 @@ The connection to a `PostgreSQL`_ database must be provided in the configuration
207149
# connection: postgresql://postgres:${POSTGRESQL_PASSWORD:-postgres}@localhost:5432/test
208150
output_dir: /tmp
209151
152+
Asynchronous support
153+
--------------------
154+
155+
By default, pygeoapi implements process execution (jobs) as synchronous mode. That is, when
156+
jobs are submitted, the process is executed and returned in real-time. Certain processes
157+
that may take time to execute, or be delegated to a scheduler/queue, are better suited to
158+
an asynchronous design pattern. This means that when a job is submitted in asynchronous
159+
mode, the server responds immediately with a reference to the job, which allows the client
160+
to periodically poll the server for the processing status of a given job.
161+
162+
In keeping with the OGC API - Processes specification, asynchronous process execution
163+
can be requested by including the ``Prefer: respond-async`` HTTP header in the request.
164+
165+
.. note::
166+
Job management is required for asynchronous functionality.
167+
168+
Processing and response handling
169+
--------------------------------
170+
171+
pygeoapi processing plugins must return a tuple of media type and native outputs. Multipart
172+
responses are not supported at this time, and it is up to the process plugin implementor to return a single
173+
payload defining multiple artifacts (or references to them).
210174

211-
Putting it all together
212-
-----------------------
175+
By default (or via the OGC API - Processes ``response: raw`` execution parameter), pygeoapi provides
176+
processing responses in their native encoding and media type, as defined by a given
177+
plugin (which needs to set the response content type and payload accordingly).
178+
179+
pygeoapi also supports a JSON-based response type (via the OGC API - Processes ``response: document``
180+
execution parameter). When this mode is requested, the response will always be a JSON encoding, embedding
181+
the resulting payload (part of which may be Base64 encoded for binary data, for example).
182+
183+
Processing examples
184+
-------------------
213185

214186
To summarize how pygeoapi processes and managers work together:
215187

216188
* process plugins implement the core processing / workflow functionality
217189
* manager plugins control and manage how processes are executed
218190

219-
Processing examples
220-
-------------------
221191

222-
Hello World (Default)
223-
^^^^^^^^^^^^^^^^^^^^^
192+
Hello World
193+
^^^^^^^^^^^
224194

225195
.. code-block:: sh
226196
@@ -259,40 +229,8 @@ Hello World (Default)
259229
-d "{\"inputs\":{\"name\": \"hi there2\"}, \
260230
\"subscriber\": {\"successUri\": \"https://www.example.com/success\"}}"
261231
262-
Shapely Functions (Optional)
263-
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
264-
265-
The `shapely-functions` process exposes some selected Shapely_ functions as sample process. The selection cut across different operations in Shapely. To avoid function collision, it uses the name of the function category as the namespace. E.g *union* operation under the *set* module is described as *set:union*.
266-
267-
The process is configured to accept a list of geometry *inputs* (WKT and/or GeoJSON geometry), *operation* and an optional *output_format*. It performs the specified operation and returns the result in the specified *output_format* (If the operation does not return a geometry, then this is ignored).
268-
269-
270-
Configuration
271-
-------------
272-
273-
.. code-block:: yaml
274-
275-
processes:
276-
shapely-functions:
277-
processor:
278-
name: ShapelyFunctions
279-
280-
281-
**Supported operations**
282-
283-
* **measurement:bounds** - Computes the bounds (extent) of a geometry.
284-
* **measurement:area** - Computes the area of a (multi)polygon.
285-
* **measurement:distance** - Computes the Cartesian distance between two geometries.
286-
* **predicates:covers** - Returns True if no point in geometry B is outside geometry A.
287-
* **predicates:within** - Returns True if geometry A is completely inside geometry B.
288-
* **set:difference** - Returns the part of geometry A that does not intersect with geometry B.
289-
* **set:union** - Merges geometries into one.
290-
* **constructive:buffer** - Computes the buffer of a geometry for positive and negative buffer distance.
291-
* **constructive:centroid** - Computes the geometric center (center-of-mass) of a geometry.
292-
293-
**Limitation**
294-
295-
There is no support for passing optional function arguments yet. E.g when computing buffer on a geometry, no option to pass in the buffer distance.
232+
Shapely Functions
233+
^^^^^^^^^^^^^^^^^
296234

297235
.. code-block:: sh
298236
@@ -331,7 +269,5 @@ There is no support for passing optional function arguments yet. E.g when comput
331269
332270
333271
.. _`OGC API - Processes`: https://ogcapi.ogc.org/processes
334-
.. _`sample`: https://github.com/geopython/pygeoapi/blob/master/pygeoapi/process/hello_world.py
335-
.. _`shapely_functions`: https://github.com/geopython/pygeoapi/blob/master/pygeoapi/process/shapely_functions.py
336272
.. _`TinyDB`: https://tinydb.readthedocs.io/en/latest
337273
.. _`Shapely`: https://shapely.readthedocs.io/

0 commit comments

Comments
 (0)