Merge pull request #135 from ESA-APEx/openeo_lifecycle

upd_writer(#61): added development flow

Author: JanssenBrm

guides/udp_writer_guide.qmd

@@ -2,6 +2,51 @@
 title: Guide for writing openEO User Defined Processes
 ---
 
+## Development Flow
+
+The development of an openEO User Defined Process (UDP) starts with the creation of a process graph. This
+graph is a visual representation of the data processing workflow, which can be built using the openEO API and its tools.
+The graph consists of nodes, each representing a specific process or operation, and edges that define the flow of data
+between these nodes.
+
+To start with the development of an openEO process graph, you need to select an openEO backend that provides access to the
+data you want to process. The backend is responsible for executing the process graph and returning the results. Once you
+have selected a backend, you can begin building your process graph by using the existing tools and libraries provided by
+openEO. These tools allow you to create, modify, and visualize your process graph in a user-friendly manner. Once you have
+created your process graph, you can test it on the selected openEO backend to ensure that it works as expected. This step
+is crucial, as it allows you to identify any issues or errors in your process graph before packaging it as a User Defined
+Process (UDP).
+
+Once your process graph is complete, you can package it into a an openEO UDP, enabling it to be executed by
+other project members or the broader community. As a final step, you may onboard the service onto the ESA NoR,
+establishing a cost model and additional revenue stream for service execution.
+
+The full development flow for creating and sharing an openEO UDP is summarized below:
+
+1. **Create an Account**: Create an account with an existing openEO API provider, preferably an\
+   [APEx-compliant hosting platform](../propagation/platforms.md). This is where the UDP will be hosted.
+2. **Create the Process Graph**: Develop a processing graph that represents your workflow using the existing tools provided\
+by your chosen.  For users with an existing Python implementation, openEO offers flexibility in developing their process\
+graph. They can:
+    a. Utilize predefined openEO processes: map your existing Python functions to equivalent openEO operations.
+    b. Create [User Defined Functions (UDF)](https://open-eo.github.io/openeo-python-client/udf.html): Create custom\
+    functions to be executed by openEO.
+3. **Test the Process Graph**: Test the graph on your selected platform.
+4. **Encapsulate as a UDP**: The process graph is then encapsulated as a UDP, complete with metadata, as described in the\
+   next section.
+5. **Test the UDP**: Test the UDP on one or more openEO-based platforms to validate its functionality.
+6. **Share the UDP**: Once validated, the UDP is shared as a JSON definition for broader use.
+7. **Register the UDP on ESA NoR**: Define a cost model, making the service available on the ESA Network of
+   Resources.
+
+::: {.callout-tip}
+Keep in mind that APEx offers [Algorithm Porting](../propagation/porting.md)
+and [Algorithm Onboarding](../propagation/onboarding.md) support to help you with transforming your algorithm into an
+openEO UDP, onboarding it onto an [APEx-compliant hosting platform](../propagation/platforms.md), registering it in the
+[APEx Algorithm Services Catalogue](../propagation/onboarding.md#apex-algorithm-services-catalogue) and its onboarding onto
+the [ESA Network of Resources](https://portfolio.nor-discover.org/).
+:::
+
 ## Creating an openEO User Defined Process {#sec-udp-writing}
 
 User defined processes (UDPs) are one out of two standardised options that APEx offers to publish algorithms as a
@@ -13,15 +58,11 @@ the[APEx Algorithm Services Catalogue](../propagation/onboarding.md#apex-algorit
 For more background on UDP's, or a basic tutorial on creating them, the open source Python client provides
 [a good starting point](https://open-eo.github.io/openeo-python-client/udp.html).
 
-Keep in mind that APEx offers [Algorithm Porting](../propagation/porting.md)
-and [Algorithm Onboarding](../propagation/onboarding.md) support to help you with transforming your algorithm into an
-openEO UDP and onboarding it onto an [APEx-compliant hosting platform](../propagation/platforms.md).
-
 ### Example cases
 
 The best way to learn how to write a UDP is to look at existing examples:
 
-- `max_ndvi_composite`: [UDP code](https://github.com/ESA-APEx/apex_algorithms/tree/main/algorithm_catalog/vito/max_ndvi_composite)
+* `max_ndvi_composite`: [UDP code](https://github.com/ESA-APEx/apex_algorithms/tree/main/algorithm_catalog/vito/max_ndvi_composite)
 and [description](https://github.com/ESA-APEx/apex_algorithms/blob/main/algorithm_catalog/vito/max_ndvi_composite/openeo_udp/max_ndvi_composite_description.md).
 
 ### Organizing your code
@@ -39,15 +80,16 @@ more discoverable.
 APEx requires you to use versioning, to ensure that changes in your algorithm are clearly communicated and visible to users.
 A good way to do this is to use [semantic versioning](https://semver.org/), which is a widely used versioning scheme.
 
-In combination with git tags, this allows you to easily track different versions of your UDP JSON, and share immutable links to
-specific versions. In addition, you can also create tags that always point to the latest development or stable version.
+In combination with git tags, this allows you to easily track different versions of your UDP JSON, and share immutable
+links to specific versions. In addition, you can also create tags that always point to the latest development or stable version.
 
 One use case is an algorithm change that requires you to change parameters. We refer to such a change as 'backward incompatible',
 because users of the existing algorithm will not be able to switch to the new version without an update on their side.
 In such a case, you can create a new UDP version, and keep the old one available for users that are not ready to switch yet.
-Also in the UDP catalog, we recommend to keep both versions side-by-side for a certain time frame before removing the old one.
+Also in the UDP catalog, we recommend to keep both versions side-by-side for a certain time frame before removing the old
+one.
 
-A changelog is also required, to document the changes between versions. Guidance on how to keep changelogs can be found 
+A changelog is also required, to document the changes between versions. Guidance on how to keep changelogs can be found
 [online](https://keepachangelog.com/en/1.1.0/).
 
 It is not mandatory to store your UDP JSON in the APEx git repository, especially if the project already maintains an open
@@ -84,8 +126,8 @@ can even be left empty (`null`).
 For maximum flexibility, compatibility and the best user experience, we recommend to perform spatial filtering in your
 algorithm:
 
-- directly in `load_collection`/`load_stac` (instead of separate `filter_bbox`/`filter_spatial` processes)
-- using a parameter named `spatial_extent` (unless there are good reasons otherwise)
+* directly in `load_collection`/`load_stac` (instead of separate `filter_bbox`/`filter_spatial` processes)
+* using a parameter named `spatial_extent` (unless there are good reasons otherwise)
   spatial filtering in these processes, and also use `spatial_extent` as the parameter name unless there are good
   reasons otherwise.
 
@@ -114,7 +156,7 @@ generated.
 
 ### Include default parameters within the UDP
 
-Via the [@openeo_processing_parameters] extension, platform specific parameters (also known as `job options`) can be 
+Via the [@openeo_processing_parameters] extension, platform specific parameters (also known as `job options`) can be
 included directly in the process definition. While the primary recommendation is to avoid using platform specific parameters,
 it is nevertheless a very good idea to include them when they are necessary.
 
@@ -155,23 +197,23 @@ the [APEx Algorithm Catalogue GitHub repository](https://github.com/ESA-APEx/ape
 
 An example json is provided below. The properties to modify are listed here:
 
-- *id*: a unique identifier for your process
-- *created* and *updated* timestamps
-- *title*: a descriptive title
-- *description*: a detailed description of the process
-- *contacts*: a list of contacts, with at least one principal investigator
-- *keywords*: a list of free form keywords
-- *themes*: Applicable concepts from a scheme. Concepts can be found in
+* *id*: a unique identifier for your process
+* *created* and *updated* timestamps
+* *title*: a descriptive title
+* *description*: a detailed description of the process
+* *contacts*: a list of contacts, with at least one principal investigator
+* *keywords*: a list of free form keywords
+* *themes*: Applicable concepts from a scheme. Concepts can be found in
   the [ESA Data Ontology](https://data.esa.int/esado)
-- *license*: the license under which the process is published. You can use
+* *license*: the license under which the process is published. You can use
   the [SPDX license list](https://spdx.org/licenses/) for this. Proprietary licenses are possible, within the terms of
   your ESA project.
 
 The *links* section is crucial, the following "rel" values are mandatory:
 
-- *openeo-process*: exactly one link to the JSON document that defines the process
-- *service*: at least one link to an openEO backend that is able to execute the process
-- *example*: at least one link to a STAC metadata file that shows the output of the process
+* *openeo-process*: exactly one link to the JSON document that defines the process
+* *service*: at least one link to an openEO backend that is able to execute the process
+* *example*: at least one link to a STAC metadata file that shows the output of the process
 
 The *type* field of these links should be set to `application/json`. Please provide a descriptive *title* for each link,
 allowing to understand what the link is about.
@@ -334,15 +376,15 @@ Example benchmark definition:
 
 Note how each benchmark scenario references:
 
-- the target openEO backend to use.
-- an openEO process graph to execute.
+* the target openEO backend to use.
+* an openEO process graph to execute.
   The process graph will typically just contain a single node
   pointing with the `namespace` field to the desired process definition
   at a URL, following
   the [remote process definition extension](https://github.com/Open-EO/openeo-api/tree/draft/extensions/remote-process-definition).
   The URL will typically be a raw GitHub URL to the JSON file in the `openeo_udp` folder, but it can also be a URL to a
   different location.
-- reference data to which actual results should be compared.
+* reference data to which actual results should be compared.
 
 #### Defining and updating benchmark reference data