renamed Spec and specification to Def and definition

Author: tclose

docs/changes.rst

@@ -108,7 +108,7 @@ Release Notes
 ---
 
 * big changes in ``ShellTask``, ``DockerTask`` and ``SingularityTask``
-    * customized input specification and output specification for ``Task``\s
+    * customized input definition and output definition for ``Task``\s
     * adding singularity checks to Travis CI
     * binding all input files to the container
 * changes in ``Workflow``

docs/components.rst

@@ -66,7 +66,7 @@ Shell Command Tasks
   The *Task* can accommodate more complex shell commands by allowing the user to
   customize inputs and outputs of the commands.
   One can generate an input
-  specification to specify names of inputs, positions in the command, types of
+  definition to specify names of inputs, positions in the command, types of
   the inputs, and other metadata.
   As a specific example, FSL's BET command (Brain
   Extraction Tool) can be called on the command line as:
@@ -76,7 +76,7 @@ Shell Command Tasks
     bet input_file output_file -m
 
   Each of the command argument can be treated as a named input to the
-  ``ShellCommandTask``, and can be included in the input specification.
+  ``ShellCommandTask``, and can be included in the input definition.
   As shown next, even an output is specified by constructing
   the *out_file* field form a template:
 
@@ -97,7 +97,7 @@ Shell Command Tasks
         ( "mask", bool,
           { "help_string": "create binary mask",
             "argstr": "-m", } ) ],
-        bases=(ShellSpec,) )
+        bases=(ShellDef,) )
 
     ShellCommandTask(executable="bet",
                      input_spec=bet_input_spec)

docs/input_spec.rst

@@ -5,7 +5,7 @@ Input Specification
 
 As it was mentioned in :ref:`shell_command_task`, the user can customize the input and output
 for the `ShellCommandTask`.
-In this section, more examples of the input specification will be provided.
+In this section, more examples of the input definition will be provided.
 
 
 Let's start from the previous example:
@@ -27,29 +27,29 @@ Let's start from the previous example:
         ( "mask", bool,
           { "help_string": "create binary mask",
             "argstr": "-m", } ) ],
-        bases=(ShellSpec,) )
+        bases=(ShellDef,) )
 
     ShellCommandTask(executable="bet",
                      input_spec=bet_input_spec)
 
 
 
-In order to create an input specification, a new `SpecInfo` object has to be created.
+In order to create an input definition, a new `SpecInfo` object has to be created.
 The field `name` specifies the type of the spec and it should be always "Input" for
-the input specification.
-The field `bases` specifies the "base specification" you want to use (can think about it as a
-`parent class`) and it will usually contains `ShellSpec` only, unless you want to build on top of
-your other specification (this will not be cover in this section).
+the input definition.
+The field `bases` specifies the "base definition" you want to use (can think about it as a
+`parent class`) and it will usually contains `ShellDef` only, unless you want to build on top of
+your other definition (this will not be cover in this section).
 The part that should be always customised is the `fields` part.
-Each element of the `fields` is a separate input field that is added to the specification.
+Each element of the `fields` is a separate input field that is added to the definition.
 In this example, three-elements tuples - with name, type and dictionary with additional
 information - are used.
 But this is only one of the supported syntax, more options will be described below.
 
-Adding a New Field to the Spec
+Adding a New Field to the Def
 ------------------------------
 
-Pydra uses `attr` classes to represent the input specification, and the full syntax for each field
+Pydra uses `attr` classes to represent the input definition, and the full syntax for each field
 is:
 
 .. code-block:: python
@@ -152,15 +152,15 @@ In the example we used multiple keys in the metadata dictionary including `help_
 `output_file_template` (`str`):
    If provided, the field is treated also as an output field and it is added to the output spec.
    The template can use other fields, e.g. `{file1}`.
-   Used in order to create an output specification.
+   Used in order to create an output definition.
 
 `output_field_name` (`str`, used together with `output_file_template`)
    If provided the field is added to the output spec with changed name.
-   Used in order to create an output specification.
+   Used in order to create an output definition.
 
 `keep_extension` (`bool`, default: `True`):
    A flag that specifies if the file extension should be removed from the field value.
-   Used in order to create an output specification.
+   Used in order to create an output definition.
 
 `readonly` (`bool`, default: `False`):
    If `True` the input field can't be provided by the user but it aggregates other input fields

docs/output_spec.rst

@@ -5,7 +5,7 @@ Output Specification
 
 As it was mentioned in :ref:`shell_command_task`, the user can customize the input and output
 for the `ShellCommandTask`.
-In this section, the output specification will be covered.
+In this section, the output definition will be covered.
 
 
 Instead of using field with `output_file_template` in the customized `input_spec` to specify an output field,
@@ -29,26 +29,26 @@ a customized `output_spec` can be used, e.g.:
                 ),
             )
         ],
-        bases=(ShellOutSpec,),
+        bases=(ShellOutDef,),
     )
 
     ShellCommandTask(executable=executable,
                      output_spec=output_spec)
 
 
 
-Similarly as for `input_spec`, in order to create an output specification,
+Similarly as for `input_spec`, in order to create an output definition,
 a new `SpecInfo` object has to be created.
 The field `name` specifies the type of the spec and it should be always "Output" for
-the output specification.
-The field `bases` specifies the "base specification" you want to use (can think about it as a
-`parent class`) and it will usually contains `ShellOutSpec` only, unless you want to build on top of
-your other specification (this will not be cover in this section).
+the output definition.
+The field `bases` specifies the "base definition" you want to use (can think about it as a
+`parent class`) and it will usually contains `ShellOutDef` only, unless you want to build on top of
+your other definition (this will not be cover in this section).
 The part that should be always customised is the `fields` part.
-Each element of the `fields` is a separate output field that is added to the specification.
+Each element of the `fields` is a separate output field that is added to the definition.
 In this example, a three-elements tuple - with name, type and dictionary with additional
 information - is used.
-See :ref:`Input Specification section` for other recognized syntax for specification's fields
+See :ref:`Input Specification section` for other recognized syntax for definition's fields
 and possible types.
 
 

new-docs/source/index.rst

@@ -3,11 +3,21 @@
 Pydra
 =====
 
-Pydra is a new lightweight dataflow engine written in Python, which provides a simple way to
-implement scientific workflows that use a mix of shell commands and Python functions.
-
-Pydra is developed as an open-source project in the neuroimaging community,
-but it is designed as a general-purpose dataflow engine to support any scientific domain.
+Pydra is a lightweight, Python 3.11+ dataflow engine for computational graph construction,
+manipulation, and distributed execution. Designed as a successor to created for [Nipype](https://github.com/nipy/nipype),
+Pydra is a general-purpose engine that supports analytics in any scientific domain.
+Pydra helps build reproducible, scalable, reusable, and fully automated, provenance
+tracked scientific workflows that combine Python functions and shell commands.
+
+The power of Pydra lies in ease of workflow creation and execution for complex
+multiparameter map-reduce operations, and the use of global cache.
+
+Pydra's key features are:
+- Modular execution backends (see [Advanced execution](../tutorial/advanced-execution.html))
+- Map-reduce like semantics (see [Splitting and combining](../explanation/splitting-combining.html))
+- Global cache support to reduce recomputation (see [Hashing and caching](../explanation/hashing-caching.html))
+- Support for execution of Tasks in containerized environments (see [Environments](../explanation/environments.html))
+- Strong type-checking and type-hinting support (see [Typing](../explanation/typing.html))
 
 See :ref:`Design philosophy` for more an explanation of the design of Pydra.
 
@@ -64,7 +74,7 @@ Indices and tables
 
 .. toctree::
     :maxdepth: 2
-    :caption: Execution
+    :caption: Task execution
     :hidden:
 
     tutorial/getting-started

new-docs/source/tutorial/advanced-execution.ipynb

@@ -11,7 +11,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Plugins"
+    "## Workers\n",
+    "\n",
+    "Pydra supports several workers with which to execute tasks\n",
+    "\n",
+    "- `ConcurrentFutures`\n",
+    "- `SLURM`\n",
+    "- `Dask` (experimental)\n",
+    "- `Serial` (for debugging)"
    ]
   },
   {