Skip to content

pythonizing reference guide

Jump to bottom Edit New page
Jeff Squyres edited this page Dec 18, 2019 · 49 revisions

Pythonizing: Reference Guide

The intent is that you invoke a few Python functions to define the MPI routine. The two main functions that you will use are function_name() and parameter(), but there are a few other functions that are necessary in some cases.

The definitive listing of these functions, parameters, and other information you may need to know are in the binding-tool/binding_tool.py script in the git repository. This is only mentioned in case this wiki documentation gets stale (gasp!).

function_name()

By style convention, this function should be called first in your mpi-bindings block.

This function is straightforward: pass in the mixed-case name of the MPI routine in question.

# Good
function_name('MPI_Send')

# Bad
function_name('mpi_send')
function_name('MPI_send')

Incorrect casing will not be fixed for you.

It is assumed that this function will be invoked in every single {mpi-binding} block. Per above, it should be first:

# Good
\begin{mpi-binding}
function_name('MPI_Foo')
parameter('comm', 'COMMUNICATOR')
\end{mpi-binding}

# Bad:
\begin{mpi-binding}
parameter('comm', 'COMMUNICATOR')
function_name('MPI_Bar')
\end{mpi-binding}

parameter()

A single invocation of this function describes a single parameter in an MPI routine.

Order of invocation

The order in which parameters are defined via the parameter() function is maintained when then final LaTeX is rendered. Meaning:

function_name('MPI_Foo')
parameter('foo', 'COMMUNICATOR', desc='the communicator')
parameter('bar', 'DATATYPE', desc='the datatype')

will render MPI_Foo(foo, bar), while:

function_name('MPI_Foo')
parameter('bar', 'DATATYPE', desc='the datatype')
parameter('foo', 'COMMUNICATOR', desc='the communicator')

will render MPI_Foo(bar, foo).

Parameters

The parameter() function can take many parameters. The first two are positional and are mandatory. The remaining are either optional or only required in certain cases.

It is highly recommended that you go try to write your bindings with just the name, kind, direction, and desc parameters to parameter(), and use the documentation in this section as a reference for when those four parameters are not sufficient.

name

The string name of the parameter. This parameter is always the first parameter, and is required.

kind

A string representing the type/kind of the parameter. This parameter is always the second parameter, and is required. The allowable kinds are:

  • Pointers

    • BUFFER: a choice buffer
    • C_BUFFER: a C choice buffer (e.g., for MPI_ALLOC_MEM, which specifically takes a C buffer argument)
    • C_BUFFER2: a C choice buffer used for MPI_Buffer_detach, uses in F90 instead of INTEGER(KIND=MPI_ADDRESS_KIND) with a "USE, INTRINSIC :: ISO_C_BINDING, ONLY : C_PTR" in the F08 binding.
    • C_BUFFER3: a C choice buffer used for MPI_CONVERSION_FN_NULL callback definition, has "VALUE" as attribute on F08 binding, instead of usual "nothing".
    • C_BUFFER4: a C choice buffer used for MPI_User_function callback definition, has "VALUE" as attribute on F08 binding, instead of usual "nothing".
    • EXTRA_STATE: extra state (e.g., for MPI attribute functions)
    • EXTRA_STATE2: extra state, used for MPI_Keyval_create and others, emits INTEGER in F90 and F08, in deprecated chapter.
    • FUNCTION: a function pointer. When this type is used, the func_type parameter must be specified to indicate the type of the function pointer.
    • STRING: a string
    • STRING_ARRAY: an array of strings (e.g., for MPI_COMM_SPAWN)
    • STRING_2DARRAY: an array of arrays of strings (e.g., for MPI_COMM_SPAWN_MULTIPLE)

  • Integers

    • ACCESS_MODE: ...
    • ALLOC_MEM_NUM_BYTES: ...
    • ARRAY_LENGTH: the integer length of an array
    • ASSERT: ...
    • ATTRIBUTE_VAL_10: the type of MPI attribute values in MPI-1.0. This type only exists because of some deprecated functions that are still listed in the standard. It should not be used for any new bindings.
    • ATTRIBUTE_VAL: the type of MPI attribute values starting with MPI-2.0.
    • BIND_TYPE: ...
    • BLOCKLENGTH: integer length of blocks
    • CAT_INDEX: ...
    • COLOR: color for algebraic operations (e.g., for MPI_COMM_SPLIT)
    • COMM_SIZE: the integer number of processes in a communicator or group
    • COMBINER: an integer MPI enum-like value
    • COORDINATE: an integer coordinate (used in topologies)
    • CVAR_INDEX: ...
    • DEGREE: an integer degree (used in topologies)
    • DISPLACEMENT_SMALL: MPI-3.1 and prior DISPLACEMENT.
    • DISPLACEMENT: MPI-3.1 "big" parameters in the _X functions.
    • DIMENSION: an integer dimension (used in topologies)
    • ENUM: an arbitrary enum-like integer
    • ERROR_CLASS: an integer MPI error class
    • ERROR_CODE: an integer MPI error code
    • ERROR_CODE_SHOW_INTENT: an integer MPI error code (only for intent OUT + F08 intent)
    • FILE_DESCRIPTOR: an integer file descriptor (e.g., for MPI_COMM_JOIN)
    • KEY: ....
    • KEYVAL: integer keyvals for MPI attribute functions
    • KEY_INDEX: ...
    • INDEX: integer index into an array
    • INFO_VALUE_LENGTH: ...
    • LOCK_TYPE: ...
    • LOGICAL: Boolean true/false value
    • LOGICAL_OPTIONAL: ...
    • MATH: integer for use in some mathematical value (e.g., precision, exponent, etc.)
    • NUM_BYTES_SMALL: MPI-3.1 and prior NUM_BYTES.
    • NUM_BYTES: Post MPI-3.1 "big" parameters in the _X functions.
    • NUM_DIMS: ...
    • ORDER: an integer MPI enum-like value
    • OFFSET: ...
    • PACK_EXTERNAL_SIZE: ...
    • PROFILE_LEVEL: ...
    • PVAR_INDEX: ...
    • PVAR_CLASS: ...
    • RANK: integer rank in a communicator or group
    • STRIDE_BYTES: an integer stride expressed as a number of bytes
    • STRIDE_ELEM: an integer stride expressed as a number of elements
    • STRING_LENGTH: the integer length of a string
    • TAG: an integer tag
    • THREAD_LEVEL: an integer MPI enum-like value
    • TIMESTAMP: ...
    • TOOLENUM_INDEX: ...
    • TOOLENUM_SIZE: ...
    • TOOL_VAR_VERBOSITY: ...
    • TOOL_VAR_VALUE: ...
    • TOOLS_ENUM: ...
    • TYPECLASS_SIZE: ...
    • UPDATE_MODE: ...
    • VARIABLE_SCOPE: ...
    • VERSION: an integer version
    • WEIGHT: an integer weight (used in topologies)
    • WINDOW_SIZE: ...
    • WIN_ATTACH_SIZE: ...
    • XFER_NUM_ELEM_SMALL: MPI-3.1 and prior XFER_NUM_ELEM.
    • XFER_NUM_ELEM: MPI-3.1 "big" parameters in the _X functions.

  • Double precision types

    • TICK_RESOLUTION: the return from MPI_WTICK
    • WALL_TIME: the return from MPI_WTIME

  • Special types

    • NOTHING: the equivalent of C void (only to be used as a function return type)

  • Polymorphic integer types

    • POLYDISPLACEMENT: a displacement that is currently setup to render as plain integer in MPI-3.1 style and "big" integer in MPI-4.0 style.
    • POLYDTYPE_NUM_ELEM: a datatype number of elements that is currently setup to render as plain integer in MPI-3.1 style and "big" integer in MPI-4.0 style.
    • POLYDISPOFFSET: ...
    • POLYTOOLS_NUM_ELEM: ...
    • POLYNUM_BYTES: a datatype number of bytes that is currently setup to render as plain integer in MPI-3.1 style and "big" integer in MPI-4.0 style.
    • POLYXFER_NUM_ELEM: a number of elements that is currently setup to render as plain integer in MPI-3.1 style and "big" integer in MPI-4.0 style.

  • Handles

    • COMMUNICATOR: an MPI communicator handle
    • DATATYPE: an MPI datatype handle
    • ERRHANDLER: an MPI errhandler handle
    • F90_STATUS: a Fortran 90 MPI status (used only in conversion routines)
    • F08_STATUS: a Fortran 08 MPI status (used only in conversion routines)
    • FILE: an MPI file handle
    • GROUP: an MPI group handle
    • INFO: an MPI info handle
    • MESSAGE: an MPI message handle
    • REQUEST: an MPI request handle
    • STATUS: an MPI status
    • WINDOW: an MPI window handle
    • OPERATION: an MPI_Op used in collectives and one-sided
    • CVAR: a control variable handle
    • PVAR: a performance variable handle
    • PVAR_SESSION: a performance variable session handle
    • TOOL_MPI_OBJ: ...
    • VARARGS: variable arguments, only used in MPI_Pcontrol for C. Is ignored in F08/F90.

desc

This string parameter is passed by name (i.e., desc="blah").

It is technically not required, but it is strongly recomended.

The string value is rendered as part of the LIS.

direction

Indicate the direction intent of this parameter. This parameter affects the rendering in most language bindings:

  • LIS: determines the IN, INOUT, or OUT label
  • C: generally determines whether the parameter is passed by value or reference
  • F90: does not affect the rendering
  • F08: generally determines the INTENT clause

The allowable values of the direction parameter are:

  • in: since the majority of MPI parameters are intent IN, in is the default value for this parameter. Parameters marked as in will be rendered as being passed by value.
  • out: the OUT intent. Parameters marked as out will be rendered as being passed by reference.
  • inout: the INOUT intent. Parameters marked as inout will be rendered as being passed by reference -- except for one special case. See below.

There is a special case: MPI's definition of INOUT has a peculiar meaning with regards to MPI handles. Specifically: if an MPI handle parameter is marked as INOUT, it may be passed by value or it may be passed by reference depending on the situation.

By default, inout-marked parameters are passed by reference. But for cases where the MPI binding actually requires the MPI handle to be passed by value, you can pass a special value to the direction indicating the disparity. For example:

function_name('MPI_Comm_set_info')

parameter('comm', 'COMMUNICATOR', direction='lis:inout,param:in',
        desc='communicator')
parameter('info', 'INFO', desc='info object')

For MPI_COMM_SET_INFO, the comm argument is marked INOUT in the LIS, but it is passed by value in the C/F08 bindings. Hence, we pass lis:inout to indicate that the LIS should be rendered as INOUT, but the C/Fortran bindings parameter should be rendered as IN.

Thanks, MPI! 😉

length

For single-dimension array parameters, this value is set to a string representing the length of the array (when that length is known). For example:

function_name('MPI_Waitall')
parameter('count', 'ARRAY_LENGTH', desc='lists length')
parameter('array_of_requests', 'REQUEST',  desc='array of requests',
          direction='inout', length='count')
parameter('array_of_statuses', 'STATUS',
          desc='array of status objects',
          direction='out', length='*')

Note the array_of_requests parameter lists count as its length, because that array is defined to be the length specified by the count parameter.

Note, too, the length for the array_of_statuses parameter is *. This is because the length of the array is not known (specifically, because it could be MPI_STATUS_IGNORE), and therefore must be rendered as (*) in Fortran (array lengths are not rendered in C; arrays are rendered as [] in C).

There are a small number of two-dimension arrays in MPI. 2D string arrays are a special beast and have their own type (STRING_2DARRAY) and do not use the length parameter. But functions like MPI_GROUP_RANGE_INCL require a fixed 2D array of integers. Consider:

function_name('MPI_Group_range_incl')

parameter('group', 'GROUP', desc='group')
parameter('n', 'DEFERRED_INT',
          desc='number of triplets in array \mpiarg{ranges}')
parameter('ranges', 'RANK', length=['n', '3'],
          desc='a one-dimensional array of integer triplets, of the form (first rank, last rank, stride) indicating ranks in \mpiarg{group} of processes to be included in \mpiarg{newgroup}')
parameter('newgroup', 'GROUP', direction='out',
          desc='new group derived from above, in the order defined by \mpiarg{ranges}')

Notice that length is an array of each of the dimension lengths.

func_type

If the kind parameter is FUNCTION, this parameter must be specified.

The string value is the type of the function parameter. For example:

function_name('MPI_Comm_create_keyval')

parameter('comm_copy_attr_fn', 'FUNCTION',
          func_type='MPI_Comm_copy_attr_function',
          desc='copy callback function for \mpiarg{comm_keyval}')
parameter('comm_delete_attr_fn', 'FUNCTION',
          func_type='MPI_Comm_delete_attr_function',
          desc='delete callback function for \mpiarg{comm_keyval}')
# ...etc.

Specifying the function pointer type allows the C/Fortran bindings to render the correct type.

pointer

This is a Boolean value that defaults to False. Specifying it as True overrides the C binding rendering -- the parameter will be passed by reference.

constant

This is a Boolean value (that defaults to False) that indicates whether the parameter is constant or not. In C, this translates to prefixing the type with const.

root_only

This is a Boolean value (that defaults to False). If set to True, the string ", significant only at root" is added to the description. It is meant as a shortcut / syntatic sugar for the many rooted MPI routings.

mpi_owned

This is a Boolean value (that defaults to False). When set to True, it indicates that MPI retains ownership of this value after the routine returns. This causes the ASYNCHRONOUS keyword to be rendered in the F08 bindings for this parameter.

suppress

This parameter is used to suppress the rendering of certain properties. They are generally very special cases, and are only needed infrequently. This parameter can take a comma-delimited list of values:

  • f08_intent: do not emit the F08 INTENT clause. Specifically, the INTENT clause is rendered for most F08 parameters. There are a few cases where INTENT is not rendered, and those are usually automatically detected by the rendering engine. However, there are a few cases where we specifically do not include an INTENT clause in the F08 bindings, but the reasons for omitting the INTENT are obscure and/or do not fit into a general rule that the rendering engine knows. Hence, you can pass suppress=f08_intent to cause the F08 bindings to not emit an INTENT clause for this parameter.
  • lis_paren: do not emit the parenthetical clause after the LIS parameter description. Sometimes an exception is needed to the general rules for the LIS parenthetical clause. Suppressing the LIS parenthetical from being generated means that you can pass in your own parenthetical at the end of the desc parameter.
  • f90_buf_paren: do not emit the parenthesis '(*)' after the buffer descriptor for a F90 binding.
  • c_parameter: do not emit this parameter in C bindings.
function_name('MPI_Buffer_attach')
parameter('buffer', 'BUFFER', desc='initial buffer address',
          mpi_owned=True, suppress='f08_intent')
parameter('size', 'POLYNUM_BYTES', desc='buffer size, in bytes')

See MPI_COMM_SPAWN for another example of both of these.

f90_suppress_paren

Suppress the printing of '(*)' for the BUFFER kind in F90 bindings.

This is used in MPI_SIZEOF, because suppress is used to suppress the 'f08_intent'.

optional

This is a Boolean value (that defaults to False). It is used to indicate optional parameters. The only parameter in MPI-4.0 that meets this description is the F08 ierror, which is automatically included in all parameter lists unless the no_ierror() function is invoked.

Therefore, you should likely never need to use the optional parameter. It is only listed here for completeness.

parameters_c_only()

This function takes the same arguments as parameter(), but these parameters are only used in the LIS+C bindings (not the Fortran bindings). MPI_INIT and MPI_INIT_THREAD are good examples where this is needed.

JMS NOT SURE THIS HAS BEEN TESTED / NOT EXACTLY SURE WHAT THE PARAMS ARE TO THIS FUNCTION

return_type()

Nearly all MPI routines return an int in C and render a SUBROUTINE in Fortran (i.e., no return value). However, there are a small number of routines that return something else. The return_type() function accepts any of the valid KINDs from above.

function_name('MPI_Wtime')
return_type('WALL_TIME')
no_ierror()

NOTE: If more return types are needed over time, we'll need to expand the list of values that it can handle and return types that are emitted. This is not difficult.

no_ierror()

A small number of MPI routines do not have an ierror parameter to the Fortran bindings. Invoking this function suppresses the ierror parameter in Fortran bindings. For example:

function_name('MPI_Wtick')
return_type('TICK_RESOLUTION')
no_ierror()

no_x_variant()

Only relevant for MPI-4.0-style rendering, which isn't done yet.

JMS TBD

no_c_binding()

When this function is invoked, the C binding is suppressed.

As of MPI-4.0, this function is necessary for a few procedures in the Language Bindings chapter that are Fortran-only. For example:

function_name('MPI_Sizeof')
no_c_binding()
# ...etc.

f90_use_colons()

When this function is invoked, F08-style "::" separators are used in the F90 binding rendering.

As of MPI-4.0, this function is necessary for a few procedures in the Language Bindings chapter that are Fortran-only. For example:

function_name('MPI_Status_f2f08')
f90_use_colons()
# ...etc.

no_lis_binding()

When this function is invoked, the LIS binding is suppressed.

This is used for the callback functions.

no_f08_binding()

When this function is invoked, the F08 binding is suppressed.

This function is necessary for several MPI-1.0 functions that are still listed in the deprecated chapter that have no F08 bindings.

This is needed for MPI_T functions, which are only defined for C.

needs_human_attention()

When this function is invoked, the emitted bindings are colored red to attract attention.

This is used throughout the automatically converted bindings to cause a person to have a look.

no_f90_binding()

When this function is invoked, the F90 binding is suppressed.

This is needed for MPI_T functions, which are only defined for C.

callback()

When this function is invoked, the binding is marked as a callback function.

This is needed in the Context, Deprecated and IO chapters.

only_annex()

When this function is invoked, the binding will only appear in the function annex.

This is needed for callback function in for the Context and Deprecated chapters.

Add a custom footer
Add a custom sidebar
Clone this wiki locally