About mapping/binding policies

[GuillaumeMercier]

In preparation for tomorrow's meeting, here are a couple of points and questions:
As far as mapping/binding is concerned, I think that there is a difference in behaviour between processes and threads.

1 Processes

There are two ways of allocating resources to processes:

1. qv_scope_split: a number of so-called pieces are passed as an argument to the function
2. qv_scope_split_at: no number of pieces but rather the type of resources that are going to be allocated to the processes. The user has to make sure that the number of resources and the number of processes are compatible.

In both cases, all available resources are allocated.

• This corresponds to the spread policy as defined in the document present in the repository
• It is possible to have a number of processes different from the number of pieces?
  I don't think it is and therefore, it is unnecessary to have policies for processes since all (last level) resources end up being used (policies can be useful when some resources are unused because of the possible gaps between used resources).
• The semantics of QV_SCOPE_USER and QV_SCOPE_SYSTEM might need revisiting. Indeed, with QV_SCOPE_USER, all resources in the system are in the scope, regardless of the mapping/binding policy used for launching the application. I suspect this will cause confusion. QV should cooperate with other tools/runtimes policies, not just get rid of them and do as it likes. @samuelkgutierrez proposed to modify the behaviour so that QV_SCOPE_SYSTEM gives all resources in the system (regardless of the previous mapping, e.g, the mapping enforced by mpiexec) while QV_SCOPE_USER would build upon the resources made available because of the previous mapping. There might be issues with this but I think it is rather sensible to make this change.

2 Threads

Threads are altogether different beasts. There are still two kinds of split functions: split and split_at. Once again, the difference being that a number of pieces or a resource type is passed as an argument to the function. But there is also an array of colors that "guides" the splitting in both cases for enhanced flexibility.

1. In the case of split_at, can there be less threads than resources? If that is the case, then there are gaps between resources and we then can define policies such packed or scatter and possibly stride or round-robin.
2. In the case of split, there can be less pieces than threads (e.g., https://github.com/hpc/quo-vadis/blob/master/tests/test-pthread-split.c) .But what kind of resources are going to be used effectively?
   - Do we want to map the threads onto the resources the highest in the hierarchy that fits the number of pieces?
   - Or do we want to use the lowest level resources, ie. PUs or Cores?
   Right now, it is the second option is implemented (see https://github.com/hpc/quo-vadis/blob/master/src/quo-vadis-thread.cc#L111)
   which is a sensible choice. Once again, the presence of gaps allow for policies.
   I think that the definitions~ I showed last time are then relevant, but for threads and not for processes as I thought. Except that the spread policiy does not make a lot of sense since not all resources are used in the end.

### Note
we also need to discuss https://github.com/hpc/quo-vadis/blob/master/src/qvi-hwloc.cc#L334

[samuelkgutierrez]

Note addressed in #376