Backend information

[jameshcorbett]

libEnsemble asked in a meeting today whether we could provide an interface to get the list of nodes from within a job. I thought it sounded reasonable and I was wondering if we might want to generalize to a kind of backend interface for getting job information out. Maybe the only thing is the nodelist, but maybe job ID too? Mostly I think this would come down to reading environment variables.

[andre-merzky]

FYI, RP exposes the following environment variables to tasks (additionally to RP specific variables):

$RP_TASK_ID          : unique task id
$RP_TASK_NAME        : user specified task name
$RP_TASK_SANDBOX     : user spefified or derived task workdir
$RP_RANK             : task rank
$RP_RANKS            : number of task ranks

Additionally, we provide an equivalent to a nodefile - a slots file which informs the application about the nodes, cores and gpus used to run the task. Here an example for an MPI task which uses one core and no gpus per rank:

{'partition_id': None,
 'ranks': [{'core_map': [3],
            'gpu_map': [],
            'lfs': 0,
            'mem': 0,
            'node_id': 'c206-024',
            'node_name': 'c206-024'},
           {'core_map': [4],
            'gpu_map': [],
            'lfs': 0,
            'mem': 0,
            'node_id': 'c206-024',
            'node_name': 'c206-024'},
           {'core_map': [5],
            'gpu_map': [],
            'lfs': 0,
            'mem': 0,
            'node_id': 'c206-024',
            'node_name': 'c206-024'},
            ...

lfs: local file storage allocated for rank
mem: memory allocated for that rank

We mostly use the information though for our internal operation and have very few use cases outside of that. The RP_TASK_ID and RP_TASK_NAME are used by applications sometimes. But anyway, just wanted to document the state of affairs in RCT in the context of this ticket...

I should add that exporting env variables is really cheap, writing node files not so much when we consider large numbers of jobs.

[hategan]

This is related to ExaWorks/psij-python#200