Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/advanced.rst
Original file line number Diff line number Diff line change
Expand Up @@ -446,7 +446,7 @@ scheduler, but don't configure one. It's time to rectify that.
# srun -N 50 -n 150 ./supermagic -a
# Note that this would run in an sbatch script within an allocation
# that conforms to the rest of the slurm settings.
- {sched.test_cmd} ./supermagic -a
- {sched.launch} ./supermagic -a


- See ``pav show sched --vars <sched_name>`` for a listing of what variables are
Expand Down
2 changes: 1 addition & 1 deletion docs/results/parse.rst
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,7 @@ depends on the **per\_file** attribute for the result parser.
cmds:
# Use the srun --output option to specify that results are
# to be written to separate files.
- {{sched.test_cmd}} --output="%N.out" env
- {{sched.launch}} --output="%N.out" env

result_parse:
regex:
Expand Down
4 changes: 2 additions & 2 deletions docs/test_run_lifecycle.rst
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,7 @@ something like this:
"result": "last_login > {{pav.timestamp - time_limit}}"},
"regex": [{"key": "last_login",
"regex": "Last Login: (\\d+)"}]},
"run": {"cmds": ["{{sched.test_cmd}} ./test1 $ARGS"],
"run": {"cmds": ["{{sched.launch}} ./test1 $ARGS"],
"env": {"ARGS": "-u {{user}}"},
"modules": ["gcc"]},
"variables": {"hours": 5,
Expand Down Expand Up @@ -457,4 +457,4 @@ Mark the Test as Complete

Finally, the test is marked as complete by saving a 'RUN_COMPLETE' file in the
test's run directory. Pavilion uses this to quickly determine which tests
might still be running.
might still be running.
8 changes: 4 additions & 4 deletions docs/tests/scheduling.rst
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ command.
| | | 1.
tasks_total | True | 180 | Total tasks to create, based on number of nodes
| | | actually acquired.
test_cmd | True | srun -N 5 -w no | Construct a cmd to run a process under this
launch | True | srun -N 5 -w no | Construct a cmd to run a process under this
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd add a (formerly sched.test_cmd) to the docs here.

| | de[05-10],node2 | scheduler, with the criteria specified by this test.
| | 3 -n 20 |

Expand Down Expand Up @@ -338,9 +338,9 @@ wrapper command before actually running the intended command.
cmds:
# The run command will be `srun -N1 -p standard valgrind ./supermagic -a`
# It will run `valgrind ./supermagic -a` on the allocation
- '{{sched.test_cmd}} ./supermagic -a'
- '{{sched.launch}} ./supermagic -a'

When using the ``raw`` scheduler, the ``{{sched.test_cmd}}`` normally returns an empty string. You can
When using the ``raw`` scheduler, the ``{{sched.launch}}`` normally returns an empty string. You can
use the wrapper setting to control a different scheduler directly.

.. code-block:: yaml
Expand All @@ -355,5 +355,5 @@ use the wrapper setting to control a different scheduler directly.
run:
cmds:
# With the schedule wrapper, this will be `mpirun -np 2 ./supermagic -a`
- '{{sched.test_cmd}} ./supermagic -a'
- '{{sched.launch}} ./supermagic -a'

2 changes: 1 addition & 1 deletion docs/tests/values.rst
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ not Python3 code.**
cmds:
# In the simple case, expressions are often just a variable
# reference.
- "{{sched.test_cmds}} ./my_cmd"
- "{{sched.launch}} ./my_cmd"

Types of Expressions
^^^^^^^^^^^^^^^^^^^^
Expand Down
4 changes: 2 additions & 2 deletions examples/acceptance/tests/ior.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -273,11 +273,11 @@ scaling_study:
- '[ -d {{scratch.path}}/ior-test-{{dirname}} ] && rm -Rf {{scratch.path}}/ior-test-{{dirname}}'
- 'mkdir {{scratch.path}}/ior-test-{{dirname}} && pushd {{scratch.path}}/ior-test-{{dirname}}'
- '# this writes out the files'
- '{{sched.test_cmd}} $IOR_EXE "${test_variables} -w'
- '{{sched.launch}} $IOR_EXE "${test_variables} -w'
- 'if [[ {{numnodes}} -ne 1 ]]'
- 'then'
- ' # this reads those files'
- ' {{sched.test_cmd}} $IOR_EXE -E -C -Q {{taskspernode}} -r'
- ' {{sched.launch}} $IOR_EXE -E -C -Q {{taskspernode}} -r'
- 'fi'
- 'cd ..'
- 'rm -rf {{scratch.path}}/ior-test-{{dirname}}'
2 changes: 1 addition & 1 deletion examples/acceptance/tests/pynamic.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ base:
# Run commands.
cmds:
- 'cd pynamic-pyMPI-2.6a1'
- '{{sched.test_cmd}} ./pynamic-mpi4py $(date +%s)'
- '{{sched.launch}} ./pynamic-mpi4py $(date +%s)'

result_parse:
regex:
Expand Down
2 changes: 1 addition & 1 deletion examples/acceptance/tests/umt.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ _base:
# seconds to update all the information. Sleeping for a few seconds allows for the seff command
# to report the data accurately.
cmds:
- "{{sched.test_cmd}} ./umt2015-crossroads/Teton/SuOlsonTest {{umt_run.gridfilename}}
- "{{sched.launch}} ./umt2015-crossroads/Teton/SuOlsonTest {{umt_run.gridfilename}}
{{umt_run.groups}} {{umt_run.quadtype}} {{umt_run.order}} {{umt_run.polar}}
{{umt_run.azim}} ; job_id=${SLURM_JOB_ID}"
- "sleep 10s"
Expand Down
2 changes: 1 addition & 1 deletion examples/acceptance/tests/vpic.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,7 @@ _base:
cmds:
- ./build/bin/vpic {{vpic_input_deck}}
- 'top -b -u {{user}} > top_out.txt 2>&1 &'
- '{{vpic_run.time_cmd}} {{sched.test_cmd}} {{vpic_run.numa}} ./lpi-input.Linux --tpp 1'
- '{{vpic_run.time_cmd}} {{sched.launch}} {{vpic_run.numa}} ./lpi-input.Linux --tpp 1'

schedule:
nodes: '{{nnodes}}'
Expand Down
2 changes: 1 addition & 1 deletion examples/pre-packaged/tests/hello_mpi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,4 +44,4 @@ base:
- "{{mpis}}"
cmds:
- "set -x"
- "{{sched.test_cmd}} ./mpi_hello"
- "{{sched.launch}} ./mpi_hello"
2 changes: 1 addition & 1 deletion examples/pre-packaged/tests/hpcg.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ base:

run:
modules: ["{{compilers}}", "{{mpis}}"]
cmds: "{{sched.test_cmd}} ./bin/xhpcg"
cmds: "{{sched.launch}} ./bin/xhpcg"

result_parse:
regex:
Expand Down
2 changes: 1 addition & 1 deletion examples/pre-packaged/tests/imb.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,4 +44,4 @@ base:

cmds:
- cd src_c
- "{{sched.test_cmd}} ./IMB-MPI1 -npmin {{sched.alloc_cpu_total}} -mem 50 alltoall"
- "{{sched.launch}} ./IMB-MPI1 -npmin {{sched.alloc_cpu_total}} -mem 50 alltoall"
4 changes: 2 additions & 2 deletions examples/pre-packaged/tests/ior.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ base:
- " rm -rf ior-test-{{block_size}}-{{a_flag}}-{{scratch.name}}"
- " mkdir ior-test-{{block_size}}-{{a_flag}}-{{scratch.name}}"
- " cd ior-test-{{block_size}}-{{a_flag}}-{{scratch.name}}"
- " {{sched.test_cmd}} {{srun_opts}} $bloc/src/ior -a {{a_flag}} -b {{block_size}} -c -e -F -i 2"
- " {{sched.launch}} {{srun_opts}} $bloc/src/ior -a {{a_flag}} -b {{block_size}} -c -e -F -i 2"
- " cd .."
- " rm -rf {{scratch.path}}/ior-test-{{block_size}}-{{a_flag}}-{{scratch.name}}"
- "fi"
Expand Down Expand Up @@ -180,7 +180,7 @@ xroads_load1:
- "rm -rf ior-test-{{outname}}"
- "mkdir ior-test-{{outname}}"
- "cd ior-test-{{outname}}"
- "{{sched.test_cmd}} {{srun_opts}} $bloc/src/ior -f $bloc/inp"
- "{{sched.launch}} {{srun_opts}} $bloc/src/ior -f $bloc/inp"
- "cd .."
- "rm -rf {{scratch.path}}/ior-test-{{outname}}"

Expand Down
2 changes: 1 addition & 1 deletion examples/pre-packaged/tests/stream.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,4 @@ base:
- "{{compilers}}"
- "{{mpis}}"
cmds:
- "{{sched.test_cmd}} ./stream"
- "{{sched.launch}} ./stream"
2 changes: 1 addition & 1 deletion examples/pre-packaged/tests/supermagic.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ base:
- "{{mpis}}"

cmds:
- "{{sched.test_cmd}} ./supermagic"
- "{{sched.launch}} ./supermagic"

result_parse:
regex:
Expand Down
1 change: 1 addition & 0 deletions lib/pavilion/commands/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
'set_status': ['status_set'],
'result': ['results'],
'list_cmd': ['list'],
'launch': ['test_cmd'],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are aliases for Pavilion commands, not the sched.test_cmd variable.

}


Expand Down
12 changes: 6 additions & 6 deletions lib/pavilion/schedulers/plugins/flux.py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the alias in the for test_cmd->launch. It should probably go in the base class for SchedulerVariables.

Original file line number Diff line number Diff line change
Expand Up @@ -66,10 +66,10 @@ class FluxVars(SchedulerVariables):

EXAMPLE = SchedulerVariables.EXAMPLE.copy()
EXAMPLE.update({
'test_cmd': 'flux run -x -N 5 -n 20',
'launch': 'flux run -x -N 5 -n 20',
})

def _test_cmd(self):
def _launch(self):
"""Construct a cmd to run a process under this scheduler, with the
criteria specified by this test.
"""
Expand All @@ -91,13 +91,13 @@ def _test_cmd(self):
return ' '.join(cmd)

@dfr_var_method
def test_cmd(self):
def launch(self):
"""Calls the actual test command and then wraps the return with the wrapper
provided in the schedule section of the configuration."""

# Removes all the None values to avoid getting a TypeError while trying to
# join two commands
return ' '.join(filter(lambda item: item is not None, [self._test_cmd(),
return ' '.join(filter(lambda item: item is not None, [self._launch(),
self._sched_config['wrapper']]))


Expand Down Expand Up @@ -210,11 +210,11 @@ def _get_config_elems(self):
yc.ListElem(name='fluxrun_extra',
sub_elem=yc.StrElem(),
help_text="Extra arguments to pass to flux run as part of "
"the 'sched.test_cmd' variable."),
"the 'sched.launch' variable."),
yc.ListElem(name='fluxbatch_extra',
sub_elem=yc.StrElem(),
help_text="Extra arguments to pass to flux batch as part of "
"the 'sched.test_cmd' variable."),
"the 'sched.launch' variable."),
]

defaults = {
Expand Down
10 changes: 5 additions & 5 deletions lib/pavilion/schedulers/plugins/slurm.py
Original file line number Diff line number Diff line change
Expand Up @@ -133,10 +133,10 @@ class SlurmVars(SchedulerVariables):

EXAMPLE = SchedulerVariables.EXAMPLE.copy()
EXAMPLE.update({
'test_cmd': 'srun -N 5 -w node[05-10],node23 -n 20',
'launch': 'srun -N 5 -w node[05-10],node23 -n 20',
})

def _test_cmd(self):
def _launch(self):
"""Construct a cmd to run a process under this scheduler, with the
criteria specified by this test.
"""
Expand Down Expand Up @@ -171,13 +171,13 @@ def _test_cmd(self):
return ' '.join(cmd)

@dfr_var_method
def test_cmd(self):
def launch(self):
"""Calls the actual test command and then wraps the return with the wrapper
provided in the schedule section of the configuration."""

# Removes all the None values to avoid getting a TypeError while trying to
# join two commands
return ' '.join(filter(lambda item: item is not None, [self._test_cmd(),
return ' '.join(filter(lambda item: item is not None, [self._launch(),
self._sched_config['wrapper']]))

def slurm_float(val):
Expand Down Expand Up @@ -276,7 +276,7 @@ def _get_config_elems(self):
yc.ListElem(name='srun_extra',
sub_elem=yc.StrElem(),
help_text="Extra arguments to pass to srun as part of the "
"'sched.test_cmd' variable."),
"'sched.launch' variable."),
yc.ListElem(name='sbatch_extra',
sub_elem=yc.StrElem(),
help_text="Extra arguments to add as sbatch header lines. "
Expand Down
4 changes: 2 additions & 2 deletions test/data/pav_config_dir/tests/sched_errors.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ a_error:
scheduler: dummy
run:
cmds:
- '{{sched.test_cmd}} echo "foo"'
- '{{sched.launch}} echo "foo"'

b_skipped:
inherits_from: a_error
Expand All @@ -25,4 +25,4 @@ d_no_nodes:

schedule:
nodes: 5
reservation: 'no_exist'
reservation: 'no_exist'
6 changes: 3 additions & 3 deletions test/tests/sched_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ def test_sched_var_values(self):
NodeSet(frozenset(['node05', 'node06', 'node07']))]

expected = {
'test_cmd': '',
'launch': '',
'tasks_per_node': '1',
'chunk_ids': ['0', '1'],
'chunk_size': '3',
Expand Down Expand Up @@ -127,7 +127,7 @@ def test_sched_var_values_basic(self):
chunks = None

expected = {
'test_cmd': '',
'launch': '',
'tasks_per_node': '1',
'chunk_ids': [],
'chunk_size': '',
Expand Down Expand Up @@ -596,7 +596,7 @@ def test_wrapper(self):

# The wrapper can by anything a command or even a string
test_cfg['schedule'] = {'wrapper': 'echo'}
test_cfg['run']['cmds'] = ['{{sched.test_cmd}} "this is the wrapper test"']
test_cfg['run']['cmds'] = ['{{sched.launch}} "this is the wrapper test"']

test = self._quick_test(test_cfg, finalize=False)

Expand Down
14 changes: 7 additions & 7 deletions test/tests/slurm_tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -297,7 +297,7 @@ def test_cancel(self):

slurm = pavilion.schedulers.get_plugin('slurm')
cfg = self._quick_test_cfg()
cfg['run']['cmds'] = ['{{sched.test_cmd}} sleep 10']
cfg['run']['cmds'] = ['{{sched.launch}} sleep 10']
cfg.update(self.slurm_mode)
cfg['schedule']['nodes'] = '5'
cfg['scheduler'] = 'slurm'
Expand All @@ -320,7 +320,7 @@ def test_slurm_params(self):
slurm = pavilion.schedulers.get_plugin('slurm')
cfg = self._quick_test_cfg()
cfg.update(self.slurm_mode)
cfg['run']['cmds'] = ['{{sched.test_cmd}} hostname']
cfg['run']['cmds'] = ['{{sched.launch}} hostname']
cfg['schedule']['nodes'] = '5'
cfg['schedule']['slurm'] = {
'sbatch_extra': ['--comment "Hiya!"'],
Expand All @@ -344,7 +344,7 @@ def test_slurm_kickoff_shared(self):
slurm = pavilion.schedulers.get_plugin('slurm')
cfg = self._quick_test_cfg()
cfg.update(self.slurm_mode)
cfg['run']['cmds'] = ['{{sched.test_cmd}} hostname']
cfg['run']['cmds'] = ['{{sched.launch}} hostname']
cfg['schedule']['nodes'] = '3'
cfg['schedule']['node_state'] = 'available'
cfg['schedule']['share_allocation'] = 'max'
Expand Down Expand Up @@ -380,7 +380,7 @@ def test_slurm_kickoff_indi(self):
slurm = pavilion.schedulers.get_plugin('slurm')
cfg = self._quick_test_cfg()
cfg.update(self.slurm_mode)
cfg['run']['cmds'] = ['{{sched.test_cmd}} hostname']
cfg['run']['cmds'] = ['{{sched.launch}} hostname']
cfg['schedule']['nodes'] = 'all'
cfg['schedule']['chunking'] = {'size': '3'}
cfg['schedule']['share_allocation'] = 'False'
Expand Down Expand Up @@ -412,7 +412,7 @@ def test_slurm_kickoff_flex(self):
slurm = pavilion.schedulers.get_plugin('slurm')
cfg = self._quick_test_cfg()
cfg.update(self.slurm_mode)
cfg['run']['cmds'] = ['{{sched.test_cmd}} hostname']
cfg['run']['cmds'] = ['{{sched.launch}} hostname']
cfg['schedule']['nodes'] = '3'
cfg['schedule']['share_allocation'] = 'False'
cfg['chunk'] = '0'
Expand Down Expand Up @@ -447,7 +447,7 @@ def test_mpirun(self):
slurm = pavilion.schedulers.get_plugin('slurm')
cfg = self._quick_test_cfg()
cfg.update(self.slurm_mode)
cfg['run']['cmds'] = ['{{sched.test_cmd}} hostname']
cfg['run']['cmds'] = ['{{sched.launch}} hostname']
cfg['run']['modules'] = ['gcc', 'openmpi']
cfg['schedule']['nodes'] = '5'
cfg['schedule']['slurm'] = {
Expand All @@ -470,7 +470,7 @@ def test_mpirun(self):
slurm = pavilion.schedulers.get_plugin('slurm')
cfg = self._quick_test_cfg()
cfg.update(self.slurm_mode)
cfg['run']['cmds'] = ['{{sched.test_cmd}} hostname']
cfg['run']['cmds'] = ['{{sched.launch}} hostname']
cfg['run']['modules'] = ['gcc', 'openmpi']
cfg['schedule']['nodes'] = '1'
cfg['schedule']['slurm'] = {'mpi_cmd': 'mpirun'}
Expand Down