Skip to content
thiell edited this page Mar 4, 2012 · 2 revisions

clush

clush is an utility provided with ClusterShell that aims to ease cluster administration.

Features (in brief)

  • execution of commands in parallel
  • smart display of command results (gathering, sorting by node, nodeset or node groups)
  • standard input text redirection to remote nodes
  • files copying in parallel
  • pdsh options backward compatibility (compare)

Man page (1.3)

Source of the latest clush reStructuredText man page is available at:

https://github.com/cea-hpc/clustershell/blob/master/doc/txt/clush.txt

=========
 clush
=========

-----------------------------------
execute shell commands on a cluster
-----------------------------------

:Date:   2010-05-25

SYNOPSIS
========

``clush`` ``-a`` | ``-g`` *group* | ``-w`` *nodes*  [ OPTIONS ]

``clush`` ``-a`` | ``-g`` *group* | ``-w`` *nodes*  [ OPTIONS ] *command*

``clush`` ``-a`` | ``-g`` *group* | ``-w`` *nodes*  [ OPTIONS ] --copy
*file* [ --dest *dest_path* ]

DESCRIPTION
===========
``clush`` is a program for executing commands in parallel on a cluster and for
gathering their results. ``clush`` executes commands interactively or can be
used within shell scripts and other applications.  It is a partial front-end
to the ClusterShell library that ensures a light, unified and robust command
execution framework.  ``clush`` currently makes use of the Ssh worker of
ClusterShell that only requires ``ssh``\(1) (OpenSSH SSH client).


INVOCATION
==========

``clush`` can be started non-interactively to run a shell *command*, or can
be invoked as an interactive shell. To start a ``clush`` interactive session,
invoke the ``clush`` command without providing *command*.

Non-interactive mode
  When ``clush`` is started non-interactively, the *command* is executed on
  the specified remote hosts in parallel. If option ``-b`` or ``--dshbak``
  is specified, ``clush`` waits for command completion and then displays
  gathered output results.

  Unless option ``--nostdin`` is specified, ``clush`` detects when its
  standard input is connected to a terminal (as determined by ``isatty``\(3)).
  If actually connected to a terminal, ``clush`` listens to standard input
  when commands are running, waiting for an `Enter` key press. Doing so will
  display the status of current nodes.  If standard input is not connected
  to a terminal, and unless option ``--nostdin`` is specified, ``clush``
  binds the standard input of the remote commands to its own standard input,
  allowing scripting methods like:

  | # echo foo | clush -w node[40-42] -b cat
  | ---------------
  | node[40-42]
  | ---------------
  | foo

  Please see some other great examples in the EXAMPLES section below.

Interactive session
  If a *command* is not specified, ``clush`` runs interactively. In this mode,
  ``clush`` uses the GNU ``readline`` library to read command lines. Readline
  provides commands for searching through the command history for lines
  containing a specified string. For instance, type Control-R to search in the
  history for the next entry matching the search string typed so far.
  ``clush`` also recognizes special single-character prefixes that allows the
  user to see and modify the current nodeset (the nodes where the commands are
  executed).

  Single-character interactive commands are:
    clush> ?
      show current nodeset
    clush> =<NODESET>
      set current nodeset
    clush> +<NODESET>
      add nodes to current nodeset
    clush> -<NODESET>
      remove nodes from current nodeset
    clush> !COMMAND
      execute COMMAND on the local system
    clush> =
      toggle the ouput format (gathered or standard mode)

  To leave an interactive session, type ``quit`` or Control-D.

File copying mode
  When ``clush`` is started with the ``-c`` or ``--copy`` option, it attempts
  to copy *file* to the provided target cluster nodes. If the ``--dest``
  option is specified, it will put the copied file there.


OPTIONS
=======

--version             show ``clush`` version number and exit
--nostdin             do not watch for possible input from stdin

Selecting target nodes:
  -w NODES            nodes where to run the command
  -x EXCLUDE          exclude nodes from the node list
  -a, --all           run command on all nodes
  -g GROUP, --group=GROUP
                      run command on a group of nodes
  -X EXGROUP          exclude nodes from this group

Output behaviour:
  -q, --quiet         be quiet, print essential output only
  -v, --verbose       be verbose, print informative messages
  -d, --debug         output more messages for debugging purpose
  -G, --groupbase     do not display group source prefix
  -L                  disable header block and order output by nodes (with -b or -B)
  -N                  disable labeling of command line
  -S                  return the largest of command return codes
  -b, --dshbak        display gathered results in a dshbak-like way
  -B                  like -b but including standard error
  -r, --regroup       fold nodeset using node groups
  -s GROUPSOURCE, --groupsource=GROUPSOURCE
                      optional ``groups.conf``\(5) group source to use

File copying:
  -c SOURCE_PATH, --copy=SOURCE_PATH
                      copy local file or directory to the nodes
  --dest=DEST_PATH    destination file or directory on the nodes
                      (optional: use the source directory path
                      when not specified)

  -p                  preserve modification times and modes

Ssh options:
  -f FANOUT, --fanout=FANOUT
                      use a specified fanout
  -l USER, --user=USER
                      execute remote command as user
  -o OPTIONS, --options=OPTIONS
                      can be used to give ssh options
  -t CONNECT_TIMEOUT, --connect_timeout=CONNECT_TIMEOUT
                      limit time to connect to a node
  -u COMMAND_TIMEOUT, --command_timeout=COMMAND_TIMEOUT
                      limit time for command to run on the node

For a short explanation of these options, see ``-h, --help``.


EXIT STATUS
===========

By default, an exit status of zero indicates success of the ``clush`` command
but gives no information about the remote commands exit status. However, when
the ``-S`` option is specified, the exit status of ``clush`` is the largest
value of the remote commands return codes.

For failed remote commands whose exit status is non-zero, ``clush`` displays
messages similar to:

:clush\: node[40-42]\: exited with exit code 1:


EXAMPLES
===========

:# clush -w node[3-5,62] uname -r:
    Run command `uname -r` on nodes: node3, node4, node5 and node62

:# clush -w node[3-5,62] -b uname -r:
    Run command `uname -r` on nodes[3-5,62] and display gathered output results (``dshbak``-like).

:# ssh node32 find /etc/yum.repos.d -type f | clush -w node[40-42] -b xargs ls -l:
    Search some files on node32 in /etc/yum.repos.d and use clush to list the matching ones on node[40-42], and use ``-b`` to display gathered results.

All/NodeGroups examples
-----------------------

:# clush -a uname -r:
    Run command `uname -r` on all cluster nodes, see ``clush.conf``\(5) to setup all cluster nodes (`nodes_all:` field).

:# clush -a -x node[5,7] uname -r:
    Run command `uname -r` on all cluster nodes except on nodes node5 and node7.

:# clush -g oss modprobe lustre:
    Run command `modprobe lustre` on nodes from node group named `oss`, see ``clush.conf``\(5) to setup node groups (`nodes_group:` field).

Copy files
----------
:# clush -w node[3-5,62] --copy /etc/motd:
    Copy local file `/etc/motd` to remote nodes node[3-5,62].

:# clush -w node[3-5,62] --copy /etc/motd --dest /tmp/motd2:
    Copy local file `/etc/motd` to remote nodes node[3-5,62] at path `/tmp/motd2`.

:# clush -w node[3-5,62] -c /usr/share/doc/clustershell:
    Recursively copy local directory `/usr/share/doc/clustershell` to the same
    path on remote nodes node[3-5,62].

FILES
=====

*/etc/clustershell/clush.conf*
  System-wide ``clush`` configuration file.
  
*~/.clush.conf*
  This is the per-user ``clush`` configuration file.

*~/.clush_history*
  File in which interactive ``clush`` command history is saved.

SEE ALSO
========

``clubak``\(1), ``nodeset``\(1), ``readline``\(3), ``clush.conf``\(5), ``groups.conf``\(5).

BUG REPORTS
===========

Use the following URL to submit a bug report or feedback:
  http://sourceforge.net/apps/trac/clustershell/report
Clone this wiki locally