CLI/Display: Add support for NO_COLOR and CLICOLOR (#428) (#432)

Adds support for NO_COLOR and CLICOLOR specification standard for ANSI
colors in terminals. --color option takes precedence over environment
variables. It also takes care of configuration file options.
Changes THREE_CHOICES index in order to test things correctly.

Closes #428.

Signed-off-by: Olivier DELHOMME <[email protected]>

Author: dupgit

doc/man/man1/clush.1

@@ -238,7 +238,7 @@ fold nodeset using node groups
 return the largest of command return codes
 .TP
 .BI \-\-color\fB= WHENCOLOR
-whether to use ANSI colors to surround node or nodeset prefix/header with escape sequences to display them in color on the terminal. \fIWHENCOLOR\fP is \fBnever\fP, \fBalways\fP or \fBauto\fP (which use color if standard output/error refer to a terminal). Colors are set to [34m (blue foreground text) for stdout and [31m (red foreground text) for stderr, and cannot be modified.
+\fBclush\fP can use NO_COLOR, CLICOLOR and CLICOLOR_FORCE environment variables. NO_COLOR takes precedence over CLICOLOR_FORCE which takes precedence over CLICOLOR.  When \fB\-\-color\fP option is used these environment variables are not taken into account. \fB\-\-color\fP tells whether to use ANSI colors to surround node or nodeset prefix/header with escape sequences to display them in color on the terminal. \fIWHENCOLOR\fP is \fBnever\fP, \fBalways\fP or \fBauto\fP (which use color if standard output/error refer to a terminal). Colors are set to [34m (blue foreground text) for stdout and [31m (red foreground text) for stderr, and cannot be modified.
 .TP
 .B \-\-diff
 show diff between common outputs (find the best reference output by focusing on largest nodeset and also smaller command return code)

doc/sphinx/tools/clush.rst

@@ -589,6 +589,8 @@ output nodeset header::
     ---------------
     ok
 
+NO_COLOR, CLICOLOR_FORCE and CLICOLOR environment variables can also
+be used to change the way *clush* uses colors to display messages.
 
 .. _clush-worker:
 

doc/txt/clubak.txt

@@ -57,7 +57,7 @@ OPTIONS
                node / line content separator string (default: `:`)
 -F, --fast     faster but memory hungry mode (preload all messages per node)
 -T, --tree     message tree trace mode; switch to enable ``ClusterShell.MsgTree`` trace mode, all keys/nodes being kept for each message element of the tree, thus allowing special output gathering
---color=WHENCOLOR   whether to use ANSI colors to surround node or nodeset prefix/header with escape sequences to display them in color on the terminal. *WHENCOLOR* is ``never``, ``always`` or ``auto`` (which use color if standard output refers to a terminal). Color is set to [34m (blue foreground text) and cannot be modified.
+--color=WHENCOLOR   ``clush`` can use NO_COLOR, CLICOLOR and CLICOLOR_FORCE environment variables. ``--color`` command line option always takes precedence over environment variables. NO_COLOR takes precedence over CLICOLOR_FORCE which takes precedence over CLICOLOR. ``--color`` tells whether to use ANSI colors to surround node or nodeset prefix/header with escape sequences to display them in color on the terminal. *WHENCOLOR* is ``never``, ``always`` or ``auto`` (which use color if standard output refers to a terminal). Color is set to [34m (blue foreground text) and cannot be modified.
 --diff         show diff between gathered outputs
 
 

doc/txt/clush.conf.txt

@@ -54,12 +54,16 @@ command_timeout
   ClusterShell library ensures that any commands complete in less than
   ( connect_timeout + command_timeout ). If set to *0*, no timeout occurs.
 color
-  Whether to use ANSI colors to surround node or nodeset prefix/header with
-  escape sequences to display them in color on the terminal. Valid arguments
-  are ``never``, ``always`` or ``auto`` (which use color if standard
-  output/error refer to a terminal). Colors are set to [34m (blue foreground
-  text) for stdout and [31m (red foreground text) for stderr, and cannot be
-  modified.
+  ``clush`` can use NO_COLOR, CLICOLOR and CLICOLOR_FORCE
+  environment variables. NO_COLOR takes precedence over CLICOLOR_FORCE which
+  takes precedence over CLICOLOR. When the option is set in configuration file
+  environment variables are taken into account only with `auto` argument.
+  ``color`` tells  whether to use ANSI colors to surround node or nodeset
+  prefix/header with escape sequences to display them in color on the terminal.
+  Valid arguments are ``never``, ``always`` or ``auto`` (which use color if
+  standard output/error refer to a terminal). Colors are set to [34m (blue
+  foreground text) for stdout and [31m (red foreground text) for stderr, and
+  cannot be modified.
 fd_max
   Maximum number of open file descriptors permitted per clush process (soft
   resource limit for open files). This limit can never exceed the system

doc/txt/clush.txt

@@ -160,7 +160,7 @@ Output behaviour:
   -B                  like -b but including standard error
   -r, --regroup       fold nodeset using node groups
   -S, --maxrc         return the largest of command return codes
-  --color=WHENCOLOR   whether to use ANSI colors to surround node or nodeset prefix/header with escape sequences to display them in color on the terminal. *WHENCOLOR* is ``never``, ``always`` or ``auto`` (which use color if standard output/error refer to a terminal). Colors are set to [34m (blue foreground text) for stdout and [31m (red foreground text) for stderr, and cannot be modified.
+  --color=WHENCOLOR   ``clush`` can use NO_COLOR, CLICOLOR and CLICOLOR_FORCE environment variables. NO_COLOR takes precedence over CLICOLOR_FORCE which takes precedence over CLICOLOR.  When ``--color`` option is used these environment variables are not taken into account. ``--color`` tells whether to use ANSI colors to surround node or nodeset prefix/header with escape sequences to display them in color on the terminal. *WHENCOLOR* is ``never``, ``always`` or ``auto`` (which use color if standard output/error refer to a terminal). Colors are set to [34m (blue foreground text) for stdout and [31m (red foreground text) for stderr, and cannot be modified.
   --diff              show diff between common outputs (find the best reference output by focusing on largest nodeset and also smaller command return code)
 
 File copying:

lib/ClusterShell/CLI/Clubak.py

@@ -107,7 +107,7 @@ def clubak():
     if options.interpret_keys == THREE_CHOICES[-1]: # auto?
         enable_nodeset_key = None # AUTO
     else:
-        enable_nodeset_key = (options.interpret_keys == THREE_CHOICES[1])
+        enable_nodeset_key = (options.interpret_keys == THREE_CHOICES[2])
 
     # Create new message tree
     if options.trace_mode:

lib/ClusterShell/CLI/Config.py

@@ -53,7 +53,7 @@ class ClushConfig(configparser.ConfigParser, object):
                      "connect_timeout": "%f" % DEFAULTS.connect_timeout,
                      "command_timeout": "%f" % DEFAULTS.command_timeout,
                      "history_size": "100",
-                     "color": THREE_CHOICES[-1], # auto
+                     "color": THREE_CHOICES[0], # ''
                      "verbosity": "%d" % VERB_STD,
                      "node_count": "yes",
                      "maxrc": "no",

lib/ClusterShell/CLI/Display.py

@@ -25,6 +25,7 @@
 
 import difflib
 import sys
+import os
 
 from ClusterShell.NodeSet import NodeSet
 
@@ -33,7 +34,7 @@
 VERB_STD = 1
 VERB_VERB = 2
 VERB_DEBUG = 3
-THREE_CHOICES = ["never", "always", "auto"]
+THREE_CHOICES = ["", "never", "always", "auto"]
 WHENCOLOR_CHOICES = THREE_CHOICES   # deprecated; use THREE_CHOICES
 
 if sys.getdefaultencoding() == 'ascii':
@@ -98,6 +99,18 @@ def __init__(self, options, config=None, color=None):
         # display may change when 'max return code' option is set
         self.maxrc = getattr(options, 'maxrc', False)
 
+        # Be compliant with NO_COLOR and CLI_COLORS trying to solve #428
+        # See https://no-color.org/ and https://bixense.com/clicolors/
+        # NO_COLOR takes precedence over CLI_COLORS. --color option always
+        # takes precedence over any environment variable.
+
+        if options.whencolor is None:
+            if (config is None) or (config.color == '' or config.color == 'auto'):
+                if 'NO_COLOR' not in os.environ:
+                    color = self._has_cli_color()
+                else:
+                    color = False
+
         if color is None:
             # Should we use ANSI colors?
             color = False
@@ -137,6 +150,26 @@ def __init__(self, options, config=None, color=None):
             if hasattr(options, 'debug') and options.debug:
                 self.verbosity = VERB_DEBUG
 
+    def _has_cli_color(self):
+        """Tests CLICOLOR environment variable to determine wether to
+        use color or not on output."""
+        # When CLICOLOR_FORCE is set to something else than 0
+        # colors must be used.
+        if os.getenv("CLICOLOR_FORCE", "0") != "0":
+            return True
+
+        cli_color = os.getenv("CLICOLOR")
+
+        if cli_color is None:
+            return None
+        elif cli_color != "0":
+            # CLICOLOR is set and colored output should
+            # be used if stdout is a tty
+            return sys.stdout.isatty()
+        else:
+            # CLICOLOR is set to not display colors.
+            return False
+
     def flush(self):
         """flush display object buffers"""
         # only used to reset diff display for now

tests/CLIConfigTest.py

@@ -35,7 +35,7 @@ def testClushConfigEmpty(self):
         parser.install_connector_options()
         options, _ = parser.parse_args([])
         config = ClushConfig(options, filename=f.name)
-        self.assertEqual(config.color, WHENCOLOR_CHOICES[-1])
+        self.assertEqual(config.color, THREE_CHOICES[0])
         self.assertEqual(config.verbosity, VERB_STD)
         self.assertEqual(config.fanout, 64)
         self.assertEqual(config.maxrc, False)
@@ -59,7 +59,7 @@ def testClushConfigAlmostEmpty(self):
         parser.install_connector_options()
         options, _ = parser.parse_args([])
         config = ClushConfig(options, filename=f.name)
-        self.assertEqual(config.color, WHENCOLOR_CHOICES[-1])
+        self.assertEqual(config.color, THREE_CHOICES[0])
         self.assertEqual(config.verbosity, VERB_STD)
         self.assertEqual(config.maxrc, False)
         self.assertEqual(config.node_count, True)
@@ -96,7 +96,7 @@ def testClushConfigDefault(self):
         display = Display(options, config)
         display.vprint(VERB_STD, "test")
         display.vprint(VERB_DEBUG, "shouldn't see this")
-        self.assertEqual(config.color, WHENCOLOR_CHOICES[2])
+        self.assertEqual(config.color, THREE_CHOICES[-1])
         self.assertEqual(config.verbosity, VERB_STD)
         self.assertEqual(config.maxrc, False)
         self.assertEqual(config.node_count, True)
@@ -134,7 +134,7 @@ def testClushConfigFull(self):
         parser.install_connector_options()
         options, _ = parser.parse_args([])
         config = ClushConfig(options, filename=f.name)
-        self.assertEqual(config.color, WHENCOLOR_CHOICES[2])
+        self.assertEqual(config.color, THREE_CHOICES[-1])
         self.assertEqual(config.verbosity, VERB_STD)
         self.assertEqual(config.maxrc, True)
         self.assertEqual(config.node_count, True)
@@ -298,7 +298,7 @@ def testClushConfigDefaultWithOptions(self):
         display = Display(options, config)
         display.vprint(VERB_STD, "test")
         display.vprint(VERB_DEBUG, "test")
-        self.assertEqual(config.color, WHENCOLOR_CHOICES[1])
+        self.assertEqual(config.color, THREE_CHOICES[2])
         self.assertEqual(config.verbosity, VERB_DEBUG) # takes biggest
         self.assertEqual(config.fanout, 36)
         self.assertEqual(config.connect_timeout, 7)
@@ -360,7 +360,7 @@ def testClushConfigUserOverride(self):
             parser.install_connector_options()
             options, _ = parser.parse_args([])
             config = ClushConfig(options) # filename=None to use defaults!
-            self.assertEqual(config.color, WHENCOLOR_CHOICES[0])
+            self.assertEqual(config.color, THREE_CHOICES[1])
             self.assertEqual(config.verbosity, VERB_VERB) # takes biggest
             self.assertEqual(config.fanout, 42)
             self.assertEqual(config.connect_timeout, 14)

tests/CLIDisplayTest.py

@@ -5,9 +5,10 @@
 
 import tempfile
 import unittest
+import os
 from io import BytesIO
 
-from ClusterShell.CLI.Display import Display, WHENCOLOR_CHOICES, VERB_STD
+from ClusterShell.CLI.Display import Display, THREE_CHOICES, VERB_STD
 from ClusterShell.CLI.OptionParser import OptionParser
 
 from ClusterShell.MsgTree import MsgTree
@@ -38,27 +39,46 @@ def testDisplay(self):
         mtree.add("hostfoo", b"message0")
         mtree.add("hostfoo", b"message1")
 
-        for whencolor in WHENCOLOR_CHOICES: # test whencolor switch
-            for label in [True, False]:     # test no-label switch
-                options.label = label
-                options.whencolor = whencolor
-                disp = Display(options)
-                # inhibit output
-                disp.out = BytesIO()
-                disp.err = BytesIO()
-                # test print_* methods...
-                disp.print_line(ns, b"foo bar")
-                disp.print_line_error(ns, b"foo bar")
-                disp.print_gather(ns, list(mtree.walk())[0][0])
-                # test also string nodeset as parameter
-                disp.print_gather("hostfoo", list(mtree.walk())[0][0])
-                # test line_mode property
-                self.assertEqual(disp.line_mode, False)
-                disp.line_mode = True
-                self.assertEqual(disp.line_mode, True)
-                disp.print_gather("hostfoo", list(mtree.walk())[0][0])
-                disp.line_mode = False
-                self.assertEqual(disp.line_mode, False)
+        list_env_vars = []
+        list_env_vars.append(dict())
+        list_env_vars.append(dict(NO_COLOR='0'))
+        list_env_vars.append(dict(CLICOLOR='0'))
+        list_env_vars.append(dict(CLICOLOR='1'))
+        list_env_vars.append(dict(CLICOLOR='0', CLICOLOR_FORCE='0'))
+        list_env_vars.append(dict(CLICOLOR_FORCE='1'))
+
+        for env_vars in list_env_vars:
+            for var_name in env_vars:
+                var_value = env_vars[var_name]
+                os.environ[var_name] = var_value
+
+            for whencolor in THREE_CHOICES: # test whencolor switch
+                if whencolor == "":
+                    options.whencolor = None
+                else:
+                    options.whencolor = whencolor
+                for label in [True, False]: # test no-label switch
+                    options.label = label
+                    disp = Display(options)
+                    # inhibit output
+                    disp.out = BytesIO()
+                    disp.err = BytesIO()
+                    # test print_* methods...
+                    disp.print_line(ns, b"foo bar")
+                    disp.print_line_error(ns, b"foo bar")
+                    disp.print_gather(ns, list(mtree.walk())[0][0])
+                    # test also string nodeset as parameter
+                    disp.print_gather("hostfoo", list(mtree.walk())[0][0])
+                    # test line_mode property
+                    self.assertEqual(disp.line_mode, False)
+                    disp.line_mode = True
+                    self.assertEqual(disp.line_mode, True)
+                    disp.print_gather("hostfoo", list(mtree.walk())[0][0])
+                    disp.line_mode = False
+                    self.assertEqual(disp.line_mode, False)
+
+            for var_name in env_vars:
+                os.environ.pop(var_name)
 
     def testDisplayRegroup(self):
         """test CLI.Display (regroup)"""