Merged master into transcript_fixes branch and resolved conflicts

Author: tleonhardt

.gitignore

@@ -25,3 +25,6 @@ htmlcov
 # mypy plugin for PyCharm
 dmypy.json
 dmypy.sock
+
+# cmd2 history file used in main.py
+cmd2_history.txt

.travis.yml

@@ -14,7 +14,9 @@ matrix:
           python: 3.6
           env: TOXENV=py36
         - os: linux
-          python: 3.7-dev
+          python: 3.7
+          dist: xenial
+          sudo: true  # Travis CI doesn't yet support official (non-development) Python 3.7 on container-based builds
           env: TOXENV=py37
         - os: linux
           python: 3.5

CHANGELOG.md

@@ -2,11 +2,37 @@
 * Bug Fixes
     * Fixed bug where ``get_all_commands`` could return non-callable attributes
     * Fixed bug where **alias** command was dropping quotes around arguments
+    * Fixed bug where running help on argparse commands didn't work if they didn't support -h
 * Enhancements
     * Added ``exit_code`` attribute of ``cmd2.Cmd`` class
         * Enables applications to return a non-zero exit code when exiting from ``cmdloop``
     * ``ACHelpFormatter`` now inherits from ``argparse.RawTextHelpFormatter`` to make it easier
     for formatting help/description text
+    * Aliases are now sorted alphabetically
+    * The **set** command now tab-completes settable parameter names
+    * Added ``async_alert``, ``async_update_prompt``, and ``set_window_title`` functions
+        * These allow you to provide feedback to the user in an asychronous fashion, meaning alerts can
+        display when the user is still entering text at the prompt. See [async_printing.py](https://github.com/python-cmd2/cmd2/blob/master/examples/async_printing.py)
+        for an example.
+    * Cross-platform colored output support 
+        * ``colorama`` gets initialized properly in ``Cmd.__init()``
+        * The ``Cmd.colors`` setting is no longer platform dependent and now has three values:
+            * Terminal (default) - output methods do not strip any ANSI escape sequences when output is a terminal, but 
+            if the output is a pipe or a file the escape sequences are stripped
+            * Always - output methods **never** strip ANSI escape sequences, regardless of the output destination
+            * Never - output methods strip all ANSI escape sequences
+    * Added ``macro`` command to create macros, which are similar to aliases, but can take arguments when called
+    * All cmd2 command functions have been converted to use argparse.
+* Deprecations
+    * Deprecated the built-in ``cmd2`` support for colors including ``Cmd.colorize()`` and ``Cmd._colorcodes``
+* Deletions (potentially breaking changes)
+    * The ``preparse``, ``postparsing_precmd``, and ``postparsing_postcmd`` methods *deprecated* in the previous release 
+    have been deleted
+        * The new application lifecycle hook system allows for registration of callbacks to be called at various points
+        in the lifecycle and is more powerful and flexible than the previous system
+    * ``alias`` is now a command with subcommands to create, list, and delete aliases. Therefore its syntax
+      has changed. All current alias commands in startup scripts or transcripts will break with this release.
+    * `unalias` was deleted since ``alias delete`` replaced it
 
 ## 0.9.4 (August 21, 2018)
 * Bug Fixes
@@ -118,7 +144,7 @@
     * Fixed ``AttributeError`` on Windows when running a ``select`` command cause by **pyreadline** not implementing ``remove_history_item``
 * Enhancements
     * Added warning about **libedit** variant of **readline** not being supported on macOS
-    * Added tab-completion of alias names in value filed of **alias** command
+    * Added tab-completion of alias names in value field of **alias** command
     * Enhanced the ``py`` console in the following ways
         * Added tab completion of Python identifiers instead of **cmd2** commands
         * Separated the ``py`` console history from the **cmd2** history

CONTRIBUTING.md

@@ -14,8 +14,8 @@ applications.  It provides a simple API which is an extension of Python's built-
 of cmd to make your life easier and eliminates much of the boilerplate code which would be necessary
 when using cmd.
 
-[![Screenshot](cmd2.png)](https://github.com/python-cmd2/cmd2/blob/master/cmd2.png)
-
+Click on image below to watch a short video demonstrating the capabilities of cmd2:
+[![Screenshot](cmd2.png)](https://youtu.be/DDU_JH6cFsA)
 
 Main Features
 -------------
@@ -31,6 +31,7 @@ Main Features
 - Multi-line commands
 - Special-character command shortcuts (beyond cmd's `?` and `!`)
 - Command aliasing similar to bash `alias` command
+- Macros, which are similar to aliases, but can take arguments when called
 - Ability to load commands at startup from an initialization script
 - Settable environment parameters
 - Parsing commands with arguments using `argparse`, including support for sub-commands
@@ -40,6 +41,7 @@ Main Features
 - Trivial to provide built-in help for all commands
 - Built-in regression testing framework for your applications (transcript-based testing)
 - Transcripts for use with built-in regression can be automatically generated from `history -t`
+- Alerts that seamlessly print while user enters text at prompt
 
 Python 2.7 support is EOL
 -------------------------
@@ -259,3 +261,15 @@ timing: False
 Note how a regular expression `/(True|False)/` is used for output of the **show color** command since
 colored text is currently not available for cmd2 on Windows.  Regular expressions can be used anywhere within a
 transcript file simply by enclosing them within forward slashes, `/`.
+
+
+Found a bug?
+------------
+
+If you think you've found a bug, please first read through the open [Issues](https://github.com/python-cmd2/cmd2/issues). If you're confident it's a new bug, go ahead and create a new GitHub issue. Be sure to include as much information as possible so we can reproduce the bug.  At a minimum, please state the following:
+
+* ``cmd2`` version
+* Python version
+* OS name and version
+* What you did to cause the bug to occur
+* Include any traceback or error message associated with the bug

README.md

@@ -23,16 +23,18 @@
     import os
     import shlex
     import sys
+    from typing import List, Tuple, Union
 
     from . import constants
     from . import utils
 
 
-    def tokens_for_completion(line, endidx):
+    def tokens_for_completion(line: str, endidx: int) -> Union[Tuple[List[str], List[str], int, int],
+                                                               Tuple[None, None, None, None]]:
         """
         Used by tab completion functions to get all tokens through the one being completed
-        :param line: str - the current input line with leading whitespace removed
-        :param endidx: int - the ending index of the prefix text
+        :param line: the current input line with leading whitespace removed
+        :param endidx: the ending index of the prefix text
         :return: A 4 item tuple where the items are
                  On Success
                      tokens: list of unquoted tokens
@@ -46,7 +48,7 @@ def tokens_for_completion(line, endidx):
                      The last item in both lists is the token being tab completed
 
                  On Failure
-                    Both items are None
+                    All 4 items are None
         """
         unclosed_quote = ''
         quotes_to_try = copy.copy(constants.QUOTES)

cmd2.png

@@ -510,6 +510,18 @@ def complete_command_help(self, tokens: List[str], text: str, line: str, begidx:
                             return self.basic_complete(text, line, begidx, endidx, completers.keys())
         return []
 
+    def format_help(self, tokens: List[str]) -> str:
+        """Supports the completion of sub-commands for commands through the cmd2 help command."""
+        for idx, token in enumerate(tokens):
+            if idx >= self._token_start_index:
+                if self._positional_completers:
+                    # For now argparse only allows 1 sub-command group per level
+                    # so this will only loop once.
+                    for completers in self._positional_completers.values():
+                        if token in completers:
+                            return completers[token].format_help(tokens)
+        return self._parser.format_help()
+
     @staticmethod
     def _process_action_nargs(action: argparse.Action, arg_state: _ArgumentState) -> None:
         if isinstance(action, _RangeAction):
@@ -654,12 +666,19 @@ def _print_action_help(self, action: argparse.Action) -> None:
             else:
                 prefix = ''
 
+        if action.help is None:
+            help_text = ''
+        else:
+            help_text = action.help
+
+        # is there anything to print for this parameter?
+        if not prefix and not help_text:
+            return
+
         prefix = '  {0: <{width}}    '.format(prefix, width=20)
         pref_len = len(prefix)
-        if action.help is not None:
-            help_lines = action.help.splitlines()
-        else:
-            help_lines = ['']
+        help_lines = help_text.splitlines()
+
         if len(help_lines) == 1:
             print('\nHint:\n{}{}\n'.format(prefix, help_lines[0]))
         else:
@@ -676,12 +695,12 @@ def basic_complete(text: str, line: str, begidx: int, endidx: int, match_against
         """
         Performs tab completion against a list
 
-        :param text: str - the string prefix we are attempting to match (all returned matches must begin with it)
-        :param line: str - the current input line with leading whitespace removed
-        :param begidx: int - the beginning index of the prefix text
-        :param endidx: int - the ending index of the prefix text
-        :param match_against: Collection - the list being matched against
-        :return: List[str] - a list of possible tab completions
+        :param text: the string prefix we are attempting to match (all returned matches must begin with it)
+        :param line: the current input line with leading whitespace removed
+        :param begidx: the beginning index of the prefix text
+        :param endidx: the ending index of the prefix text
+        :param match_against: the list being matched against
+        :return: a list of possible tab completions
         """
         return [cur_match for cur_match in match_against if cur_match.startswith(text)]
 
@@ -731,7 +750,7 @@ def _format_usage(self, usage, actions, groups, prefix) -> str:
 
             # build full usage string
             format = self._format_actions_usage
-            action_usage = format(positionals + required_options + optionals, groups)
+            action_usage = format(required_options + optionals + positionals, groups)
             usage = ' '.join([s for s in [prog, action_usage] if s])
 
             # wrap the usage parts if it's too long
@@ -742,15 +761,15 @@ def _format_usage(self, usage, actions, groups, prefix) -> str:
 
                 # break usage into wrappable parts
                 part_regexp = r'\(.*?\)+|\[.*?\]+|\S+'
+                req_usage = format(required_options, groups)
                 opt_usage = format(optionals, groups)
                 pos_usage = format(positionals, groups)
-                req_usage = format(required_options, groups)
+                req_parts = _re.findall(part_regexp, req_usage)
                 opt_parts = _re.findall(part_regexp, opt_usage)
                 pos_parts = _re.findall(part_regexp, pos_usage)
-                req_parts = _re.findall(part_regexp, req_usage)
+                assert ' '.join(req_parts) == req_usage
                 assert ' '.join(opt_parts) == opt_usage
                 assert ' '.join(pos_parts) == pos_usage
-                assert ' '.join(req_parts) == req_usage
 
                 # End cmd2 customization
 
@@ -780,13 +799,15 @@ def get_lines(parts, indent, prefix=None):
                 if len(prefix) + len(prog) <= 0.75 * text_width:
                     indent = ' ' * (len(prefix) + len(prog) + 1)
                     # Begin cmd2 customization
-                    if opt_parts:
-                        lines = get_lines([prog] + pos_parts, indent, prefix)
-                        lines.extend(get_lines(req_parts, indent))
+                    if req_parts:
+                        lines = get_lines([prog] + req_parts, indent, prefix)
                         lines.extend(get_lines(opt_parts, indent))
+                        lines.extend(get_lines(pos_parts, indent))
+                    elif opt_parts:
+                        lines = get_lines([prog] + opt_parts, indent, prefix)
+                        lines.extend(get_lines(pos_parts, indent))
                     elif pos_parts:
                         lines = get_lines([prog] + pos_parts, indent, prefix)
-                        lines.extend(get_lines(req_parts, indent))
                     else:
                         lines = [prog]
                     # End cmd2 customization
@@ -795,13 +816,13 @@ def get_lines(parts, indent, prefix=None):
                 else:
                     indent = ' ' * len(prefix)
                     # Begin cmd2 customization
-                    parts = pos_parts + req_parts + opt_parts
+                    parts = req_parts + opt_parts + pos_parts
                     lines = get_lines(parts, indent)
                     if len(lines) > 1:
                         lines = []
-                        lines.extend(get_lines(pos_parts, indent))
                         lines.extend(get_lines(req_parts, indent))
                         lines.extend(get_lines(opt_parts, indent))
+                        lines.extend(get_lines(pos_parts, indent))
                     # End cmd2 customization
                     lines = [prog] + lines
 
@@ -870,6 +891,9 @@ def _format_args(self, action, default_metavar) -> str:
             result = super()._format_args(action, default_metavar)
         return result
 
+    def format_help(self):
+        return super().format_help() + '\n'
+
 
 # noinspection PyCompatibility
 class ACArgumentParser(argparse.ArgumentParser):