Merge pull request #370 from python-cmd2/ply

Switch parsing logic from pyparsing to shlex

Author: kotfu

CHANGELOG.md

@@ -1,10 +1,24 @@
 ## 0.9.0 (TBD, 2018)
+* Bug Fixes
+    * If self.default_to_shell is true, then redirection and piping are now properly passed to the shell. Previously it was truncated.
+    * Submenus now call all hooks, it used to just call precmd and postcmd.
 * Enhancements
     * Automatic completion of ``argparse`` arguments via ``cmd2.argparse_completer.AutoCompleter``
         * See the [tab_autocompletion.py](https://github.com/python-cmd2/cmd2/blob/master/examples/tab_autocompletion.py) example for a demonstration of how to use this feature
     * ``cmd2`` no longer depends on the ``six`` module
     * ``cmd2`` is now a multi-file Python package instead of a single-file module
     * New pyscript approach that provides a pythonic interface to commands in the cmd2 application.
+    * Switch command parsing from pyparsing to custom code which utilizes shlex.
+        * The object passed to do_* methods has changed. It no longer is the pyparsing object, it's a new Statement object, which is a subclass of ``str``. The statement object has many attributes which give you access to various components of the parsed input. If you were using anything but the string in your do_* methods, this change will require you to update your code.
+        * ``commentGrammers`` is no longer supported or available. Comments are C-style or python style.
+        * Input redirection no longer supported. Use the load command instead.
+        * ``multilineCommand`` attribute is ``now multiline_command``
+        * ``identchars`` is now ignored. The standardlibrary cmd uses those characters to split the first "word" of the input, but cmd2 hasn't used those for a while, and the new parsing logic parses on whitespace, which has the added benefit of full unicode support, unlike cmd or prior versions of cmd2.
+        * ``set_posix_shlex`` function and ``POSIX_SHLEX`` variable have been removed. Parsing behavior is now always the more forgiving ``posix=false``.
+        * ``set_strip_quotes`` function and ``STRIP_QUOTES_FOR_NON_POSIX`` have been removed. Quotes are stripped from arguments when presented as a list (a la ``sys.argv``), and present when arguments are presented as a string (like the string passed to do_*).
+* Changes
+    * ``strip_ansi()`` and ``strip_quotes()`` functions have moved to new utils module
+    * Several constants moved to new constants module
 * Deletions (potentially breaking changes)
     * Deleted all ``optparse`` code which had previously been deprecated in release 0.8.0
         * The ``options`` decorator no longer exists

CONTRIBUTING.md

@@ -45,7 +45,6 @@ The tables below list all prerequisites along with the minimum required version
 | Prerequisite                                        | Minimum Version |
 | --------------------------------------------------- | --------------- |
 | [Python](https://www.python.org/downloads/)         | `3.4`           |
-| [pyparsing](http://pyparsing.wikispaces.com)        | `2.1`           |
 | [pyperclip](https://github.com/asweigart/pyperclip) | `1.6`           |
 
 #### Additional prerequisites to run cmd2 unit tests
@@ -63,15 +62,13 @@ The tables below list all prerequisites along with the minimum required version
 ### Optional prerequisites for enhanced unit test features
 | Prerequisite                                | Minimum Version |
 | ------------------------------------------- | --------------- |
-| [pytest-forked](https://pypi.python.org/pypi/pytest-forked)| `0.2` |
-| [pytest-xdist](https://pypi.python.org/pypi/pytest-xdist)| `1.15` |
 | [pytest-cov](https://pypi.python.org/pypi/pytest-cov) | `1.8` |
 
 If Python is already installed in your machine, run the following commands to validate the versions:
 
 ```shell
 python -V
-pip freeze | grep pyparsing
+pip freeze | grep pyperclip
 ```
 
 If your versions are lower than the prerequisite versions, you should update.
@@ -190,16 +187,16 @@ Once you have cmd2 cloned, before you start any cmd2 application, you first need
 
 ```bash
 # Install cmd2 prerequisites
-pip install -U pyparsing pyperclip
+pip install -U pyperclip
 
 # Install prerequisites for running cmd2 unit tests
 pip install -U pytest
 
 # Install prerequisites for building cmd2 documentation
 pip install -U sphinx sphinx-rtd-theme
 
-# Install optional prerequisites for running unit tests in parallel and doing code coverage analysis
-pip install -U pytest-xdist pytest-cov pytest-forked
+# Install optional prerequisites for doing code coverage analysis
+pip install -U pytest-cov
 ```
 
 For doing cmd2 development, you actually do NOT want to have cmd2 installed as a Python package.  
@@ -259,35 +256,21 @@ py.test
 
 and ensure all tests pass.
 
-If you have the `pytest-xdist` pytest distributed testing plugin installed, then you can use it to
-dramatically speed up test execution by running tests in parallel on multiple cores like so:
-
-```shell
-py.test -n4
-```
-where `4` should be replaced by the number of parallel threads you wish to run for testing.
-
-If you have the `pytest-forked` pytest plugin (not avilable on Windows) for running tests in isolated formed processes, 
-you can speed things up even further:
-
-```shell
-py.test -nauto --forked
-```
 
 #### Measuring code coverage
 
 Code coverage can be measured as follows:
 
 ```shell
-py.test -nauto --cov=cmd2 --cov-report=term-missing --cov-report=html --forked
+py.test --cov=cmd2 --cov-report=term-missing --cov-report=html
 ```
 
 Then use your web browser of choice to look at the results which are in `<cmd2>/htmlcov/index.html`.
 
 ### Squash Your Commits
 When you make a pull request, it is preferable for all of your changes to be in one commit.
 
-If you have made more then one commit, then you will can _squash_ your commits.
+If you have made more then one commit, then you can _squash_ your commits.
 
 To do this, see [Squashing Your Commits](http://forum.freecodecamp.com/t/how-to-squash-multiple-commits-into-one-with-git/13231).
 

README.md

@@ -23,7 +23,7 @@ Main Features
 - Python scripting of your application with ``pyscript``
 - Run shell commands with ``!``
 - Pipe command output to shell commands with `|`
-- Redirect command output to file with `>`, `>>`; input from file with `<`
+- Redirect command output to file with `>`, `>>`
 - Bare `>`, `>>` with no filename send output to paste buffer (clipboard)
 - `py` enters interactive Python console (opt-in `ipy` for IPython console)
 - Option to display long output using a pager with ``cmd2.Cmd.ppaged()``
@@ -57,7 +57,7 @@ pip install -U cmd2
 ```
 
 cmd2 works with Python 3.4+ on Windows, macOS, and Linux. It is pure Python code with
-the only 3rd-party dependencies being on [pyparsing](http://pyparsing.wikispaces.com), and [pyperclip](https://github.com/asweigart/pyperclip).
+the only 3rd-party dependencies being on [colorama](https://github.com/tartley/colorama), and [pyperclip](https://github.com/asweigart/pyperclip).
 Windows has an additional dependency on [pyreadline](https://pypi.python.org/pypi/pyreadline). Non-Windows platforms
 have an additional dependency on [wcwidth](https://pypi.python.org/pypi/wcwidth). Finally, Python
 3.4 has an additional dependency on [contextlib2](https://pypi.python.org/pypi/contextlib2).
@@ -91,7 +91,7 @@ Instructions for implementing each feature follow.
 
 - Multi-line commands
 
-    Any command accepts multi-line input when its name is listed in `Cmd.multilineCommands`.
+    Any command accepts multi-line input when its name is listed in `Cmd.multiline_commands`.
     The program will keep expecting input until a line ends with any of the characters
     in `Cmd.terminators` .  The default terminators are `;` and `/n` (empty newline).
 
@@ -165,7 +165,7 @@ class CmdLineApp(cmd2.Cmd):
     MUMBLE_LAST = ['right?']
 
     def __init__(self):
-        self.multilineCommands = ['orate']
+        self.multiline_commands = ['orate']
         self.maxrepeats = 3
 
         # Add stuff to settable and shortcuts before calling base class initializer