Merge pull request #205 from haskell/sphinx

Author: andreasabel

.gitignore

@@ -7,15 +7,6 @@
 /data/
 /dist
 /dist-newstyle
-/doc/alex.1
-/doc/alex.pdf
-/doc/alex/*.html
-/doc/alex/fptools.css
-/doc/autom4te.cache/
-/doc/configure
-/doc/config.log
-/doc/config.mk
-/doc/config.status
 /examples/*.alex.hs
 /examples/*.happy.hs
 /examples/*.bin

CHANGELOG.md

@@ -111,23 +111,112 @@
  * Unicode support (contributed mostly by Jean-Philippe Bernardy,
    with help from Alan Zimmerman).
 
-   * An Alex lexer now takes a UTF-8 encoded byte sequence as input
-     (see Section 5.1, “Unicode and UTF-8”. If you are using the
+   * An Alex lexer now takes a UTF-8 encoded byte sequence as input.
+     If you are using the
      "basic" wrapper or one of the other wrappers that takes a
      Haskell String as input, the string is automatically encoded
-     into UTF-8 by Alex. If your input is a ByteString, you are
-     responsible for ensuring that the input is UTF-8 encoded. The
-     old 8-bit behaviour is still available via the --latin1
-     option.
+     into UTF-8 by Alex. If your input is a `ByteString`, you are
+     responsible for ensuring that the input is UTF-8 encoded.
 
    * Alex source files are assumed to be in UTF-8, like Haskell
      source files. The lexer specification can use Unicode
      characters and ranges.
 
    * `alexGetChar` is renamed to `alexGetByte` in the generated code.
 
-   * There is a new option, `--latin1`, that restores the old
+   * There is a new option, `--latin1`, that restores the old 8-bit
      behaviour.
 
  * Alex now does DFA minimization, which helps to reduce the size
    of the generated tables, especially for lexers that use Unicode.
+
+## Release Notes for version 2.2
+
+-   `Cabal-1.2` is now required.
+
+-   `ByteString` wrappers: use Alex to lex ByteStrings directly.
+
+## Release Notes for version 2.1.0
+
+-   Switch to a Cabal build system: you need a recent version of Cabal
+    (1.1.6 or later). If you have GHC 6.4.2, then you need to upgrade
+    Cabal before building Alex. GHC 6.6 is fine.
+
+-   Slight change in the error semantics: the input returned on error is
+    before the erroneous character was read, not after. This helps to
+    give better error messages.
+
+## Release Notes for version 2.0
+
+Alex has changed a *lot* between versions 1.x and 2.0. The following is
+supposed to be an exhaustive list of the changes:
+
+### Syntax changes
+
+-   Code blocks are now surrounded by `{...}` rather than `%{...%}`.
+
+-   Character-set macros now begin with ‘`$`’ instead of ‘`^`’ and have
+    multi-character names.
+
+-   Regular expression macros now begin with ‘`@`’ instead of ‘`%`’ and
+    have multi-character names.
+
+-   Macro definitions are no longer surrounded by `{ ... }`.
+
+-   Rules are now of the form
+
+        <c1,c2,...>  regex   { code }
+
+    where `c1`, `c2` are startcodes, and `code` is an arbitrary Haskell
+    expression.
+
+-   Regular expression syntax changes:
+
+    -   `()` is the empty regular expression (used to be ‘`$`’)
+
+    -   set complement can now be expressed as `[^sets]` (for similarity
+        with lex regular expressions).
+
+    -   The `'abc'` form is no longer available, use `[abc]` instead.
+
+    -   ‘`^`’ and ‘`$`’ have the usual meanings: ‘`^`’ matches just
+        after a ‘`\n`’, and ‘`$`’ matches just before a ‘`\n`’.
+
+    -   ‘`\n`’ is now the escape character, not ‘`^`’.
+
+    -   The form `"..."` means the same as the sequence of characters
+        inside the quotes, the difference being that special characters
+        do not need to be escaped inside `"..."`.
+
+-   Rules can have arbitrary predicates attached to them. This subsumes
+    the previous left-context and right-context facilities (although
+    these are still allowed as syntactic sugar).
+
+### Changes in the form of an Alex file
+
+-   Each file can now only define a single grammar. This change was made
+    to simplify code generation. Multiple grammars can be simulated
+    using startcodes, or split into separate modules.
+
+-   The API has been simplified, and at the same time made more
+    flexible.
+
+-   You no longer need to import the `Alex` module.
+
+### Usage changes
+
+The command-line syntax is quite different.
+
+### Implementation changes
+
+-   A more efficient table representation, coupled with standard
+    table-compression techniques, are used to keep the size of the
+    generated code down.
+
+-   When compiling a grammar with GHC, the `-g` switch causes an even
+    faster and smaller grammar to be generated.
+
+-   Startcodes are implemented in a different way: each state
+    corresponds to a different initial state in the DFA, so the scanner
+    doesn't have to check the startcode when it gets to an accept state.
+    This results in a larger, but quicker, scanner.

alex.cabal

@@ -45,15 +45,6 @@ data-files:
 extra-source-files:
         CHANGELOG.md
         README.md
-        TODO
-        doc/Makefile
-        doc/aclocal.m4
-        doc/alex.1.in
-        doc/alex.xml
-        doc/config.mk.in
-        doc/configure.ac
-        doc/docbook-xml.mk
-        doc/fptools.css
         examples/Makefile
         examples/Tokens.x
         examples/Tokens_gscan.x

doc/.gitignore

@@ -0,0 +1 @@
+_build/

doc/Makefile

@@ -1,6 +1,28 @@
-include config.mk
+# Minimal makefile for Sphinx documentation
+#
 
-XML_DOC = alex
-INSTALL_XML_DOC = alex
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = _build
 
-include docbook-xml.mk
+# Flag -n ("nitpick") warns about broken references
+# Flag -W turns warnings into errors
+# Flag --keep-going continues after errors
+SPHINXOPTS   := -n -W --keep-going -E
+
+.PHONY: help html Makefile
+
+# default goal, first
+html: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

doc/README.md

@@ -0,0 +1,62 @@
+
+.. _about:
+
+About Alex
+==========
+
+Alex can always be obtained from its `home
+page <http://www.haskell.org/alex>`__. The latest source code lives in
+the `git repository <https://github.com/haskell/alex>`__ on ``GitHub``.
+
+.. _bug-reports:
+
+Reporting bugs in Alex
+----------------------
+
+Please report bugs on the
+`Alex issue tracker <https://github.com/haskell/alex/issues>`__.
+There are no specific mailing lists for the discussion of Alex-related
+matters, but such topics should be fine on the `Haskell Cafe
+<http://www.haskell.org/mailman/listinfo/haskell-cafe>`__ mailing
+list.
+
+License
+-------
+
+Copyright (c) 1995-2011, Chris Dornan and Simon Marlow. All rights
+reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+-  Redistributions of source code must retain the above copyright
+   notice, this list of conditions and the following disclaimer.
+
+-  Redistributions in binary form must reproduce the above copyright
+   notice, this list of conditions and the following disclaimer in the
+   documentation and/or other materials provided with the distribution.
+
+-  Neither the name of the copyright holders, nor the names of the
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
+IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
+OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+About this documentation
+------------------------
+
+This documentation is based on a DocBook documentation originally written by
+Chris Dornan, Isaac Jones, and Simon Marlow.
+
+Converted to RST / Sphinx / readthedocs.org by Andreas Abel in February 2022.