[ #71 ] Fixed crash in DFAMin

DFA minimization used to crash on tokens of the form c* which produce
automata with only accepting states.  Considering the empty set of
non-accepting states as an equivalence class caused minimization to
crash with exception

  Prelude.head: empty list

Now, DFA minimization succeeds.

There is still a problem with nullable tokens like c*.  Alex produces an
infinite token sequence at the end of the input.

Author: andreasabel

alex.cabal

@@ -109,6 +109,7 @@ extra-source-files:
         tests/posn_typeclass_bytestring.x
         tests/strict_typeclass.x
         tests/unicode.x
+        tests/issue_71.x
 
 source-repository head
     type:     git

src/DFAMin.hs

@@ -112,7 +112,10 @@ groupEquivStates DFA { dfa_states = statemap }
     accept_groups :: [EquivalenceClass]
     accept_groups = map IS.fromList (Map.elems accept_map)
 
-    init_p = nonaccepting_states : accept_groups
+    init_p, init_q :: [EquivalenceClass]
+    init_p  -- Issue #71: each EquivalenceClass needs to be a non-empty set
+      | IS.null nonaccepting_states = accept_groups
+      | otherwise                   = nonaccepting_states : accept_groups
     init_q = accept_groups
 
     -- map token T to

tests/Makefile

@@ -40,6 +40,7 @@ TESTS = \
         basic_typeclass_bytestring.x \
         default_typeclass.x \
         gscan_typeclass.x \
+        issue_71.x \
         monad_typeclass.x \
         monad_typeclass_bytestring.x \
         monadUserState_typeclass.x \

tests/issue_71.x

@@ -0,0 +1,49 @@
+{
+-- Issue #71
+-- reported 2015-10-20 by Ian Duncan
+-- fixed 2020-01-22 by Andreas Abel
+--
+-- Problem was:
+-- DFA minimization crashed with "Prelude head: empty list" because
+-- empty set of non-accepting states was treated as empty equivalence
+-- class of states.
+
+module Main (main) where
+
+import System.Exit
+}
+
+%wrapper "posn"
+%token   "Token"
+
+$whitespace  = [\ \n\t]
+@whitespaces = $whitespace*
+
+:-
+
+@whitespaces { \ _ _ -> Whitespaces }
+"a"          { \ _ _ -> A           }
+
+{
+data Token = Whitespaces | A
+  deriving (Eq, Show)
+
+input           = "aa    \n\taa   \t  \n  a"
+expected_result = [A,A,Whitespaces,A,A,Whitespaces,A]
+
+main :: IO ()
+main
+    -- Since the whitespaces token is nullable, Alex
+    -- will recognize an infinite number of those
+    -- at the end of file.  This behavior is problematic,
+    -- but we don't fix it here.
+    -- We just test here whether the expected result
+    -- is a prefix of the produced result.
+  | take (length expected_result) result == expected_result = do
+      exitWith ExitSuccess
+  | otherwise = do
+      print $ take 20 result
+      exitFailure
+  where
+  result = alexScanTokens input
+}