tokenize.jl: don‘t hardcode unicode ops range

pthariensflame · web-flow · commit 7ace8d7f94e9 · 2024-01-04T13:05:29.000-08:00
diff --git a/src/tokenize.jl b/src/tokenize.jl
@@ -171,7 +171,7 @@ function optakessuffix(k)
         k == K"!"   ||
         k == K".'"  ||
         k == K"->"  ||
-        K"¬" <= k <= K"∜"
+        K"BEGIN_UNICODE_OPS" <= k <= K"END_UNICODE_OPS"
     )
 end
 

Original file line number	Diff line number	Diff line change
`@@ -171,7 +171,7 @@ function optakessuffix(k)`
`171`	`171`	`k == K"!" \|\|`
`172`	`172`	`k == K".'" \|\|`
`173`	`173`	`k == K"->" \|\|`
`174`		`- K"¬" <= k <= K"∜"`
	`174`	`+ K"BEGIN_UNICODE_OPS" <= k <= K"END_UNICODE_OPS"`
`175`	`175`	`)`
`176`	`176`	`end`
`177`	`177`