22
33|Packaging status |
44
5- Mathics Scanner
6- ===============
5+ Mathics Character Tables and Scanner
6+ =====================================
77
8- This is the tokeniser or scanner portion for the Wolfram Language.
8+ This repository really contains two things:
99
10- As such, it also contains a full set of translation between Wolfram Language
11- named characters, their Unicode/ASCII equivalents and code-points.
10+ * extensive tables describing WL symbols and operators their properties
11+ * a tokenizer or scanner portion for the Wolfram Language.
12+
13+ With respect to the first item, there is a commented YAML that contains a
14+ full set of translation between:
15+
16+ * Wolfram Language named characters,
17+ * their Unicode/ASCII equivalents and Unicode and WL code-points,
18+ * Operator name (if symbol is an operator),
19+ * Operator precedence (if an operator)
20+ * Keyboard escape sequences for the symbol
1221
1322Uses
1423----
1524
16- This is used as the scanner inside `Mathics <https://mathics.org >`_ but it can
17- also be used for tokenizing and formatting Wolfram Language code. In fact we
18- intend to write one. This library is also quite usefull if you need to work
19- with Wolfram Language named character and convert them to various formats.
25+ The scanner and character tables are used inside `Mathics <https://mathics.org >`_. However information can
26+ also be used by other programs for tokenizing and formatting Wolfram Language code.
27+
28+ For example, tables are used in `mathics-pygments <https://pypi.org/project/Mathics-Scanner/ >`_, a Pygments-based
29+ lexer and highlighter for Mathematica/Wolfram Language source code.
30+
31+ This library may be useful if you need to work with Wolfram Language
32+ named character and convert them to various formats.
2033
2134Usage
2235-----
@@ -30,17 +43,24 @@ Usage
3043 ``\[FormalA] ``) and Wolfram's internal representation use the
3144 ``m̀athics_scanner.characters.named_characters `` dictionary.
3245
33- To regenerate scanner tables run:
46+ To regenerate JSON-format tables run:
3447
3548::
3649
3750 $ mathics-generate-json-table
3851
52+ Without options ``mathics-generate-json-table `` produces the maximum set of correspondences.
53+
54+ In most applications though you may need just a few of these. The
55+ ``--field `` option can be used to narrow the list of entries to output in JSON. Run
56+ ``mathics-generate-json-table --help `` for a full list of fields.
57+
58+
3959Implementation
4060--------------
4161
4262For notes on the implementation of the packages or details on the conversion
43- scheme please read `` implementation.rst `` .
63+ scheme please read `Implementation < https://mathics-scanner.readthedocs.io/en/latest/ implementation.html >`_ .
4464
4565Contributing
4666------------
0 commit comments