Skip to content

Releases: Mathics3/mathics-scanner

2.0.0

28 Aug 19:24

Choose a tag to compare

Supports Python 3.13. Python 3.8 and 3.9 support has been dropped.

Note: There are incompatible changes. Use with Mathics-core 9.0.0 or greater.

Support for saving token position information was started. This is in module mathics_scanner.location. Additional location information is saved when mathics_scanner.location.TRACK_LOCATIONS is set to True.

Scanning was revised to handle more kinds of escape sequences and to be sensitive to their placement inside and outside a string. This is in module mathics_scanner.escape_sequences.

Tokenization support for Box Input operator \* was added.

Small YML changes were made to reflect a better understanding of boxing operators. In particular, \* is not a (prefix) operator.

Internals

Mathics3 scanner exceptions of class TranslateError are incompatible with previous versions, and now store error parameters, "name", "tag", and "args".

Bugs Fixed

  • #125 Hex escape sequence in string literal doesn't work.
  • #136 Is YAML file valid?

1.4.1

26 Jan 17:39

Choose a tag to compare

Re-release to include operators.yml into the tarball/wheel.

1.4.0

26 Jan 16:49

Choose a tag to compare

A new operator table, operators.yml was added to contain operator information. This information is based on information from Robert Jacobson.

See https://github.com/WLTools/LanguageSpec/blob/master/docs/Specification/Syntax/Operator%20Table.csv

From operators.yml, operators.json is created, and this holds operator information that the Mathics3 Kernel uses.

Things like operator precedence, operator arity, associativity, and AMSLaTeX equivalent notation are some of the information we store.

All of the 100 or so Unicode operators without an initial built-in meaning, .e.g., [Cup], [Cap], ... have been added.

The tokenizer and parser in the Mathics3 Kernel use more YAML table information via JSON extraction. However, more will be done in the future.

A new utility program, mathics3-tokens can be used to show tokenization of an input stream, with the -C or --CodeTokenize option, the program shows the tokens more closely in the form that the WMA CodeTokens package uses. Over time, we expect that our tokenizer will be more compliant with CodeTokens.

named-characters.yml was gone over, mostly to fill out information, such as URL links to Unicode pages.

Operator precedence values have been gone over.

Merge pull request #66 from Mathics3/release-1.3.1

09 Aug 12:01

Choose a tag to compare

Python 3.8 is now the minimum Python supported. Python 3.12 supported.
Various dependencies elsewhere force 3.8 or newer.

  • Packaging was redone to be able to support Python 3.12.
  • Files now follow current Python black formatting and isort import ordering
  • Some Python code linting

Too late for Valentines Day

25 Feb 16:02

Choose a tag to compare

  • Add escape-code sequence for 32-bit Unicode. Issue #48.
  • Correct Infix and Tilde character symbols
  • Support double backslash (\\) as a single backslash character (\).
  • Correct Unicode for ScriptN and ScriptCaptialN
  • Correct a number of is-letter-like entries.
  • Accept \u21A6 as symbol for Function.
  • Change the precedence of |->(Function symbol) to 800 so it isn't interpreted as a |
    followed by ->
  • ASCII operator tables can now be generated
  • Add DifferentialD and Integrate even though we don't have a full set of prefix operators.
  • more precedence values added to operators
  • Python 3.11 operation verified

1.2.4

10 Jul 23:28

Choose a tag to compare

  • Start adding AMSLateX names.
  • Add ApplyTo
  • Add Factorial2. PR #30
  • Adjust Tilde, and Factorial.
  • Regularize unicode equivalents.
  • Add named-characters.yml to distribution packages; Issue #32.
  • Use SPDX identifier in license; PR #31.

This time, for sure!

27 Jun 09:59

Choose a tag to compare

Redo for packaging problems.

Many thanks to Victor the packager for AUR for pointing this out.

1.2.2

26 Jun 10:50

Choose a tag to compare

Redo for packaging problems

Many thanks to Victor the packager for AUR for pointing this out quickly.

1.2.1

26 Jun 06:41

Choose a tag to compare

  • Add tables for operator precedence.
  • Start to add AMSLaTeX symbols. (A future release will finish this)
  • Revise README.rst.
  • Some small corrections: Implies
  • Make ujson optional

L-4

15 May 00:04

Choose a tag to compare

L-4

Tag unicode operators which have no definition and add the ability to dump them. This is useful for mathics-pygments.