Use KeYParser.g4 for parsing proof scripts

[wadoon]

This PR removes the hand-written parser for proof scripts and uses a few rules in KeYParser.g4 instead.

Since the start of proof scripts, the KeYParser has changed and became an ANTLR4 grammar. This allows us to easily write a grammar for our proof scripts. (Or just copy the few rules from Sarah and mine proof script parser). This eliminates the handwritten proof script parser with the following benefits:

1. Proof Scripts are a first-class citizen in KeY files. You do not need to put your proof script into a string literal:

   \proofscript {
       andLeft;
       andRight;  
       rule a ==> b;
   }
   

   Please note an ambiguity in the grammar: rule b ==> c could either be interpreted as a command with one or two arguments:

   1. a semi-sequent b ==> and a term c, or
   2. a sequent b ==> c.
   3. or a term b and a semi-sequent ==> c.

   Use quotes or (better) parentheses to clarify these situations.

   The grammar should follow the KeY grammar as best as possible. Backwards-compatibility is tried to preserve.

2. Earlier and better syntax errors, during reading of the KeY file instead of proof script execution.

3. Better positioning information, as these are generated by using the ParseContexts.

4. Proper data structure: no triple of strings is pushed through KeY.

5. You do not need to put your arguments in quotes. Literals, terms and sequents are parsed.

Breaking Change

• The following is now prohibited:

  my-command x;
  

  Commands should be valid KeY identifiers, hence they should be free of dashes (aka subtraction operation).

• Single quotes are reserved for character literals. Please only use ".

Backward compatibility

Tried to achieve backward-compatibility.

1. The proof script can still be given as a string: \proofscript "..."; inside a KeY file.

2. Term arguments can still be given as a string. The conversion mechanism are still in place to convert the data types in many directions (string to int, string to term, but also term to string, etc.)

   Note there is a difference between \f(x)`and"f(x)". The first one is parsed directly as a term and can only be passed to term or string arguments (or any other data type for which a converter is registered). The second one "f(x)"` is parsed as a string. By meta-information on commands, the string is lazily converted to a term if necessary. This conversion is delayed until the command execution.

   The term f(x) might be parsed early with the input file, but the expression is evaluated (translated from parse tree to Term) on command-execution time, hence, the use of goal-local variables should be possible.

Side changes to Proof Scripts

"Higher-order proof scripts"

A proof command can receive a block of commands as an argument. Consider this:

try { auto; }

where try is a regular proof command. And { auto; } is a block. This constructs replace tryclose; but gives you more power on what will be tried.

This feature arose by the current complicated version of the AllCommand (e.g., onAll hide \f(x)`) executes the sub-command specified by the arguments on all goals). In SaG scripts we used foreach { ... }` to achieve this behavior.

The extension adapts the command syntax to take a list of sub-commands in curly brackets.

proofScriptCommand: 
    name  args  (  '{'   <sub-command>  '}' ';'? | ';' )
    ; 

The new syntax for onAll is also onAll { hide \f(x)`; }. Commands receive the code block using the key #blockinside the map as a parse tree, that can be sent back to theScriptEngineusing#execute(state, statements)`. The new syntax allows building new commands like

• try {<sub>} -- executes <sub> ignoring errors
• repeat { <sub> } -- executes <sub> as long as there are changes on the sequent.
• matching <term> { <sub> } -- execute on all goals where <term> is prescense.

Removal of the XML for command documentation.

We used an XML file to store documentation of proof script commands. This file was deleted, and we use text block literals instead.

Remove of @-prefix

• TODO

[mattulbrich]

I agree that the current parser is more a proof-of-concept solution than future-proof.

However: With many ideas around proof scripts, which should possibly discuss how this should go now: The script debugger has its ways of communicating to the server, scripts can be in .key files, they were at a point interactively editable, and the plan is to have them in JML code. Do they all go to the same background linear script language or is the base language itself more than a sequence of commands?

@WolframPfeifer @jwiesler

[mattulbrich]

First round of review. Some feedback. Could not review the whole thing.

[mattulbrich]

Reply to me-from-the-past:

However: With many ideas around proof scripts, which should possibly discuss how this should go now: The script debugger has its ways of communicating to the server, scripts can be in .key files, they were at a point interactively editable, and the plan is to have them in JML code. Do they all go to the same background linear script language or is the base language itself more than a sequence of commands?

After implementing a/the script language for JML: This was compiled down to a linear script. So it seems that a list of individual proof commands looks like the right level of abstraction.

[wadoon]

After implementing a/the script language for JML: This was compiled down to a linear script. So it seems that a list of individual proof commands looks like the right level of abstraction.

This PR gives the current state a proper syntax in alignment with the KeY grammar. So the list of proof commands are still preserved. Giving the commands the possibility to receive a block of sub-commands arises from your AllCommand and allows future extensions w/o needing syntax changes.

We need a quoting symbol for terms for the ambiguities, " is used for string values, and ' is also occupied (do we use it for char literals?). So backticks \`` are a possible option, but we can also go for parentheses, e.g., let @A=(f(x))` etc. I do not care.

[mattulbrich]

There seem to be accidental artifacts of other PRs in this one.

The old principle was: Every argument is essentially a string. If it is a single identifier the quotes can be omitted.
Why not keep this convention? I'd vote for this. (It's heavily inspired by "isar.")

If dropping this in favour of type safety, then probably this should be handled more strictly in order to not add confusion instead of removing it.

[mattulbrich]

This PR seems to change the infrastructure quite considerably. It touches > 260 files, some of them rather far from the topic at hand.

I do like the idea to reuse the KeY parser instead of the handmade parser.
Still, there is nothing wrong (I think) with the character bases handmade parser, but the ANTLR machine of course allows for more flexibility and cleaner code.

I was not able to understand the higher order feature, but that looked nice indeed.

I'd argue against the (voluntary) strong typing. The design of the language is untyped. Everything is a string. I would like to stick to this on the KeY level. It is a different story on the JML level.

Can we break this down into manageable units (by cherrypicking aspects e.g.)?

The script proof engine can an overhaul if/when the JML scripts are on master-track.

[mattulbrich]

Consider force pushing this to weigl/psg4-cleaned where most non-related aspects have been removed by rebasing the branch.

[mattulbrich]

Afterwards consider https://github.com/KeYProject/key/tree/psg4-cleaned-continued where a few ideas from other branches regarding the non-JML bit of proof scripts have come together.

[mattulbrich]

The ambiguity of the parser still is really a problem. command a = b ==> can be at least three different calls. Should ANTLR not complain? Parser generators usually have guarantees regarding ambiguities. ...

[wadoon]

I'll rebase this PR. Splitting into multiple PRs should only bring complexity.

[mattulbrich]

Just read the commit message "Introduce ScriptCommandAst to keep the ProofEngine separated from ANT…"

I'd propose to limit this PR to its original purpose and merge it into master.
Going on from that point with ScriptCommandAPI etc is then up to further development in a next PR.
This would also allow to add new feature in other PRs.