Add complete constant folding in kmath-ast by introducing TypedMst, some minor changes

Author: CommanderTvis

README.md

@@ -247,6 +247,12 @@ One can still use generic algebras though.
 > **Maturity**: PROTOTYPE
 <hr/>
 
+* ### [kmath-tensorflow](kmath-tensorflow)
+> 
+>
+> **Maturity**: PROTOTYPE
+<hr/>
+
 * ### [kmath-tensors](kmath-tensors)
 > 
 >

kmath-ast/README.md

@@ -1,6 +1,6 @@
 # Module kmath-ast
 
-Performance and visualization extensions to MST API.
+Extensions to MST API: transformations, dynamic compilation and visualization.
 
  - [expression-language](src/commonMain/kotlin/space/kscience/kmath/ast/parser.kt) : Expression language and its parser
  - [mst-jvm-codegen](src/jvmMain/kotlin/space/kscience/kmath/asm/asm.kt) : Dynamic MST to JVM bytecode compiler
@@ -35,55 +35,68 @@ dependencies {
 }
 ```
 
+## Parsing expressions
+
+In this module there is a parser from human-readable strings like `"x^3-x+3"` (in the more specific [grammar](reference/ArithmeticsEvaluator.g4)) to MST instances.
+
+Supported literals:
+1. Constants and variables (consist of latin letters, digits and underscores, can't start with digit): `x`, `_Abc2`.
+2. Numbers: `123`, `1.02`, `1e10`, `1e-10`, `1.0e+3`—all parsed either as `kotlin.Long` or `kotlin.Double`.
+
+Supported binary operators (from the highest precedence to the lowest one):
+1. `^`
+2. `*`, `/`
+3. `+`, `-`
+
+Supported unary operator:
+1. `-`, e. g. `-x`
+
+Arbitrary unary and binary functions are also supported: names consist of latin letters, digits and underscores, can't start with digit. Examples:
+1. `sin(x)`
+2. `add(x, y)`
+
 ## Dynamic expression code generation
 
 ### On JVM
 
 `kmath-ast` JVM module supports runtime code generation to eliminate overhead of tree traversal. Code generator builds a
 special implementation of `Expression<T>` with implemented `invoke` function.
 
-For example, the following builder:
+For example, the following code:
 
 ```kotlin
-import space.kscience.kmath.expressions.Symbol.Companion.x
-import space.kscience.kmath.expressions.*
-import space.kscience.kmath.operations.*
-import space.kscience.kmath.asm.*
+import space.kscience.kmath.asm.compileToExpression
+import space.kscience.kmath.complex.ComplexField
 
-MstField { x + 2 }.compileToExpression(DoubleField)
-``` 
+"x+2".parseMath().compileToExpression(ComplexField)
+```
 
-... leads to generation of bytecode, which can be decompiled to the following Java class:
+&mldr; leads to generation of bytecode, which can be decompiled to the following Java class:
 
 ```java
-package space.kscience.kmath.asm.generated;
-
 import java.util.Map;
-
 import kotlin.jvm.functions.Function2;
 import space.kscience.kmath.asm.internal.MapIntrinsics;
+import space.kscience.kmath.complex.Complex;
 import space.kscience.kmath.expressions.Expression;
 import space.kscience.kmath.expressions.Symbol;
 
-public final class AsmCompiledExpression_45045_0 implements Expression<Double> {
+public final class CompiledExpression_45045_0 implements Expression<Complex> {
     private final Object[] constants;
 
-    public final Double invoke(Map<Symbol, ? extends Double> arguments) {
-        return (Double) ((Function2) this.constants[0]).invoke((Double) MapIntrinsics.getOrFail(arguments, "x"), 2);
-    }
-
-    public AsmCompiledExpression_45045_0(Object[] constants) {
-        this.constants = constants;
+    public Complex invoke(Map<Symbol, ? extends Complex> arguments) {
+        Complex var2 = (Complex)MapIntrinsics.getOrFail(arguments, "x");
+        return (Complex)((Function2)this.constants[0]).invoke(var2, (Complex)this.constants[1]);
     }
 }
-
 ```
 
-#### Known issues
+Setting JVM system property `space.kscience.kmath.ast.dump.generated.classes` to `1` makes the translator dump class files to program's working directory, so they can be reviewed manually.
+
+#### Limitations
 
-- The same classes may be generated and loaded twice, so it is recommended to cache compiled expressions to avoid class
-  loading overhead.
-- This API is not supported by non-dynamic JVM implementations (like TeaVM and GraalVM) because of using class loaders.
+- The same classes may be generated and loaded twice, so it is recommended to cache compiled expressions to avoid class loading overhead.
+- This API is not supported by non-dynamic JVM implementations like TeaVM or GraalVM Native Image because they may not support class loaders.
 
 ### On JS
 
@@ -129,7 +142,7 @@ An example of emitted Wasm IR in the form of WAT:
 )
 ```
 
-#### Known issues
+#### Limitations
 
 - ESTree expression compilation uses `eval` which can be unavailable in several environments.
 - WebAssembly isn't supported by old versions of browsers (see https://webassembly.org/roadmap/).

kmath-ast/docs/README-TEMPLATE.md

@@ -1,60 +1,98 @@
 # Module kmath-ast
 
-Performance and visualization extensions to MST API.
+Extensions to MST API: transformations, dynamic compilation and visualization.
 
 ${features}
 
 ${artifact}
 
+## Parsing expressions
+
+In this module there is a parser from human-readable strings like `"x^3-x+3"` (in the more specific [grammar](reference/ArithmeticsEvaluator.g4)) to MST instances.
+
+Supported literals:
+1. Constants and variables (consist of latin letters, digits and underscores, can't start with digit): `x`, `_Abc2`.
+2. Numbers: `123`, `1.02`, `1e10`, `1e-10`, `1.0e+3`—all parsed either as `kotlin.Long` or `kotlin.Double`.
+
+Supported binary operators (from the highest precedence to the lowest one):
+1. `^`
+2. `*`, `/`
+3. `+`, `-`
+
+Supported unary operator:
+1. `-`, e. g. `-x`
+
+Arbitrary unary and binary functions are also supported: names consist of latin letters, digits and underscores, can't start with digit. Examples:
+1. `sin(x)`
+2. `add(x, y)`
+
 ## Dynamic expression code generation
 
 ### On JVM
 
 `kmath-ast` JVM module supports runtime code generation to eliminate overhead of tree traversal. Code generator builds a
 special implementation of `Expression<T>` with implemented `invoke` function.
 
-For example, the following builder:
+For example, the following code:
 
 ```kotlin
-import space.kscience.kmath.expressions.Symbol.Companion.x
-import space.kscience.kmath.expressions.*
-import space.kscience.kmath.operations.*
-import space.kscience.kmath.asm.*
+import space.kscience.kmath.asm.compileToExpression
+import space.kscience.kmath.operations.DoubleField
 
-MstField { x + 2 }.compileToExpression(DoubleField)
-``` 
+"x^3-x+3".parseMath().compileToExpression(DoubleField)
+```
 
-... leads to generation of bytecode, which can be decompiled to the following Java class:
+&mldr; leads to generation of bytecode, which can be decompiled to the following Java class:
 
 ```java
-package space.kscience.kmath.asm.generated;
+import java.util.*;
+import kotlin.jvm.functions.*;
+import space.kscience.kmath.asm.internal.*;
+import space.kscience.kmath.complex.*;
+import space.kscience.kmath.expressions.*;
+
+public final class CompiledExpression_45045_0 implements Expression<Complex> {
+    private final Object[] constants;
 
-import java.util.Map;
+    public Complex invoke(Map<Symbol, ? extends Complex> arguments) {
+        Complex var2 = (Complex)MapIntrinsics.getOrFail(arguments, "x");
+        return (Complex)((Function2)this.constants[0]).invoke(var2, (Complex)this.constants[1]);
+    }
+}
+```
 
-import kotlin.jvm.functions.Function2;
-import space.kscience.kmath.asm.internal.MapIntrinsics;
-import space.kscience.kmath.expressions.Expression;
-import space.kscience.kmath.expressions.Symbol;
+For `LongRing`, `IntRing`, and `DoubleField` specialization is supported for better performance:
 
-public final class AsmCompiledExpression_45045_0 implements Expression<Double> {
-    private final Object[] constants;
+```java
+import java.util.*;
+import space.kscience.kmath.asm.internal.*;
+import space.kscience.kmath.expressions.*;
 
-    public final Double invoke(Map<Symbol, ? extends Double> arguments) {
-        return (Double) ((Function2) this.constants[0]).invoke((Double) MapIntrinsics.getOrFail(arguments, "x"), 2);
+public final class CompiledExpression_-386104628_0 implements DoubleExpression {
+    private final SymbolIndexer indexer;
+
+    public SymbolIndexer getIndexer() {
+        return this.indexer;
     }
 
-    public AsmCompiledExpression_45045_0(Object[] constants) {
-        this.constants = constants;
+    public double invoke(double[] arguments) {
+        double var2 = arguments[0];
+        return Math.pow(var2, 3.0D) - var2 + 3.0D;
     }
-}
 
+    public final Double invoke(Map<Symbol, ? extends Double> arguments) {
+        double var2 = ((Double)MapIntrinsics.getOrFail(arguments, "x")).doubleValue();
+        return Math.pow(var2, 3.0D) - var2 + 3.0D;
+    }
+}
 ```
 
-#### Known issues
+Setting JVM system property `space.kscience.kmath.ast.dump.generated.classes` to `1` makes the translator dump class files to program's working directory, so they can be reviewed manually.
+
+#### Limitations
 
-- The same classes may be generated and loaded twice, so it is recommended to cache compiled expressions to avoid class
-  loading overhead.
-- This API is not supported by non-dynamic JVM implementations (like TeaVM and GraalVM) because of using class loaders.
+- The same classes may be generated and loaded twice, so it is recommended to cache compiled expressions to avoid class loading overhead.
+- This API is not supported by non-dynamic JVM implementations like TeaVM or GraalVM Native Image because they may not support class loaders.
 
 ### On JS
 
@@ -100,7 +138,7 @@ An example of emitted Wasm IR in the form of WAT:
 )
 ```
 
-#### Known issues
+#### Limitations
 
 - ESTree expression compilation uses `eval` which can be unavailable in several environments.
 - WebAssembly isn't supported by old versions of browsers (see https://webassembly.org/roadmap/).