[GR-42711] Make foreign numbers inherit from ForeignNumberType, foreign strings from str and fix TCK

PullRequest: graalpython/3513

Author: eregon

CHANGELOG.md

@@ -9,9 +9,9 @@ language runtime. The main focus is on user-observable behavior of the engine.
 * When calling a method on a foreign object in Python code, Python methods are now prioritized over foreign members.
 * Added `polyglot.register_interop_type` and `@polyglot.interop_type` to define custom Python methods for a given foreign class/type. See [the documentation](https://github.com/oracle/graalpython/blob/master/docs/user/Interoperability.md#the-interoperability-extension-api) for more information.
 * Foreign objects are now given a Python class corresponding to their interop traits.
-** Foreign lists now inherit from Python `list`, foreign dictionaries from `dict`, foreign iterators from `iterator`, foreign exceptions from `BaseException` and foreign none/null from `NoneType`.
-** This means all Python methods of these types are available on the corresponding foreign objects, which behave as close as possible as if they were Python objects.
-** See [the documentation](https://github.com/oracle/graalpython/blob/master/docs/user/Interoperability.md#interacting-with-foreign-objects-from-python-scripts) for more information.
+  * Foreign lists now inherit from Python `list`, foreign dictionaries from `dict`, foreign strings from `str`, foreign iterators from `iterator`, foreign exceptions from `BaseException`, foreign numbers from `ForeignNumberType` and foreign none/null from `NoneType`.
+  * This means all Python methods of these types are available on the corresponding foreign objects, which behave as close as possible as if they were Python objects.
+  * See [the documentation](https://github.com/oracle/graalpython/blob/master/docs/user/Interoperability.md#interacting-with-foreign-objects-from-python-scripts) for more information.
 * Remove support for running with Sulong managed both in embeddings as well as through the `graalpy-managed` launcher.
 
 ## Version 24.1.0
@@ -40,9 +40,9 @@ language runtime. The main focus is on user-observable behavior of the engine.
 * `multiprocessing` module now uses the `spawn` method (creates new processes) by default. The formerly default method that uses threads and multiple Truffle contexts can be selected using `multiprocessing.set_start_method('graalpy')`.
 * `polyglot` module: add API to redefine Truffle interop messages for external / user defined types. For more details see [The Truffle Interoperability Extension API](docs/user/Interoperability.md).
 * Adding integration with jBang (https://www.jbang.dev/)
-** running example via `jbang hello@oracle/graalpython` or `jbang hello@oracle/graalpython "print(1*4)"`
-** creating new script via: `jbang init --template=graalpy@oracle/graalpython myscript.java`
-** creating new script with local maven repo for testing: `jbang init --template=graalpy_local_repo@oracle/graalpython -Dpath_to_local_repo=/absolute/path/to/local/maven/repository myscript.java'
+  * running example via `jbang hello@oracle/graalpython` or `jbang hello@oracle/graalpython "print(1*4)"`
+  * creating new script via: `jbang init --template=graalpy@oracle/graalpython myscript.java`
+  * creating new script with local maven repo for testing: `jbang init --template=graalpy_local_repo@oracle/graalpython -Dpath_to_local_repo=/absolute/path/to/local/maven/repository myscript.java'
 
 ## Version 23.1.0
 * GraalPy distributions (previously known as GraalPy Enterprise) are now available under the [GFTC license](https://www.oracle.com/downloads/licenses/graal-free-license.html). The community builds published on Github have been renamed to `graalpy-community-<version>-<os>-<arch>.tar.gz`.

docs/user/Interoperability.md

@@ -92,9 +92,15 @@ h == {1: 2, 3: 6} # True
 Specifically:
 * Foreign lists inherit from Python `list`
 * Foreign dictionaries inherit from `dict`
+* Foreign strings inherit from `str`
 * Foreign iterators inherit from `iterator`
 * Foreign exceptions inherit from `BaseException`
+* Foreign numbers inherit from `ForeignNumberType` (since `InteropLibrary` has no way to differentiate integers and floats, but see below)
 * Foreign none/null inherit from `NoneType`
+* Other foreign objects inherit from `foreign`
+
+Note that Java primitives `byte`, `short`, `int`, `long` and `BigInteger` values are considered Python `int` objects,
+and Java primitives `float`, `double` values are considered Python `float` objects.
 
 ## Interacting with other dynamic languages from Python scripts
 

graalpython/com.oracle.graal.python.tck/src/com/oracle/graal/python/tck/PythonProvider.java

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2018, 2023, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2018, 2024, Oracle and/or its affiliates. All rights reserved.
  * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
  *
  * The Universal Permissive License (UPL), Version 1.0
@@ -78,6 +78,7 @@
 import org.graalvm.polyglot.tck.ResultVerifier;
 import org.graalvm.polyglot.tck.Snippet;
 import org.graalvm.polyglot.tck.TypeDescriptor;
+import org.junit.Assert;
 
 public class PythonProvider implements LanguageProvider {
 
@@ -226,28 +227,37 @@ public Collection<? extends Snippet> createValueConstructors(Context context) {
     public Collection<? extends Snippet> createExpressions(Context context) {
         List<Snippet> snippets = new ArrayList<>();
 
+        TypeDescriptor numeric = union(BOOLEAN, NUMBER);
+        TypeDescriptor numericArray = array(numeric);
+
         // @formatter:off
-        addExpressionSnippet(context, snippets, "+", "lambda x, y: x + y", NUMBER, new NonPrimitiveNumberParameterThrows(AddVerifier.INSTANCE), union(BOOLEAN, NUMBER), union(BOOLEAN, NUMBER));
-        addExpressionSnippet(context, snippets, "+", "lambda x, y: x + y", union(STRING, array(ANY)), AddVerifier.INSTANCE, union(BOOLEAN, NUMBER), union(STRING, array(ANY)));
-        addExpressionSnippet(context, snippets, "+", "lambda x, y: x + y", union(STRING, array(ANY)), AddVerifier.INSTANCE, union(STRING, array(ANY)), union(BOOLEAN, NUMBER));
+        addExpressionSnippet(context, snippets, "+", "lambda x, y: x + y", NUMBER, new NonPrimitiveNumberParameterThrows(AddVerifier.INSTANCE), numeric, numeric);
+        addExpressionSnippet(context, snippets, "+", "lambda x, y: x + y", union(STRING, array(ANY)), AddVerifier.INSTANCE, numeric, union(STRING, array(ANY)));
+        addExpressionSnippet(context, snippets, "+", "lambda x, y: x + y", union(STRING, array(ANY)), AddVerifier.INSTANCE, union(STRING, array(ANY)), numeric);
         addExpressionSnippet(context, snippets, "+", "lambda x, y: x + y", union(STRING, array(ANY)), AddVerifier.INSTANCE, union(STRING, array(ANY)), union(STRING, array(ANY)));
-        addExpressionSnippet(context, snippets, "*", "lambda x, y: x * y", NUMBER, new NonPrimitiveNumberParameterThrows(MulVerifier.INSTANCE), union(BOOLEAN, NUMBER), union(BOOLEAN, NUMBER));
-        addExpressionSnippet(context, snippets, "*", "lambda x, y: x * y", union(STRING, array(ANY)), MulVerifier.INSTANCE, union(BOOLEAN, NUMBER), union(STRING, array(ANY)));
-        addExpressionSnippet(context, snippets, "*", "lambda x, y: x * y", union(STRING, array(ANY)), MulVerifier.INSTANCE, union(STRING, array(ANY)), union(BOOLEAN, NUMBER));
+
+        addExpressionSnippet(context, snippets, "*", "lambda x, y: x * y", NUMBER, new NonPrimitiveNumberParameterThrows(MulVerifier.INSTANCE), numeric, numeric);
+        addExpressionSnippet(context, snippets, "*", "lambda x, y: x * y", union(STRING, array(ANY)), MulVerifier.INSTANCE, numeric, union(STRING, array(ANY)));
+        addExpressionSnippet(context, snippets, "*", "lambda x, y: x * y", union(STRING, array(ANY)), MulVerifier.INSTANCE, union(STRING, array(ANY)), numeric);
         addExpressionSnippet(context, snippets, "*", "lambda x, y: x * y", union(STRING, array(ANY)), MulVerifier.INSTANCE, union(STRING, array(ANY)), union(STRING, array(ANY)));
 
-        addExpressionSnippet(context, snippets, "-", "lambda x, y: x - y", NUMBER, NonPrimitiveNumberParameterThrows.INSTANCE, union(BOOLEAN, NUMBER), union(BOOLEAN, NUMBER));
+        addExpressionSnippet(context, snippets, "-", "lambda x, y: x - y", NUMBER, NonPrimitiveNumberParameterThrows.INSTANCE, numeric, numeric);
 
-        addExpressionSnippet(context, snippets, "/", "lambda x, y: x / y", NUMBER, new NonPrimitiveNumberParameterThrows(PDivByZeroVerifier.INSTANCE), union(BOOLEAN, NUMBER), union(BOOLEAN, NUMBER));
+        addExpressionSnippet(context, snippets, "/", "lambda x, y: x / y", NUMBER, new NonPrimitiveNumberParameterThrows(PDivByZeroVerifier.INSTANCE), numeric, numeric);
 
         addExpressionSnippet(context, snippets, "list-from-foreign", "lambda x: list(x)", array(ANY), union(STRING, iterable(ANY), iterator(ANY), array(ANY), hash(ANY, ANY)));
 
         addExpressionSnippet(context, snippets, "==", "lambda x, y: x == y", BOOLEAN, ANY, ANY);
         addExpressionSnippet(context, snippets, "!=", "lambda x, y: x != y", BOOLEAN, ANY, ANY);
-        addExpressionSnippet(context, snippets, ">", "lambda x, y: x > y", BOOLEAN, NonPrimitiveNumberParameterThrows.INSTANCE, union(BOOLEAN, NUMBER), union(BOOLEAN, NUMBER));
-        addExpressionSnippet(context, snippets, ">=", "lambda x, y: x >= y", BOOLEAN, NonPrimitiveNumberParameterThrows.INSTANCE, union(BOOLEAN, NUMBER), union(BOOLEAN, NUMBER));
-        addExpressionSnippet(context, snippets, "<", "lambda x, y: x < y", BOOLEAN, NonPrimitiveNumberParameterThrows.INSTANCE, union(BOOLEAN, NUMBER), union(BOOLEAN, NUMBER));
-        addExpressionSnippet(context, snippets, "<=", "lambda x, y: x <= y", BOOLEAN, NonPrimitiveNumberParameterThrows.INSTANCE, union(BOOLEAN, NUMBER), union(BOOLEAN, NUMBER));
+
+        addExpressionSnippet(context, snippets, ">", "lambda x, y: x > y", BOOLEAN, NonPrimitiveNumberParameterThrows.INSTANCE, numeric, numeric);
+        addExpressionSnippet(context, snippets, ">", "lambda x, y: x > y", BOOLEAN, ExpectTypeErrorForDifferentPythonSequenceTypes.INSTANCE, numericArray, numericArray);
+        addExpressionSnippet(context, snippets, ">=", "lambda x, y: x >= y", BOOLEAN, NonPrimitiveNumberParameterThrows.INSTANCE, numeric, numeric);
+        addExpressionSnippet(context, snippets, ">=", "lambda x, y: x >= y", BOOLEAN, ExpectTypeErrorForDifferentPythonSequenceTypes.INSTANCE, numericArray, numericArray);
+        addExpressionSnippet(context, snippets, "<", "lambda x, y: x < y", BOOLEAN, NonPrimitiveNumberParameterThrows.INSTANCE, numeric, numeric);
+        addExpressionSnippet(context, snippets, "<", "lambda x, y: x < y", BOOLEAN, ExpectTypeErrorForDifferentPythonSequenceTypes.INSTANCE, numericArray, numericArray);
+        addExpressionSnippet(context, snippets, "<=", "lambda x, y: x <= y", BOOLEAN, NonPrimitiveNumberParameterThrows.INSTANCE, numeric, numeric);
+        addExpressionSnippet(context, snippets, "<=", "lambda x, y: x <= y", BOOLEAN, ExpectTypeErrorForDifferentPythonSequenceTypes.INSTANCE, numericArray, numericArray);
 
         addExpressionSnippet(context, snippets, "isinstance", "lambda x, y: isinstance(x, y)", BOOLEAN, ANY, META_OBJECT);
         addExpressionSnippet(context, snippets, "issubclass", "lambda x, y: issubclass(x, y)", BOOLEAN, META_OBJECT, META_OBJECT);
@@ -360,6 +370,35 @@ private static Source createSource(String resourceName) throws IOException {
     private abstract static class PResultVerifier implements ResultVerifier {
     }
 
+    private static class ExpectTypeErrorForDifferentPythonSequenceTypes extends PResultVerifier {
+
+        public void accept(SnippetRun snippetRun) throws PolyglotException {
+            List<? extends Value> parameters = snippetRun.getParameters();
+            assert parameters.size() == 2;
+
+            Value a = parameters.get(0);
+            Value b = parameters.get(1);
+
+            // If both parameter values are Python sequences, then ignore if they have different
+            // types. E.g. ignore '(1,2) < [3,4]'.
+            if (a.hasArrayElements() && b.hasArrayElements() && isPythonObject(a) && isPythonObject(b) && !a.getMetaObject().equals(b.getMetaObject())) {
+                // Expect TypeError
+                var exception = snippetRun.getException();
+                assert exception != null;
+                Assert.assertEquals("TypeError", exception.getGuestObject().getMetaObject().getMetaQualifiedName());
+            } else {
+                ResultVerifier.getDefaultResultVerifier().accept(snippetRun);
+            }
+        }
+
+        private static boolean isPythonObject(Value value) {
+            var metaObject = value.getMetaObject();
+            return metaObject.hasMember("__mro__");
+        }
+
+        private static final ExpectTypeErrorForDifferentPythonSequenceTypes INSTANCE = new ExpectTypeErrorForDifferentPythonSequenceTypes();
+    }
+
     /**
      * Only accepts exact matches of types.
      */
@@ -378,16 +417,16 @@ public void accept(SnippetRun snippetRun) throws PolyglotException {
                 if (par0.getMetaObject() == par1.getMetaObject()) {
                     assert snippetRun.getException() == null;
                     TypeDescriptor resultType = TypeDescriptor.forValue(snippetRun.getResult());
-                    assert array(ANY).isAssignable(resultType);
+                    assert array(ANY).isAssignable(resultType) : resultType;
                 }
             } else if (par0.isString() && par1.isString()) {
                 assert snippetRun.getException() == null;
                 TypeDescriptor resultType = TypeDescriptor.forValue(snippetRun.getResult());
-                assert STRING.isAssignable(resultType);
+                assert STRING.isAssignable(resultType) : resultType;
             } else if ((par0.isNumber() || par0.isBoolean()) && (par1.isNumber() || par1.isBoolean())) {
                 assert snippetRun.getException() == null;
                 TypeDescriptor resultType = TypeDescriptor.forValue(snippetRun.getResult());
-                assert NUMBER.isAssignable(resultType);
+                assert NUMBER.isAssignable(resultType) : resultType;
             } else {
                 assert snippetRun.getException() != null;
                 TypeDescriptor argType = union(STRING, BOOLEAN, NUMBER, array(ANY));
@@ -426,17 +465,17 @@ public void accept(SnippetRun snippetRun) throws PolyglotException {
                 // string * number => string
                 assert snippetRun.getException() == null;
                 TypeDescriptor resultType = TypeDescriptor.forValue(snippetRun.getResult());
-                assert STRING.isAssignable(resultType);
+                assert STRING.isAssignable(resultType) : resultType;
             } else if ((par0.isNumber() || par0.isBoolean()) && (par1.isNumber() || par1.isBoolean())) {
-                // number * number has greater precendence than array * number
+                // number * number has greater precedence than array * number
                 assert snippetRun.getException() == null;
                 TypeDescriptor resultType = TypeDescriptor.forValue(snippetRun.getResult());
-                assert NUMBER.isAssignable(resultType) : resultType.toString();
+                assert NUMBER.isAssignable(resultType) : resultType;
             } else if (isArrayMul(par0, par1) || isArrayMul(par1, par0)) {
                 // array * number => array
                 assert snippetRun.getException() == null;
                 TypeDescriptor resultType = TypeDescriptor.forValue(snippetRun.getResult());
-                assert array(ANY).isAssignable(resultType);
+                assert array(ANY).isAssignable(resultType) : resultType;
             } else {
                 assert snippetRun.getException() != null;
                 TypeDescriptor argType = union(STRING, BOOLEAN, NUMBER, array(ANY));
@@ -461,34 +500,34 @@ public void accept(SnippetRun snippetRun) throws PolyglotException {
             List<? extends Value> parameters = snippetRun.getParameters();
             assert parameters.size() == 2;
 
-            Value par0 = parameters.get(0);
-            Value par1 = parameters.get(1);
+            Value self = parameters.get(0);
+            Value index = parameters.get(1);
 
             long len = -1;
 
-            if (par0.hasArrayElements()) {
-                len = par0.getArraySize();
-            } else if (par0.isString()) {
-                len = par0.asString().length();
+            if (self.isString()) {
+                len = self.asString().length();
+            } else if (self.hasArrayElements()) {
+                len = self.getArraySize();
             }
             if (len >= 0) {
                 int idx;
-                if (par1.isBoolean()) {
-                    idx = par1.asBoolean() ? 1 : 0;
-                } else if (par1.isNumber() && par1.fitsInInt()) {
-                    idx = par1.asInt();
+                if (index.isBoolean()) {
+                    idx = index.asBoolean() ? 1 : 0;
+                } else if (index.isNumber() && index.fitsInInt()) {
+                    idx = index.asInt();
                 } else {
-                    assert snippetRun.getException() != null;
+                    assert snippetRun.getException() != null : snippetRun.getResult();
                     return;
                 }
-                if ((idx >= 0 && len > idx) || (idx < 0 && idx + len >= 0 && len > idx + len)) {
-                    assert snippetRun.getException() == null : snippetRun.getException().toString();
+                if ((idx >= 0 && idx < len) || (idx < 0 && idx + len >= 0 && idx + len < len)) {
+                    assert snippetRun.getException() == null : snippetRun.getException();
                 } else {
-                    assert snippetRun.getException() != null;
+                    assert snippetRun.getException() != null : snippetRun.getResult();
                 }
-            } else if (par0.hasHashEntries()) {
-                if (par1.getMetaObject() != null) {
-                    String metaName = par1.getMetaObject().getMetaQualifiedName();
+            } else if (self.hasHashEntries()) {
+                if (index.getMetaObject() != null) {
+                    String metaName = index.getMetaObject().getMetaQualifiedName();
                     for (String s : UNHASHABLE_TYPES) {
                         if (metaName.equals(s)) {
                             // those don't work, but that's expected
@@ -497,11 +536,11 @@ public void accept(SnippetRun snippetRun) throws PolyglotException {
                         }
                     }
                 }
-                Value v = par0.getHashValueOrDefault(par1, PythonProvider.class.getName());
+                Value v = self.getHashValueOrDefault(index, PythonProvider.class.getName());
                 if (v.isString() && v.asString().equals(PythonProvider.class.getName())) {
-                    assert snippetRun.getException() != null;
+                    assert snippetRun.getException() != null : snippetRun.getResult();
                 } else {
-                    assert snippetRun.getException() == null : snippetRun.getException().toString();
+                    assert snippetRun.getException() == null : snippetRun.getException();
                 }
             } else {
                 // argument type error, rethrow

graalpython/com.oracle.graal.python.test.integration/src/com/oracle/graal/python/test/integration/interop/JavaInteropTest.java

@@ -496,7 +496,7 @@ def foo(obj):
             foo.execute(ProxyObject.fromMap(m));
         }
 
-        public class JavaObject {
+        public static class JavaObject {
             public byte byteValue = 1;
             public short shortValue = 2;
             public int intValue = 3;
@@ -563,7 +563,7 @@ public void accessJavaObjectFields() throws IOException {
                             "5.0 <class 'float'>\n" +
                             "6.0 <class 'float'>\n" +
                             "True <class 'bool'>\n" +
-                            "c <class 'str'>\n", out.toString("UTF-8"));
+                            "c <class 'polyglot.ForeignString'>\n", out.toString("UTF-8"));
         }
 
         @Test
@@ -590,7 +590,7 @@ public void accessJavaObjectGetters() throws IOException {
                             "5.0 <class 'float'>\n" +
                             "6.0 <class 'float'>\n" +
                             "True <class 'bool'>\n" +
-                            "c <class 'str'>\n", out.toString("UTF-8"));
+                            "c <class 'polyglot.ForeignString'>\n", out.toString("UTF-8"));
         }
 
         @Test