[Backport] Support for new reachability-metadata schema v1.1.0

[zakkak]

What would you like to backport?

GraalVM 25.0 seems to support a slightly different reachability-metadata schema which we would need to backport support for.

diff --git a/reachability-metadata-schema-v1.0.0.json b/reachability-metadata-schema-v1.1.0.json
index a3a75dd..d0ed2de 100644
--- a/reachability-metadata-schema-v1.0.0.json
+++ b/reachability-metadata-schema-v1.1.0.json
@@ -1,6 +1,6 @@
 {
   "$schema": "https://json-schema.org/draft/2019-09/schema",
-  "$id": "https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/assets/reachability-metadata-schema-v1.0.0.json",
+  "$id": "https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/assets/reachability-metadata-schema-v1.1.0.json",
   "title": "JSON schema for the reachability metadata used by GraalVM Native Image",
   "type": "object",
   "default": {},
@@ -64,37 +64,69 @@
       "type": "array",
       "default": [],
       "items": {
-        "title": "Resource that should be available",
-        "type": "object",
-        "properties": {
-          "reason": {
-            "title": "Reason for the resource's inclusion in the metadata",
-            "$ref": "#/$defs/reason"
-          },
-          "condition": {
-            "title": "Condition under which the resource should be registered for runtime access",
-            "$ref": "#/$defs/condition"
-          },
-          "module": {
-            "title": "Module containing the resource",
-            "type": "string",
-            "default": ""
-          },
:...skipping...
diff --git a/reachability-metadata-schema-v1.0.0.json b/reachability-metadata-schema-v1.1.0.json
index a3a75dd..d0ed2de 100644
--- a/reachability-metadata-schema-v1.0.0.json
+++ b/reachability-metadata-schema-v1.1.0.json
@@ -1,6 +1,6 @@
 {
   "$schema": "https://json-schema.org/draft/2019-09/schema",
-  "$id": "https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/assets/reachability-metadata-schema-v1.0.0.json",
+  "$id": "https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/assets/reachability-metadata-schema-v1.1.0.json",
   "title": "JSON schema for the reachability metadata used by GraalVM Native Image",
   "type": "object",
   "default": {},
@@ -64,37 +64,69 @@
       "type": "array",
       "default": [],
       "items": {
-        "title": "Resource that should be available",
-        "type": "object",
-        "properties": {
-          "reason": {
-            "title": "Reason for the resource's inclusion in the metadata",
-            "$ref": "#/$defs/reason"
-          },
-          "condition": {
-            "title": "Condition under which the resource should be registered for runtime access",
-            "$ref": "#/$defs/condition"
-          },
-          "module": {
-            "title": "Module containing the resource",
-            "type": "string",
-            "default": ""
-          },
-          "glob": {
-            "title": "Resource name or pattern matching multiple resources (accepts * and ** wildcards)",
-            "type": "string"
+        "oneOf": [
+          {
+            "title": "Resource that should be available",
+            "type": "object",
+            "properties": {
+              "reason": {
+                "title": "Reason for the resource's inclusion in the metadata",
+                "$ref": "#/$defs/reason"
+              },
+              "condition": {
+                "title": "Condition under which the resource should be registered for runtime access",
+                "$ref": "#/$defs/condition"
+              },
+              "module": {
+                "title": "Module containing the resource",
+                "type": "string",
+                "default": ""
+              },
+              "glob": {
+                "title": "Resource name or pattern matching multiple resources (accepts * and ** wildcards)",
+                "type": "string"
+              }
+            },
+            "required": [
+              "glob"
+            ],
+            "additionalProperties": false
+          },
+          {
+            "title": "Resource bundle that should be available",
+            "type": "object",
+            "properties": {
+              "reason": {
+                "title": "Reason for the resource bundle's inclusion in the metadata",
+                "$ref": "#/$defs/reason"
+              },
+              "condition": {
+                "title": "Condition under which the resource bundle should be registered for runtime access",
+                "$ref": "#/$defs/condition"
+              },
+              "module": {
+                "title": "Module containing the resource bundle",
+                "type": "string",
+                "default": ""
+              },
+              "bundle": {
+                "title": "Resource bundle name",
+                "type": "string"
+              }
+            },
+            "required": [
+              "bundle"
+            ],
+            "additionalProperties": false
           }
-        },
-        "required": [
-          "glob"
-        ],
-        "additionalProperties": false
+        ]
       }
     },
     "bundles": {
       "title": "Metadata to ensure resource bundles are available",
       "type": "array",
       "default": [],
+      "deprecated": true,
       "items": {
         "title": "Resource bundle that should be available",
         "type": "object",
@@ -125,6 +157,148 @@
         ],
         "additionalProperties": false
       }
+    },
+    "foreign": {
+      "properties": {
+        "downcalls": {
+          "default": [],
+          "items": {
+            "properties": {
+              "condition": {
+                "$ref": "config-condition-schema-v1.0.0.json",
+                "title": "Condition under which the downcall stub should be registered"
+              },
+              "returnType": {
+                "type": "string",
+                "title": "Memory layout definition (allows canonical layouts; see `java.lang.foreign.Linker`)"
+              },
+              "parameterTypes": {
+                "default": [],
+                "items": {
+                  "type": "string",
+                  "title": "Memory layout definition (allows canonical layouts; see `java.lang.foreign.Linker`)"
+                },
+                "type": "array",
+                "title": "List of the function descriptor's parameter types"
+              },
+              "options": {
+                "type": "object",
+                "title": "Linker options (see `java.lang.foreign.Linker.Option`)",
+                "properties": {
+                  "captureCallState": {
+                    "type": "boolean",
+                    "title": "Specifies whether a call state should be captured. The specific states to capture are determined at run time. See also: `java.lang.foreign.Linker.Option.captureCallState`"
+                  },
+                  "critical": {
+                    "type": "object",
+                    "title": "See `java.lang.foreign.Linker.Option.critical`",
+                    "properties": {
+                      "allowHeapAccess": {
+                        "type": "boolean"
+                      }
+                    },
+                    "additionalProperties": false
+                  },
+                  "firstVariadicArg": {
+                    "type": "integer",
+                    "title": "See `java.lang.foreign.Linker.Option.firstVariadicArg`"
+                  }
+                },
+                "additionalProperties": false
+              }
+            },
+            "additionalProperties": false,
+            "type": "object",
+            "title": "Function descriptor to be registered for a downcall"
+          },
+          "type": "array",
+          "title": "List of function descriptors that should be registered for downcalls"
+        },
+        "upcalls": {
+          "default": [],
+          "items": {
+            "properties": {
+              "condition": {
+                "$ref": "config-condition-schema-v1.0.0.json",
+                "title": "Condition under which the upcall stub should be registered"
+              },
+              "returnType": {
+                "type": "string",
+                "title": "Memory layout definition (allows canonical layouts; see `java.lang.foreign.Linker`)"
+              },
+              "parameterTypes": {
+                "default": [],
+                "items": {
+                  "type": "string",
+                  "title": "Memory layout definition (allows canonical layouts; see `java.lang.foreign.Linker`)"
+                },
+                "type": "array",
+                "title": "List of the function descriptor's parameter types"
+              },
+              "options": {
+                "type": "object",
+                "title": "Linker options (see `java.lang.foreign.Linker.Option`)",
+                "description": "Currently, no linker options are allowed for upcalls. This may change in the future.",
+                "properties": { },
+                "additionalProperties": false
+              }
+            },
+            "additionalProperties": false,
+            "type": "object",
+            "title": "Function descriptor to be registered for an upcall"
+          },
+          "type": "array",
+          "title": "List of function descriptors that should be registered for upcalls"
+        },
+        "directUpcalls": {
+          "default": [],
+          "items": {
+            "properties": {
+              "condition": {
+                "$ref": "config-condition-schema-v1.0.0.json",
+                "title": "Condition under which the direct upcall stub should be registered"
+              },
+              "class": {
+                "type": "string",
+                "title": "Fully-qualified class name (e.g. `org.package.OuterClass$InnerClass`)"
+              },
+              "method": {
+                "type": "string",
+                "title": "Method name"
+              },
+              "returnType": {
+                "type": "string",
+                "title": "Memory layout definition (allows canonical layouts; see `java.lang.foreign.Linker`)"
+              },
+              "parameterTypes": {
+                "default": [],
+                "items": {
+                  "type": "string",
+                  "title": "Memory layout definition (allows canonical layouts; see `java.lang.foreign.Linker`)"
+                },
+                "type": "array",
+                "title": "List of the function descriptor's parameter types "
+              },
+              "options": {
+                "type": "object",
+                "title": "Linker options (see `java.lang.foreign.Linker.Option`)",
+                "description": "Currently, no linker options are allowed for direct upcalls. This may change in the future.",
+                "properties": { },
+                "additionalProperties": false
+              }
+            },
+            "additionalProperties": false,
+            "type": "object",
+            "title": "Java method and function descriptor to be registered for a direct upcall"
+          },
+          "type": "array",
+          "title": "List of Java methods and function descriptors that should be registered for direct upcalls"
+        }
+      },
+      "type": "object",
+      "additionalProperties": false,
+      "title": "JSON schema for the FFM API configuration used by GraalVM Native Image",
+      "description": "For a description and examples of writing an FFM API configuration, see: https://github.com/oracle/graal/blob/master/docs/reference-manual/native-image/FFM-API.md"
     }
   },
   "required": [],
@@ -202,6 +376,16 @@
             "title": "Allow objects of this class to be instantiated with a call to jdk.internal.misc.Unsafe#allocateInstance or JNI's AllocObject",
             "type": "boolean",
             "default": false
+          },
+          "serializable": {
+            "title": "Allow objects of this class to be serialized and deserialized",
+            "type": "boolean",
+            "default": false
+          },
+          "jniAccessible": {
+            "title": "Register the type, including all registered fields and methods, for runtime JNI access",
+            "type": "boolean",
+            "default": false
           }
         },
         "additionalProperties": false
@@ -310,13 +494,50 @@
               "type": "array",
               "default": [],
               "items": {
-                "title": "Fully qualified name of the interface defining the proxy class",
+                "title": "Fully-qualified name of the interface defining the proxy class",
                 "type": "string"
               }
+            },
+            "lambda": {
+              "title": "Lambda class descriptor",
+              "type": "object",
+              "properties": {
+                "declaringClass": {
+                  "title": "The class in which the lambda class is defined",
+                  "$ref": "#/$defs/type"
+                },
+                "declaringMethod": {
+                  "title": "The method in which the lambda class is defined",
+                  "$ref": "#/$defs/method",
+                  "default": {}
+                },
+                "interfaces": {
+                  "title": "Non-empty list of interfaces implemented by the lambda class",
+                  "type": "array",
+                  "items": {
+                    "title": "Fully-qualified name of the interface implemented by the lambda class",
+                    "type": "string"
+                  }
+                }
+              },
+              "required": [
+                "declaringClass",
+                "interfaces"
+              ],
+              "additionalProperties": false
             }
           },
-          "required": [
-            "proxy"
+          "oneOf": [
+            {
+              "required": [
+                "proxy"
+              ]
+            },
+            {
+              "required": [
+                "lambda"
+              ]
+            }
           ],
           "additionalProperties": false
         }

This includes backporting the following PRs:

• [GR-60237] Introduce serializable flag to reflection metadata oracle/graal#10215
• [GR-60238] Include JNI reachability metadata with reflection oracle/graal#11066
• [GR-63591] Include resource bundles in resource configuration oracle/graal#11313
• [GR-48191] Enable lambda reflection queries oracle/graal#11350
• [GR-65773] Integrate foreign configuration into reachability-metadata.json oracle/graal#11805 (?)
• [GR-58549] Update FFM API docs. oracle/graal#11962 (?)

Describe the motivation for the backport.

Make GraalVM for JDK 21 compatible with the reachability-metadata.json files used with GraalVM 25.0

cc @loicottet @vjovanov

[vjovanov]

@zakkak I would recommend to backport the schema from the following PR:

oracle/graal#12255

We also need to ship it with GraalVM so we can use that in the Native Build Tools for smoother migration.

I recommend cherry picking required commits from the PR when it is merged (ETA 1-2 days).

[zakkak]

Hi @vjovanov, IIRC during the discussions around introducing reachability-metadata.json the GraalVM team had committed to backporting the changes to GraalVM Community Edition for JDK 21 as well.

Did that include only the initial backport?

Are there no plans of the GraalVM team backporting the required changes to bring support for the latest reachability-metadata.json schema to GraalVM Community Edition for JDK 21?

I would recommend to backport the schema from the following PR:

That would still require backporting the changes mentioned here, since oracle/graal#12255 builds on top of v1.1.0, right?

[vjovanov]

Yes, we will backport those changes as well as the necessary semantic changes.

@loicottet is currently running the metadata repo tests on 21 and 17.

[zakkak]

Thank you @vjovanov and @loicottet !