[GR-37533] Merge master into 24.0 release branch for feature freeze

PullRequest: truffleruby/4144

Author: eregon

3rd_party_licenses.txt

@@ -1308,7 +1308,7 @@ of
 */
 ================================================================================
 
-ffi 1.14.2
+ffi 1.15.5
 
 Copyright (c) 2008-2016, Ruby FFI project contributors
 All rights reserved.

CHANGELOG.md

@@ -3,6 +3,8 @@
 New features:
 
 * C/C++ extensions are now compiled using the system toolchain and executed natively instead of using GraalVM LLVM (Sulong). This leads to faster startup, no warmup, better compatibility, smaller distribution and faster installation for C/C++ extensions (#3118, @eregon).
+* Full suport for the Ruby 3.2 and Ruby 3.3 syntax by adopting the [Prism](https://github.com/ruby/prism) parser, which is about twice as fast as the old parser (#3117, #3038, #3039, @andrykonchin, @eregon).
+* Pattern matching is now fully supported, with the exception of Find pattern (`in [*, a, *]`) (#3332, #2683, @eregon, @razetime).
 
 Bug fixes:
 
@@ -12,6 +14,13 @@ Bug fixes:
 * Fix `Proc#parameters` and return all the numbered parameters lower than the used explicitly ones (@andrykonchin).
 * Fix some C API functions which were failing when called with Ruby values represented as Java primitives (#3352, @eregon).
 * Fix `IO.select([io], nil, [io])` on macOS, it was hanging due to a bug in macOS `poll(2)` (#3346, @eregon, @andrykonchin).
+* Run context cleanup such as showing the output of tools when `SignalException` and `Interrupt` escape (@eregon).
+* Handle a new variable inside the `case` target expression correctly (#3377, @eregon).
+* The arguments of `Thread.new(*args, &block)` need to be marked as shared between multiple threads (#3179, @eregon).
+* Fix `Range#bsearch` and raise `TypeError` when range boundaries are non-numeric and block not passed (@andrykonchin).
+* Fix using the `--cpusampler` profiler when there are custom unblock functions for `rb_thread_call_without_gvl()` (#3013, @eregon).
+* Fix recursive raising `FrozenError` exception when redefined `#inspect` modifies an object (#3388, @andrykonchin).
+* Fix `Integer#div` returning the wrong object type when the divisor is a `Rational` (@simonlevasseur, @nirvdrum).
 
 Compatibility:
 
@@ -21,6 +30,21 @@ Compatibility:
 * Promote `File#path` and `File#to_path` to `IO#path` and `IO#to_path` and make IO#new accept an optional `path:` keyword argument (#3039, @moste00)
 * Display "unhandled exception" as the message for `RuntimeError` instances with an empty message (#3255, @nirvdrum).
 * Set `RbConfig::CONFIG['configure_args']` for openssl and libyaml (#3170, #3303, @eregon).
+* Support `Socket.sockaddr_in(port, Socket::INADDR_ANY)` (#3361, @mtortonesi).
+* Implement the `Data` class from Ruby 3.2 (#3039, @moste00, @eregon).
+* Make `Coverage.start` and `Coverage.result` accept parameters (#3149, @mtortonesi, @andrykonchin).
+* Implement `rb_check_funcall()` (@eregon).
+* Implement `MatchData#{byteoffset,deconstruct,deconstruct_keys}` from Ruby 3.2 (#3039, @rwstauner).
+* Add `Integer#ceildiv` method (#3039, @simonlevasseur, @nirvdrum).
+* Implement `Class#attached_object` method (#3039, @andrykonchin).
+* Fix `ENV#{clone,dup}` and raise `TypeError` (#3039, @andrykonchin).
+* Fix `Coverage.supported?` and raise `TypeError` if argument is not Symbol (#3039, @andrykonchin).
+* Accept options argument to `Regexp.{new,compile}` of String and warn for unknown types (#3039, @rwstauner).
+* Implement `Time#deconstruct_keys` from Ruby 3.2 (#3039, @rwstauner).
+* Do not autosplat a proc that accepts a single positional argument and keywords (#3039, @andrykonchin).
+* Support passing anonymous * and ** parameters as method call arguments (#3039, @andrykonchin).
+* Handle either positional or keywords arguments by default in `Struct.new` (#3039, @rwstauner).
+* Promote `Set` class to core library (#3039, @andrykonchin).
 
 Performance:
 

LICENCE.md

@@ -1,6 +1,6 @@
 # TruffleRuby Licence
 
-TruffleRuby is copyright (c) 2013-2023 Oracle and/or its affiliates, and is made
+TruffleRuby is copyright (c) 2013-2024 Oracle and/or its affiliates, and is made
 available to you under the terms of any one of the following three licenses:
 
 * Eclipse Public License version 2.0, or

README.md

@@ -90,7 +90,7 @@ TruffleRuby is actively tested on the following systems:
 * Ubuntu 16.04, 18.04, 20.04, 22.04 (all LTS)
 * Fedora 37, 38
 * Debian 10, 11, 12
-* macOS 10.14 (Mojave) for amd64, 11 (Big Sur) for aarch64
+* macOS 11 (Big Sur)
 
 Architectures:
 
@@ -182,7 +182,7 @@ For known vulnerabilities in Ruby, please refer to the [known-cves](doc/user/kno
 
 ## Licence
 
-TruffleRuby is copyright (c) 2013-2023 Oracle and/or its affiliates, and is made
+TruffleRuby is copyright (c) 2013-2024 Oracle and/or its affiliates, and is made
 available to you under the terms of any one of the following three licenses:
 
 * Eclipse Public License version 2.0, or

ci.jsonnet

@@ -251,7 +251,7 @@ local part_definitions = {
       platform_name:: "DarwinAMD64",
       "$.cap":: {
         # GR-45839, GR-46279: exclude macmini_late_2014_8gb, they are too slow, have too little RAM and cause various timeouts
-        normal_machine: ["darwin_mojave", "!macmini_late_2014_8gb"],
+        normal_machine: ["darwin_bigsur", "!macmini_late_2014_8gb"],
       },
       environment+: {
         LANG: "en_US.UTF-8",
@@ -521,14 +521,14 @@ local composition_environment = utils.add_inclusion_tracking(part_definitions, "
       "ruby-test-specs-linux-amd64-stable":    $.platform.linux  + $.jdk.stable + $.env.jvm + gate_no_build + $.use.build + $.run.test_unit_tck + native_config + $.run.test_specs + { timelimit: "01:20:00" },
       "ruby-test-specs-linux-amd64-latest":    $.platform.linux  + $.jdk.latest + $.env.jvm + gate_no_build + $.use.build + $.run.test_unit_tck + native_config + $.run.test_specs + { timelimit: "01:20:00" },
       "ruby-test-specs-darwin-amd64-stable":   $.platform.darwin_amd64 + $.jdk.stable + $.env.jvm + gate_no_build + $.use.build + $.run.test_unit_tck + native_config + $.run.test_specs + { timelimit: "01:40:00" },
-      # "ruby-test-specs-darwin-amd64-latest":   $.platform.darwin_amd64 + $.jdk.latest + $.env.jvm + gate_no_build + $.use.build + $.run.test_unit_tck + native_config + $.run.test_specs + { timelimit: "01:40:00" }, # GR-49652
+      "ruby-test-specs-darwin-amd64-latest":   $.platform.darwin_amd64 + $.jdk.latest + $.env.jvm + gate_no_build + $.use.build + $.run.test_unit_tck + native_config + $.run.test_specs + { timelimit: "01:40:00" },
       "ruby-test-specs-darwin-aarch64-stable": $.platform.darwin_aarch64 + $.jdk.stable + $.env.jvm + gate_no_build + $.use.build + $.run.test_unit_tck + native_config + $.run.test_specs + { timelimit: "01:40:00" },
       "ruby-test-specs-darwin-aarch64-latest": $.platform.darwin_aarch64 + $.jdk.latest + $.env.jvm + gate_no_build + $.use.build + $.run.test_unit_tck + native_config + $.run.test_specs + { timelimit: "01:40:00" },
       "ruby-test-fast-linux-aarch64":   $.platform.linux_aarch64 + $.jdk.stable + $.env.jvm + gate + $.run.test_fast + native_config + { timelimit: "45:00" },
       "ruby-test-fast-linux-amd64":     $.platform.linux  + $.jdk.stable + $.env.jvm + gate + $.run.test_fast + { timelimit: "45:00" },  # To catch missing slow tags
-      "ruby-test-mri-asserts":          $.platform.linux  + $.jdk.stable + $.env.jvm + gate + $.run.test_mri_fast + { timelimit: "01:30:00" },
+      "ruby-test-mri-asserts":          $.platform.linux  + $.jdk.stable + $.env.jvm + gate + $.run.test_mri_fast + { timelimit: "01:15:00" },
       "ruby-test-mri-linux-amd64":      $.platform.linux  + $.jdk.stable + $.env.native + gate + $.run.test_mri + { timelimit: "01:20:00" },
-      "ruby-test-mri-linux-aarch64":    $.platform.linux_aarch64 + $.jdk.stable + $.env.native + gate + $.run.test_mri + { timelimit: "01:20:00" },
+      # "ruby-test-mri-linux-aarch64":  $.platform.linux_aarch64 + $.jdk.stable + $.env.native + gate + $.run.test_mri + { timelimit: "01:30:00" }, # GR-51361
       "ruby-test-mri-darwin-amd64":     $.platform.darwin_amd64 + $.jdk.stable + $.env.native + gate + $.run.test_mri + { timelimit: "01:30:00" },
       "ruby-test-mri-darwin-aarch64":   $.platform.darwin_aarch64 + $.jdk.stable + $.env.native + gate + $.run.test_mri + { timelimit: "01:30:00" },
       "ruby-test-integration-linux-amd64": $.platform.linux  + $.jdk.stable + $.env.jvm + gate + $.run.test_integration,
@@ -551,7 +551,7 @@ local composition_environment = utils.add_inclusion_tracking(part_definitions, "
       "ruby-test-svm-ce-linux-amd64-stable":    $.platform.linux          + $.jdk.stable + $.env.native    + $.env.gdb_svm + gate + native_tests + $.env.host_inlining_log,
       "ruby-test-svm-ce-linux-amd64-latest":    $.platform.linux          + $.jdk.latest + $.env.native    + $.env.gdb_svm + gate + native_tests,
       "ruby-test-svm-ce-darwin-amd64-stable":   $.platform.darwin_amd64   + $.jdk.stable + $.env.native    + $.env.gdb_svm + gate + native_tests,
-      # "ruby-test-svm-ce-darwin-amd64-latest":   $.platform.darwin_amd64   + $.jdk.latest + $.env.native    + $.env.gdb_svm + gate + native_tests, # GR-49652
+      "ruby-test-svm-ce-darwin-amd64-latest":   $.platform.darwin_amd64   + $.jdk.latest + $.env.native    + $.env.gdb_svm + gate + native_tests,
       "ruby-test-svm-ce-darwin-aarch64-stable": $.platform.darwin_aarch64 + $.jdk.stable + $.env.native    +                 gate + native_tests,
       "ruby-test-svm-ce-darwin-aarch64-latest": $.platform.darwin_aarch64 + $.jdk.latest + $.env.native    +                 gate + native_tests,
       "ruby-test-svm-ee-linux-amd64":    $.platform.linux          + $.jdk.stable + $.env.native_ee + $.env.gdb_svm + gate + native_tests + $.env.host_inlining_log + { timelimit: "01:30:00" },

ci/common.jsonnet

@@ -344,7 +344,8 @@ local common_json = import "../common.json";
     },
 
     local linux   = { os:: "linux",   capabilities+: [self.os] },
-    local darwin  = { os:: "darwin",  capabilities+: [self.os] },
+    # Run darwin jobs on Big Sur or later by excluding all older versions
+    local darwin  = { os:: "darwin",  capabilities+: [self.os, "!darwin_sierra", "!darwin_mojave", "!darwin_catalina"] },
     local windows = { os:: "windows", capabilities+: [self.os] },
 
     local amd64   = { arch:: "amd64",   capabilities+: [self.arch] },

common.json

@@ -44,13 +44,13 @@
     "labsjdk-ee-21Debug": {"name": "labsjdk",   "version": "ee-21.0.1+11-jvmci-23.1-b26-debug", "platformspecific": true },
     "labsjdk-ee-21-llvm": {"name": "labsjdk",   "version": "ee-21.0.1+11-jvmci-23.1-b26-sulong", "platformspecific": true },
 
-    "oraclejdk-latest":       {"name": "jpg-jdk",   "version": "22",      "build_id": "27", "release": true, "platformspecific": true, "extrabundles": ["static-libs"]},
-    "labsjdk-ce-latest":      {"name": "labsjdk",   "version": "ce-22+27-jvmci-b01", "platformspecific": true },
-    "labsjdk-ce-latestDebug": {"name": "labsjdk",   "version": "ce-22+27-jvmci-b01-debug", "platformspecific": true },
-    "labsjdk-ce-latest-llvm": {"name": "labsjdk",   "version": "ce-22+27-jvmci-b01-sulong", "platformspecific": true },
-    "labsjdk-ee-latest":      {"name": "labsjdk",   "version": "ee-22+27-jvmci-b01", "platformspecific": true },
-    "labsjdk-ee-latestDebug": {"name": "labsjdk",   "version": "ee-22+27-jvmci-b01-debug", "platformspecific": true },
-    "labsjdk-ee-latest-llvm": {"name": "labsjdk",   "version": "ee-22+27-jvmci-b01-sulong", "platformspecific": true }
+    "oraclejdk-latest":       {"name": "jpg-jdk",   "version": "22",      "build_id": "33", "release": true, "platformspecific": true, "extrabundles": ["static-libs"]},
+    "labsjdk-ce-latest":      {"name": "labsjdk",   "version": "ce-22+33-jvmci-b01", "platformspecific": true },
+    "labsjdk-ce-latestDebug": {"name": "labsjdk",   "version": "ce-22+33-jvmci-b01-debug", "platformspecific": true },
+    "labsjdk-ce-latest-llvm": {"name": "labsjdk",   "version": "ce-22+33-jvmci-b01-sulong", "platformspecific": true },
+    "labsjdk-ee-latest":      {"name": "labsjdk",   "version": "ee-22+33-jvmci-b01", "platformspecific": true },
+    "labsjdk-ee-latestDebug": {"name": "labsjdk",   "version": "ee-22+33-jvmci-b01-debug", "platformspecific": true },
+    "labsjdk-ee-latest-llvm": {"name": "labsjdk",   "version": "ee-22+33-jvmci-b01-sulong", "platformspecific": true }
   },
 
   "eclipse": {

doc/contributor/how-to-guide.md

@@ -1139,24 +1139,6 @@ jt -q ruby -e 'puts Truffle::Debug.java_class_of([])'
 RubyArray
 ```
 
-### Truffle::Debug.yarp_serialize
-
-```
-jt -q ruby -e 'puts Truffle::Debug.yarp_serialize("1").dump'
-"YARP\x00\x04\x00@7\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00L\f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00S\x19\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00&\b\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00"
-```
-
-### Truffle::Debug.yarp_parse
-
-```
-jt -q ruby -e 'puts Truffle::Debug.yarp_parse("[].to_ary")'
-ProgramNode
-  Scope
-  StatementsNode
-    CallNode
-      ArrayNode
-```
-
 ### Truffle::Debug.ast
 
 ```

doc/contributor/interop_details.md

@@ -78,6 +78,28 @@
 
 # Behavior of interop messages for Ruby objects
 
+## toDisplayString()
+
+When interop message `toDisplayString` is sent
+- to 
+  it returns a String representing the object.
+- otherwise
+  it returns the same as `subject.inspect`.
+
+## MetaObject related messages
+
+When interop message `hasMetaObject` is sent
+- to **`nil`**, **`:symbol`**, **a `String`**, **a `BigDecimal`**, **an `Object`**, **a frozen `Object`**, **a `StructWithValue`**, **a `Class`**, **a `Hash`**, **an `Array`**, **an `Exception`**, **an `Exception` with a cause**, **`proc {...}`**, **`lambda {...}`**, **a `Method`**, **a `Truffle::FFI::Pointer`**, **polyglot pointer**, **polyglot members**, **polyglot array** or **polyglot hash**
+  it returns true.
+- otherwise
+  it returns false.
+
+When interop message `getMetaObject` is sent
+- to 
+  it returns the Ruby exception's class.
+- otherwise
+  it returns the same Ruby class as `subject.class`.
+
 ## `null` related messages
 
 When interop message `isNull` is sent
@@ -329,7 +351,7 @@ When interop message `writeMember` is sent
   it writes the given value under the given name.
 - to **a `StructWithValue`**
   it writes the value to the given struct member.
-- to **a `BigDecimal`**, **`nil`**, **`:symbol`** or **a frozen `Object`**
+- to **`nil`**, **`false`**, **`true`**, **`:symbol`**, **an `Integer`**, **a `Float`**, **a `BigDecimal`** or **a frozen `Object`**
   it fails with `UnsupportedMessageError` when the receiver is frozen.
 - otherwise
   it fails with `UnsupportedMessageError`.