Use Vector API in the Java Extension #824
Conversation
…fter refactoring the vectorized class.
samyron
force-pushed
the
sm/java-vector-simd
branch
from
194ba01 to
15c7187
Compare
|
Using hsdis to examine the generated assembly I can verify that on my Macbook Air the Hotspot C2 Compiler does indeed use Neon instructions. |
|
@samyron OMG I look away for a few days and you just go and do it! Bravo! I'll have a look at these changes soon and see if I can offer any suggestions. This API is still a bit of a moving target, but I think we can work around that with a little Ruby magic here and there. I will also point the Vector API folks at this PR so they can see what we're doing and provide additional input. Amazing work! |
|
I've posted a thread to the panama-dev list here: https://mail.openjdk.org/pipermail/panama-dev/2025-July/021080.html |
…for better inlining. (#8)
|
I decided to try a different approach after looking at the HotSpot C2 output. Unlike in the C extension, where we mostly control method inlining, HotSpot isn't so easily influenced. I merged Performance of
|
|
@samyron This is interesting progress! I am looking forward to trying it myself now that I'm back in the office. Yes, HotSpot can be a tricky beast to manipulate. We will want to look at some deeper logging of the JIT and inlining decisions to see whether everything that should be is getting inlined. There's potentially other parts of json unrelated to your changes that are also interfering with inlining (such as the double-dispatching logic to find an appropriate formatter for output text). Have you tried running on a newer JDK? There's continuous improvements in this area. |
|
It's also possible that we are losing too much performance to excessive allocation. I'll try to do some profiling once I get your code up and running. |
|
Oh, BTW, I got one response to my email about your work, pointing me to a Java library that has already been attempting to use the vector API to speed up json processing. It may provide some interesting pointers: https://github.com/simdjson/simdjson-java |
I have tried running the same benchmarks using JDK 24. These benchmarks have some WIP changes that aren't reflected in this branch but I have seen the highest peak performance in the Edit: Some quick testing shows changing the order of benchmarks does change the results a bit. I moved the Run 1Run 2 |
|
Those numbers are quite a bit better, albeit unpredictable! Order changes likely indicate that there's some polymorphism interfering with optimization, breaking inlining and falling back on slower calls with less optimization. That's the sort of JIT logging I'm hoping to dig into soon. Once I can focus on this we have some OpenJDK folks interested in seeing the results and helping us tune things. |
|
After reading How we made JSON.stringify more than twice as fast and having the VisualVM results fresh in my mind... I figured we could try segmenting the output buffer into chunks to completely avoid the I crated two very similar Here is a screenshot of profiling results using 
The benchmarks are also much more consistent between invocations. First runSecond run immediately after the firstNote: The C extension's |
|
I added a SWAR Implementation of basic escape scanning when the Vector-API based implementation is disabled. Performance is quite good. |
What, you mean my super-naïve implementation was not efficient? 😆 This is an excellent improvement! Could you move the stream improvements to a separate PR so it doesn't get tied up with the Vector work? I'd expect we can merge it immediately! It's on my list to revisit this work this week, now that I'm back from holiday. I'd also like to feature this work as an example of JRuby's potential on newer JVMs in my upcoming conference talks. |
@headius Unfortunately the segmented output stream by itself didn't have much (or really any) of a performance impact by itself. The bottleneck is the See #835. |
|
@samyron Perhaps the segmented version would show more impact with larger output? We may be chasing our tails here, though... ideally if you are generating tens of megabytes of json you're streaming it somewhere and not buffering it. The ByteList form is just to fulfill the API returning a String if you don't provide an output stream. The SWAR results are excellent though! |
PLEASE DO NOT MERGE
Overview
This PR uses the jdk.incubator.vector module as mentioned in issue #739 to accelerate generating JSON with the same algorithm as the C extension.
The PR as it exists right now, it will attempt to build the
json.ext.VectorizedEscapeScannerclass with a target release of16. This is the first version of Java with support for thejdk.incubator.vectormodule. The remaining code is built for Java 1.8. The code will attempt to load thejson.ext.VectorizedEscapeScanneronly if thejson.enableVectorizedEscapeScannersystem property is set totrue(or1).I'm not entirely sure how this is packaged / included with JRuby so I'd love @byroot and @headius's (and others?) thought about how to potential package and/or structure the JARs. I did consider adding the
json.ext.VectorizedEscapeScannerto a separategenerator-vectorized.jarbut I thought I'd solicit feedback before spending any more time on the build / package process.Benchmarks
Machine M1 Macbook Air
Note: I've had trouble modifying the
compare.rbI was using for the C extension to work reliability with the Java extension. I'll probably spend more time trying to get it to work, but as of right now these are pretty raw benchmarks.Below are two sample runs of the real-world benchmarks. The benchmarks are much more variable then the C extension for some reason. I'm not sure if HotSpot is doing something slightly different per execution.
Vector API Enabled
Vector API Disabled
masteras of commitc5af1b68c582335c2a82bbc4bfa5b3e41ead1ebaObservations
activitypub.jsonandtwitter.jsonseem to be consistently faster with the Vector API enabled.citm_catalog.jsonseems consistently a bit slower andohai.jsonis fairly close to even.