Architectural overview?

[creichen]

If I want to understand the high-level architecture of TornadoVM, where would you recommend me to look before I dig into the source code? I am looking for documents/places in the code that would help me understand e.g.:

• What components are there that analyse Java and translate it? (IRs beyond the extended GraalVM IR mentioned in the FAQ and the IR mentioned in the VEE'19 paper?)
• Which of those components are backend-specific vs. shared across backends?
• Which are part of the adaptive run-time system vs. run only once ahead-of-time?
• What (if any) are the design assumptions that TornadoVM makes wrt the kinds of kernels, memory usage patterns, and task dependency patterns that it aims to support that go beyond the usual restrictions of GPU kernels (wrt recursion, dynamic memory binding, I/O etc.)?

Apologies if I missed something obvious, but this is the one thing I haven't been able to spot in the documentation-- I would greatly appreciate any pointers to papers, technical documentation, source files etc. that you'd recommend me to look at!

As background, I am particularly interested in understanding:

• options for supporting specific embedded DSLs in kernels (i.e., under what conditions it may be easier to support DSL features inside of TornadoVM instead of doing it in a separate system, prior to passing the code to TornadoVM's frontend)
• support for shared memory workloads (fine-grained GPU/CPU interaction, e.g. on unified memory setups)

If you have specific pointers for (prior TornadoVM work on) these questions, that would be fantastic, but I assume that they are orthogonal to the project's main scope.

[mikepapadim]

Thank you @creichen for your interest in TornadoVM.

If I want to understand the high-level architecture of TornadoVM, where would you recommend me to look before I dig into the source code? I am looking for documents/places in the code that would help me understand e.g.:

Regarding high level architecture of Tornadovm I suggest the following papers:

• https://dl.acm.org/doi/pdf/10.1145/3313808.3313819 VEE 2019
• https://dl.acm.org/doi/pdf/10.1145/3237009.3237016 MPLR2018
• https://pure.manchester.ac.uk/ws/files/190176058/MPapadimitriou_VEE2021_MTMD.pdf VEE 2021
• research.manchester.ac.uk/files/233043755/MXekalaki_vldb2023.pdf VLDB 2023

There are also a few talks:

• https://www.youtube.com/watch?v=WNQ5ylMs4Ok TornadoVM Deep Dive Devoxx Belgium 2025
• https://www.youtube.com/watch?v=nPlacnadR6k TornadoVM JavaOne
• https://www.youtube.com/watch?v=kOo07M7MXGA TornadoVM Multi-device tutorial
• https://www.youtube.com/watch?v=Bo0zMdcdCk4 TornadoVM Qcon
• https://www.youtube.com/watch?v=VTzGlnv6nuA TornadoVM JVMLS

• What components are there that analyse Java and translate it? (IRs beyond the extended GraalVM IR mentioned in the FAQ and the IR mentioned in the VEE'19 paper?)

I think what you are looking for here is the Sketch Phase of the TornadoVM
TornadoSketchTier is where TornadoVM turns “arbitrary Java” into “GPU-legal Java” before extracting Tornado IR and lowering to PTX/SPIR-V/OpenCL. TornadoSketchTier rewrites Graal IR into a restricted, parallel, GPU-legal form from which Tornado IR can be extracted.

https://github.com/beehive-lab/TornadoVM/blob/master/tornado-runtime/src/main/java/uk/ac/manchester/tornado/runtime/graal/compiler/TornadoSketchTier.java

• Graal IR → analyzed & reshaped by TornadoSketchTier
• No new IR yet — still Graal IR, but restricted
• Tornado IR is generated after this tier

Some passes that it make sense to look at are:

• TornadoNativeTypeElimination: Eliminates or rewrites Java/native types that cannot be represented on accelerator backends.
• TornadoApiReplacement: Lowers TornadoVM API calls into canonical IR constructs understood by later compiler stages.
• TornadoKernelContextReplacement: Rewrites kernel context accesses (thread IDs, dimensions) into backend-agnostic IR nodes.
• TornadoAutoParalleliser: Identifies data-parallel loops and rewrites them to expose parallel execution semantics.
• TornadoDataflowAnalysis: Builds explicit dataflow information to determine dependencies and legal parallel execution.
• TornadoPanamaPrivateMemory: Analyzes and assigns private (thread-local) memory regions for kernel execution.

• Which of those components are backend-specific vs. shared across backends?

Sketch Passes are backend-agnostic. Now, for the compiler infra of each backend there is a mixture of passes shared between backends and backend-specific ones:

Take a look:

• OpenCl: https://github.com/beehive-lab/TornadoVM/tree/master/tornado-drivers/opencl/src/main/java/uk/ac/manchester/tornado/drivers/opencl/graal/phases
• PTX: https://github.com/beehive-lab/TornadoVM/tree/master/tornado-drivers/ptx/src/main/java/uk/ac/manchester/tornado/drivers/ptx/graal/phases
• SPIRV: https://github.com/beehive-lab/TornadoVM/tree/master/tornado-drivers/spirv/src/main/java/uk/ac/manchester/tornado/drivers/spirv/graal/phases
• Common among backends: https://github.com/beehive-lab/TornadoVM/tree/master/tornado-drivers/drivers-common/src/main/java/uk/ac/manchester/tornado/drivers/common/compiler/phases

• Which are part of the adaptive run-time system vs. run only once ahead-of-time?

Not sure about that, can you please give a bit more details.

• What (if any) are the design assumptions that TornadoVM makes wrt the kinds of kernels, memory usage patterns, and task dependency patterns that it aims to support that go beyond the usual restrictions of GPU kernels (wrt recursion, dynamic memory binding, I/O etc.)?

If i get that right what you are looking for is

1. No JVM object graph semantics inside kernels :Kernels operate on primitive arrays / structs, not heap object graphs or virtual dispatch.
2. No memory overflows: Tornadovm always check max available GPU;s ram available and does not allocate over this limit.
3. No reflection, no dynamic memory allocation, no library inlining apart of standard JVM math libraries (with some restrictions.)
4. Memory lifetime: buffers are expected to live within the scope of a TaskGraph exect is stated differently with the Consume/Persist API that allows buffers to live between TaskGraphs.
5. TaskDeps: Tasks within a Taskgraph can share multiple deps, usually scheduled by the driver and tornadovm ensures data consistansy among runs. Deps can be shared also between TaskGraph-level granularity but user needs to explitcly handle deps with consume persist api

• https://github.com/beehive-lab/TornadoVM/blob/master/tornado-unittests/src/main/java/uk/ac/manchester/tornado/unittests/tasks/TestMultipleTasksSingleDevice.java Intra-TaskGraph Buffer Sharing
• https://github.com/beehive-lab/TornadoVM/blob/master/tornado-unittests/src/main/java/uk/ac/manchester/tornado/unittests/api/TestSharedBuffers.java Multi-TaskGraph Buffer Sharing

Apologies if I missed something obvious, but this is the one thing I haven't been able to spot in the documentation-- I would greatly appreciate any pointers to papers, technical documentation, source files etc. that you'd recommend me to look at!

As background, I am particularly interested in understanding:

• options for supporting specific embedded DSLs in kernels (i.e., under what conditions it may be easier to support DSL features inside of TornadoVM instead of doing it in a separate system, prior to passing the code to TornadoVM's frontend)

I would really need a bit more info in order to answer this on the semantics of DSL you want to target? What kind of primitives and operators would like to support?

• support for shared memory workloads (fine-grained GPU/CPU interaction, e.g. on unified memory setups)

Regarding shared memory workload i think what i a good pointer is this paper, we had a POC a couple of years ago. This work enabled unified memory for the JVM heap allows managed runtimes like TornadoVM to offload computation to GPUs without blocking the garbage collector, improving host–device interoperability while still requiring explicit synchronization. This work and the fact that we based for our data-structured on project Panama (FFM) make fine-grained CPU-GPU interaction possible.

https://dl.acm.org/doi/pdf/10.1145/3617651.3622984

If you have specific pointers for (prior TornadoVM work on) these questions, that would be fantastic, but I assume that they are orthogonal to the project's main scope.

I hope these are answer your questions! If you need any more info please let me know

[creichen]

Thank you very much, @mikepapadim ! I have some reading to do now (I haven't really used GraalVM before, either). :-)

I believe that I have the necessary pointers to figure out the rest myself now, but since you were curious, I'll follow up on the "semantics of the DSL I want to target":

That DSL is very much a WIP, due to the scope of the project being a bit up in the air (how much of the code to refactor to use such a DSL etc.), and I expect that I'll need to do some prototyping to understand what makes sense here. My workloads are from compilers and program analysis and normally live on the Java heap; the most computationally heavy bits involve constructing and (especially) propagating information along (sparse) graph structures. At least the following will likely be of interest:

• rewriting dynamically dispatched calls to conditionals (we often have only 2-3 cases, determined by dynamic type information of the graph node)
• a purely functional "SmallSet" API, probably with a bit vector encoding under the hood
  • possibly restricting to static vector sizes (e.g., 64 bit), if that covers enough workloads
• rewriting specific method calls and constants to symbolic values depending on where they flow, e.g. (in combination with the Set API), something like return myset.insert(this.getFoo()) might want to be rewritten into return myset | (1L << f(kernelcontext, localThreadIndex)), where f might e.g. be a table lookup, a constant function, or a simple conditional or arithmetic expression.
• operations for emitting workloads to work queues that the next "iteration" will dispatch kernels over-- e.g., a kernel might compute some "local" value for a graph node, then check if the local value has changed, and if so, propagate information along all directed edges to update neighbouring nodes, adding 0, 1, or 2 entries to the work queue per kernel thread (assuming max out-degree of 2 for now; the literature describes some strategies for higher out-degrees, but we can probably start without those).

For controlling execution, the following will be relevant:

• fixed point iteration (iterate kernels until work queues are empty or so short that it may be faster to continue on the CPU-- we may have some long tails!)
• eliminating (mutual) recursion (I expect this to be easier to do in a pre-TornadoVM pass)

While I am not sure how much this aligns with the aims of TornadoVM: if there are any TornadoVM-related pointers you can think of, that would be fantastic! :-)