Development language #16
Replies: 20 comments 52 replies
-
|
Generating WASM from Java See https://www.infoworld.com/article/3692456/hands-on-with-java-and-wasm.html for a 6-months old photography of the tooling. Tools description:
Pros:
Cons:
|
Beta Was this translation helpful? Give feedback.
-
|
Google now provides a java to wasm GC, still in experimental state, see https://github.com/google/j2cl/blob/master/docs/getting-started-j2wasm.md |
Beta Was this translation helpful? Give feedback.
-
|
I've played around with this, building the sample wasm project (a main function returning "Hello from Java!"). The generated wasm is small (600 bytes) and doesn't seem to bring unused glue code (glue code source is 14.7MB, dowloaded glue code 38.5k). |
Beta Was this translation helpful? Give feedback.
-
|
Generating WASM from AssemblyScript See https://www.assemblyscript.org. A very promising thing, but very disappointing in the end. I tried it 18 months ago when I started exploring the WebAssembly topic for antlr. It's unusable for a real project. Pros:
Cons:
|
Beta Was this translation helpful? Give feedback.
-
|
Apart from Compose which is WIP, this is the only language designed for WebAssembly. |
Beta Was this translation helpful? Give feedback.
-
|
Generating WASM from Kotlin See https://kotlinlang.org/docs/wasm-overview.html In 'alpha' state as of writing. Couldn't find a link to their technical design, notably re runtime libraries. Pros:
Cons:
|
Beta Was this translation helpful? Give feedback.
-
What do you mean by massive tooling downloads? |
Beta Was this translation helpful? Give feedback.
-
|
When I ran the getting started, it triggered gigs of downloads |
Beta Was this translation helpful? Give feedback.
-
I believe it could be even smaller if you apply Binaryen, like here. |
Beta Was this translation helpful? Give feedback.
-
|
After Binaryen, it should be around 10K in the prod build. |
Beta Was this translation helpful? Give feedback.
-
|
JS files in the prod build also should be smaller for this example, around 4K, unoptimized. |
Beta Was this translation helpful? Give feedback.
-
|
Generating WASM from Dart See https://docs.flutter.dev/platform-integration/web/wasm In development as of writing. Pros:
Cons:
|
Beta Was this translation helpful? Give feedback.
-
|
Generating WASM from C#/Blazor See https://learn.microsoft.com/en-us/aspnet/core/blazor/?view=aspnetcore-8.0 Pros:
Cons:
|
Beta Was this translation helpful? Give feedback.
-
|
Generating WASM from Rust See https://rustwasm.github.io/book/ Pros:
Cons:
|
Beta Was this translation helpful? Give feedback.
-
|
Generating WASM from Go see https://zchee.github.io/golang-wiki/WebAssembly/ Pros:
Cons:
|
Beta Was this translation helpful? Give feedback.
-
|
Tbh with wasm this OOP debate is irrelevant. Regardless of which language is used to generate the wasm, only functions can be exported, not classes, making it actually easier for non OOP targets to implement a runtime wrapper. I have to agree that at this point the lack of wasm GC support in Go makes it impossible to implement a wrapper. That said I'm a bit skeptical with the arguments used by the Go guys for not implementing GC (interior pointers). That seems pretty weak. Since the host language is responsible for both the allocation and the GC, they have full control on whether or not interior pointers should be supported. But that's not something I can influence, so... |
Beta Was this translation helpful? Give feedback.
-
|
I'll take a look at the GC stuff. I suspect that there is more to it than just interior pointers. |
Beta Was this translation helpful? Give feedback.
-
|
So I did a bit of a dive into the whole interior pointer thing. It is indeed a major barrier for having Go generate wasm, as the Go runtime relies on it all over the place. So Go would have to use fat pointers, or do boxing (which would be slow). It isn't practical until WasmGC supports interior pointers. That is on the list of things to do, but I suspect it will be a long way off as really, only Go would benefit to any great degree. However, if the wasm isn't generated by Go, then maybe it can integrate (more playing is required on that, and I am skeptical). I am still skeptical about generating wasm at all - other than browsers, I am not seeing the advantage. Golang via LLVM IR (in whatever way), seems to offer the most flexibility. I am not necessarilly in a position to judge, but is running a parser in the browser really in such demand? |
Beta Was this translation helpful? Give feedback.
-
|
Parser in browser is in-demand for editing code-like stuff on the web. |
Beta Was this translation helpful? Give feedback.
-
Even I wrote a tool from scratch, I wouldn't choose Go anyway because of objective disadvantages highlighed in the current discussion and personal opinion. |
Beta Was this translation helpful? Give feedback.
-
|
Generating WASM from C++ see https://developer.mozilla.org/en-US/docs/WebAssembly/C_to_Wasm Pros:
Cons:
|
Beta Was this translation helpful? Give feedback.
-
|
From more readings, it looks like Emscripten, the tool for converting LLVM (intermediate compiler target) to wasm, will probably never support Wasm GC features. This is inherent to LLVM, a low-level representation for compiling native code to platforms (x64, ARM...), whereas Wasm GC is a paradigm shift from a platform to VM. |
Beta Was this translation helpful? Give feedback.
-
|
Generating WASM from Compose <-- what is that ? 😲 see https://github.com/compose-lang, but there isn't much there yet. Compose is a new language that I'm currently prototyping. It targets WebAssembly only. It has a ts-like syntax for easy migration from ts, and supports composition out of the box. Unlike js/ts/AssemblyScript, It (will) also support runtime reflection à la Java. Pros:
Cons:
Only mentioning it for completeness. That said, if there is appetite to help accelerate, why not... |
Beta Was this translation helpful? Give feedback.
-
|
Eric, what's your recommendation? A lot of cons overall. |
Beta Was this translation helpful? Give feedback.
-
|
My recommendation is to postpone that specific decision a bit . Testing with current NodeJS is a showstopper, but I just tried the new version v22, planned for April, and it works (on ARM). In parallel, we need to experiment with tools, notably this one: https://github.com/google/j2cl/blob/master/docs/getting-started-j2wasm.md which I just found out about and sounds very promising, and also Kotlin ? Re the decision being discussed here, I'd suggest the following plan:
|
Beta Was this translation helpful? Give feedback.
-
|
I just realized that a requirement is for the generated parsers language's wasm compiler to run in the web. The reason is that we want antlr to run fully in the browser. |
Beta Was this translation helpful? Give feedback.
-
I think Ter has probably forgotten a lot of the code by now. I realized that I had when coming back to do the Go stuff! |
Beta Was this translation helpful? Give feedback.
-
|
I think I have understanding of the current code base (not all details but general architecture and algorithms). I'd say it's much simpler than Kotlin compiler that I currently work on. |
Beta Was this translation helpful? Give feedback.
-
|
FYI I've been playing around a bit with Deno, migrating my Compose project (which uses antlr) from Node. |
Beta Was this translation helpful? Give feedback.
-
|
I have never used Deno myself, but it was used as part of a project under the LionWeb initiative (this project). The project was later moved out of Deno, because while development in Deno had interesting advantages, it was a nightmare to get it running in the browser. |
Beta Was this translation helpful? Give feedback.
-
|
FWIW I managed today to successfully run the following experiment (using my WIP Compose language):
This is galaxies away from any form of production quality stuff but it's evidence that what we want to achieve on the long term is feasible ! |
Beta Was this translation helpful? Give feedback.
-
|
One factor I would consider is the popularity of the language, as picking a well-known language could mean getting more contributions. For this reason, something as popular as Java or Typescript may be beneficial. |
Beta Was this translation helpful? Give feedback.
-
|
I think that that there are lots of candidate languages. I would prefer not to use Java and I don't really think Typescript is bad at all, but I'm not sure that it is the right thing for this kind of tool. |
Beta Was this translation helpful? Give feedback.
-
|
Typescript is ruled out because it can't compile to WebAssembly. |
Beta Was this translation helpful? Give feedback.
-
|
Unless something comes up, I suspect we have to postpone generating the parsers in a browser for now. As of writing, the only language that can be compiled in the browser is WIP. Changing language in the future is not a big deal if we're careful enough to keep dependencies minimal. It does not affect our runtime architecture. An unexpected consequence is that we don't urgently need to generate the tool as WebAssembly: if we can't compile the generated parsers in the browser, the value of running the tool in a browser is limited to antlr5-lab (antlr5 version of antlr4-lab). So what we need to decide now is which desktop language we want to use for the runtime and the generated parsers (I don't see any value in using different ones). That language needs to support Wasm GC and generate small binaries, which leaves us with few options:
Go tooling hasn't been explored. I understand that Go itself runs faster than Java, but we're not looking at the speed of Go, rather at the speed of the generated Wasm (not tested, so we have to assume it is equivalent across above languages). I believe we should focus on the following concerns:
It'd be great if you could express your preference, so we can move forward and get cracking !!! |
Beta Was this translation helpful? Give feedback.
-
|
Because you can use a maven plugin in gradle but you can't use a gradle plugin in maven (afaik) |
Beta Was this translation helpful? Give feedback.
-
I've just tried to add a Kotlin file that prints DetailsSubject: [PATCH] Add kotlin dependencies
---
Index: tool/pom.xml
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/tool/pom.xml b/tool/pom.xml
--- a/tool/pom.xml (revision 1855d9aa4c5ebb15284b95a07048b6260304629c)
+++ b/tool/pom.xml (date 1704409074876)
@@ -20,6 +20,10 @@
<maven>3.8</maven>
</prerequisites>
+ <properties>
+ <kotlin.version>1.9.22</kotlin.version>
+ </properties>
+
<dependencies>
<dependency>
<groupId>org.antlr</groupId>
@@ -46,7 +50,23 @@
<artifactId>icu4j</artifactId>
<version>72.1</version>
</dependency>
- </dependencies>
+ <dependency>
+ <groupId>org.jetbrains.kotlin</groupId>
+ <artifactId>kotlin-stdlib-jdk8</artifactId>
+ <version>${kotlin.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.jetbrains.kotlin</groupId>
+ <artifactId>kotlin-test</artifactId>
+ <version>${kotlin.version}</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.jetbrains.kotlin</groupId>
+ <artifactId>kotlin-stdlib-jdk8</artifactId>
+ <version>${kotlin.version}</version>
+ </dependency>
+ </dependencies>
<build>
<sourceDirectory>src</sourceDirectory>
<testResources>
@@ -223,6 +243,66 @@
</execution>
</executions>
</plugin>
- </plugins>
+ <plugin>
+ <groupId>org.jetbrains.kotlin</groupId>
+ <artifactId>kotlin-maven-plugin</artifactId>
+ <version>${kotlin.version}</version>
+ <executions>
+ <execution>
+ <id>compile</id>
+ <phase>compile</phase>
+ <goals>
+ <goal>compile</goal>
+ </goals>
+ <configuration>
+ <sourceDirs>
+ <source>src</source>
+ <source>target/generated-sources/annotations</source>
+ <source>target/generated-sources/antlr3</source>
+ <source>target/generated-sources/tool/src</source>
+ </sourceDirs>
+ </configuration>
+ </execution>
+ <execution>
+ <id>test-compile</id>
+ <phase>test-compile</phase>
+ <goals>
+ <goal>test-compile</goal>
+ </goals>
+ </execution>
+ </executions>
+ <configuration>
+ <jvmTarget>1.8</jvmTarget>
+ </configuration>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>default-compile</id>
+ <phase>none</phase>
+ </execution>
+ <execution>
+ <id>default-testCompile</id>
+ <phase>none</phase>
+ </execution>
+ <execution>
+ <id>compile</id>
+ <phase>compile</phase>
+ <goals>
+ <goal>compile</goal>
+ </goals>
+ </execution>
+ <execution>
+ <id>testCompile</id>
+ <phase>test-compile</phase>
+ <goals>
+ <goal>testCompile</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
</build>
</project> |
Beta Was this translation helpful? Give feedback.
-
|
My preference would be to not use WASM - I think it is very likely to drag us into solving incompatibility problems and issues that have nothing to do with the project. If I was doing this entirely on my own, would:
Anyway, that's my take on it. LLVM solved the codegen things many moons ago :) |
Beta Was this translation helpful? Give feedback.
-
If I started a new version, I would not use Java v4 as reference since it contains a lot of ineffective and excess code in my opinion.
I don't see big disadvantages of OOP (sure they exist but other approaches have other disadvantages) and this technology has proven its effectiveness over the years.
By the way, Kotlin can produce LLVM IR via Kotlin Native
I don't think using Go is reasonable for now.
It's a good idea. Probably it makes sense to raise a new topic about standalone assembiles (asemblies without dependencies). |
Beta Was this translation helpful? Give feedback.
-
|
To clarify, don't copy the v4 java tool or runtime, but use as a reference for the algorithms and implementations. Using OOP in the runtime would make implementing other runtimes that are not OOP a PITA, which is why we don't have a C runtime for V4. Does the runtime actually need anything from OOP - no? The tool is a different thing of course as it woudl be written once. But if the tool needs a jVM installed and this that and the other, then it is more complex to set up; though probably not a barrier to most. I know that you are a fan of Kotlin of course, and so you should be :). I think languages such as Go and perhaps Rust are more suitable to such tooling. But it's all a matter of opinion of course. I have not got around to learning Kotlin, but that's because nobody has asked me to. The ANTLR tool and the algoritihms are of course a lot less complex than compiler algorithms of course. But, unless there are plans to improve/change the v4 algorithms then it's basically the algorithmic reference implementation for what ever is decided to write this stuff in :) |
Beta Was this translation helpful? Give feedback.
-
|
Sorry if I wasn’t clear, the question was not whether a maven plugin can generate a .class from Kotlin, but whether it can generate WebAssembly.
|
Beta Was this translation helpful? Give feedback.
-
|
My colleagues answered that Kotlin KMP is bound to Gradle ecosystem and it's inconvenient to use it outside of Gradle. But they also don't fully understand why using Gradle is not an option. |
Beta Was this translation helpful? Give feedback.
-
|
I have no problem with Gradle, but this is not about the antlr5 project using maven or gradle, it's about antlr5 users needing to automate compilation of generated parsers to WebAssembly. Many of them use maven and we can't force them into gradle. |
Beta Was this translation helpful? Give feedback.
-
|
I want to write command-line tools that pass parse tree info around. (Being able to parse input is an important but just one task of working with the parse trees using XQuery.) What language do you all prefer? Initially I plan to work on this outside this project but eventually will want it to be integrated. |
Beta Was this translation helpful? Give feedback.
-
|
These days my language of choice is TypeScript, not because I like it, but because it’s the current best compromise for a statically typed language that can run everywhere. I miss reflection a lot but I’d rather do without it than giving up the browser.
|
Beta Was this translation helpful? Give feedback.
-
|
I speak as someone quite ignorant about TS, but I wonder if reflection could be partially replaced through compiler plugins. For another project written in Kotlin, we are planning to use compiler plugins to overcome the limitations that reflection has on certain targets. We plan to generate methods that would return the information we would have normally asked the reflection to provide. Compiler plugins in Kotlin (and especially in Kotlin 2) are very easy to configure (you just add something in the gradle script). Maybe in TS there are alternatives to compiler plugins that we could use to get the same result. Something like https://github.com/nonara/ts-patch . There also seems to be active discussion about adding this sort of compiler plugins in TS itself microsoft/TypeScript#54276 |
Beta Was this translation helpful? Give feedback.
-
|
Compiler plugins can certainly help with all sorts of activities, but not sure they can help bring reflection to JavaScript (which is the underlying culprit for the lack of proper reflection). |
Beta Was this translation helpful? Give feedback.
-
|
What I have in mind is this scenario (connected to our AST library): Normally I would figure out that A has a field called However, through a compiler plugin I could compile the same code to something like: In this case, if I know in advance what kind of reflection calls I would need I can sort of pre-calculate their results during compilation. For us the reflection is mostly uses to inspect Node classes so that we can have algorithms that are generic (given any node, we can figure out their properties and process all of them according to some logic). So we are pretty sure we can replace those usages in this way (by defining our methods like In the case of TS, my colleagues who know TS better than I do are using decorators to achieve some limited results (they do that in our framework for AST in TS, called Tylasu), but I would like to see if with compiler plugins we can get to a similar level of what we do with Kotlin on the JVM. |
Beta Was this translation helpful? Give feedback.
-
|
One is mad if one uses Python, Java, Typescript or Javascript for
Serverless - at least in AWS ;). The costs can be high without a good
design already, nevermind adding the startup times of interpreters or JVM
startup time. Depends on the requirements I suppose but I would not do that
myself unless I find myself working somewhere where they have already done
it and need to stick with the plan :) But, I digress from the discussion so
leave it there.
I am thinking that I might not have much to add to ANTLR5 given the
direction that you guys are taking it so far. I'm not against
the direction, but I think you guys already have this tech covered.
I will stick around though and see if I can help in any way. But I was
mostly interested in the Go side of things of course.
…On Fri, Feb 16, 2024 at 11:52 AM ericvergnaud ***@***.***> wrote:
Parser in browser is in-demand for editing code-like stuff on the web.
Also JavaScript and TypeScript is growing in popularity for serverless.
Not speaking of Python for AI.
—
Reply to this email directly, view it on GitHub
<#16 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAJ7TMD32VKZ4R6KJO5J4J3YT6METAVCNFSM6AAAAABAXARN5OVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DIOJVGUZDO>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Looks like there's lots of mad people out there then...
These are obviously just suggestions ... |
Beta Was this translation helpful? Give feedback.
-
|
I have completely upgraded the Maven build and I think I have solved the issue where people have to manually delete the .m2 directory. However, as I am currently in a B&B in Belize I don't have a good working environment to test this right now. But I will. The slight complication is there is a Maven 2 plugin for generating .dot output. I will l have to rewrite that as it is already unsupported. It should only take a couple of hours. The tests all work on my local Maven. But it's been a few months now so I will have to retest. I agree that a C target would be good. It is still the base point for a lot of systems, especially things like embedded. |
Beta Was this translation helpful? Give feedback.
-
|
Cool! |
Beta Was this translation helpful? Give feedback.
-
|
@ericvergnaud does it make sense to use antlr4rust as a starting point instead of c target? |
Beta Was this translation helpful? Give feedback.
-
|
Hi, we're not using the C target, we're using the Kotlin one. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
As mentioned in the Roadmap discussion, WebAssembly is still bleeding edge.
GC features are still minimal, strings are postponed...
Current Node v21 does not support Wasm GC. Node v22 does, but that's still experimental and ARM64 only.
Choosing a language now would be extremely risky, but that shouldn't prevent us from listing possible ones, discussing their pros and cons, and providing updates.
Disclaimer: as developers I suspect we are all strongly opinionated in favor of some languages and against others. I don't think that's a problem, but those opinions shouldn't leak here, and I will shut down the discussion if it drifts 😍.
Not sure how to organize this to prevent it from becoming a mess.
Maybe I start with 1 comment per language, and the per language discussion happens in the corresponding thread ?
Beta Was this translation helpful? Give feedback.
All reactions