Low-provenance conversion from Expr to SyntaxTree #17
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mlechu
force-pushed
the
expr-to-st
branch
3 times, most recently
from
78a76c3 to
d94b710
Compare
|
Just a couple of drive-by notes. I can't comment on the lowering-specific concerns, but overall changes look quite good to me! Thanks for getting the ball rolling here @mlechu
Depending on how @c42f would like to review / merge, it may make sense to set your |
topolarity
reviewed
Jul 2, 2025
mlechu
force-pushed
the
expr-to-st
branch
4 times, most recently
from
81af71e to
e8dcfc7
Compare
mlechu
force-pushed
the
expr-to-st
branch
2 times, most recently
from
12ceddd to
a90555f
Compare
Co-authored-by: Claire Foster <[email protected]>
This used to implicitly refer to a module-level name, but lowering is now expected to wrap it in a `globalref`. Part of JuliaLang/julia#54772
JuliaLang/julia#54773, JuliaLang/julia#56713, JuliaLang/julia#57470. Some changes omitted from `expand-decls` and `expand-assignment`. Note that the two-argument IR "const" is K"constdecl", whereas the one-argument K"const" only appears in the AST. Also note that the `wrap` parameter is omitted throughout assignment desugaring. As far as I'm aware, all this plumbing was just to support `const a,b,c = 1,2,3` having `b` and `c` inherit the `const`. TODO: find a better way of doing the same thing (a ScopedValue might be a clean solution; we currently throw an error). The check for `let; const x = 1; end`, (which should throw) is in scope analysis (lisp has it in `compile`). Co-authored-by: Claire Foster <[email protected]>
JuliaLang/julia#56746. Also call :slot and :static_parameter valid (for now)
Adapt to bpart changes in JuliaLang/julia#54788
JuliaLang/julia#56497; Add self-referencing struct shim I have doubts about how long this solution will stay in the base repository, and how complete it is (doesn't seem to work with M1.M2.S), but we are testing for it here. Also change the expected value of a test changed in the same PR.
For method defs, `latestworld` is produced in desugaring rather than closure conversion for now (our closure conversion doesn't seem to cover the same cases as lisp lowering yet). Covers JuliaLang/julia#56523, JuliaLang/julia#56509, JuliaLang/julia#57299. Also includes changes from JuliaLang/julia#57102 (bpart: Start enforcing minimum world age for const bparts) and JuliaLang/julia#57150 (bpart: Start enforcing min_world for global variable definitions) since the lowering changes from those appear to be amendments to the changes above (missing world age increments). Co-authored-by: Claire Foster <[email protected]>
`Core._typebody!` now takes a new "prev" argument, which we don't use yet here. Changes from JuliaLang/julia#57253
Changes from JuliaLang/julia#57253 (bpart: Fully switch to partitioned semantics). This fixes one failing test and realigns struct desugaring to match lisp for now. Also changed: the expected result of redefining a primitive type (now allowed).
Fix segfaulting test. Thanks for the TODO
Signature changed in JuliaLang/julia#57230. Thanks @aviatesk for the help!
As of JuliaLang/julia#57765, `jl_module_public` is no longer exported. Change our runtime to handle it like `public` and `export` like we handle `import` or `using` for now
I believe this was a world age issue
Too many to count.
Latest julia works. Changes are needed to work with the latest JuliaSyntax, but
that isn't in base julia yet, and more changes are likely to come.
The change lifted the scope of `note`, so it was being changed in the loop
Ping me if you'd like this squashed into the original const/global commit! Co-authored-by: Claire Foster <[email protected]>
Co-authored-by: Claire Foster <[email protected]>
No longer needed since we no longer put `global` or `local` forms back into the
expand_forms machine. Some error messages change slightly as a result.
Thanks! Co-authored-by: Cody Tapscott <[email protected]>
Doesn't handle hygiene yet, but necessary for it.
TODO: We cannot handle intentional un-hygiene from the user with the current
data structures, as far as I'm aware.
|
Closed so I can extract parts of this and create branches in the non-forked repository. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As discussed, this conversion is a bit ugly, but necessary for backwards compatibility with existing macros.
Provenance is determined by using the latest seen LineNumberNode in the Expr traversal. I don't think the line number is well-defined for all subexpressions in general, but this method should work for the obvious cases (i.e. using a linenode that immediately precedes the current expr, or its parent if none, etc.)
This PR also updates JuliaLowering to use the latest JuliaSyntax (JuliaLang/JuliaSyntax.jl@c029996), where dotops are handled differently, generated function IR tests needed updating, and we needed some tweaks for JuliaLang/JuliaSyntax.jl#560. Separating this change would have required some churn.
TODO
syntaxtree(Expr(syntaxtree(string))) ~= syntaxtree(string)tests adapted from JuliaSyntax's expr.jl, though many of these are due to overly strict~=(e.g. Expr does not preserve infix calls orvar"")