New Core.declare_global and Core.declare_const lowering

[xal-0]

These are the JuliaLowering changes for JuliaLang/julia#58187 and
JuliaLang/julia#58279, as well as a handful of small changes to get us closer to
passing the Julia syntax test suite.

Fixes #28.

Change summary

• const foo() = ... (where const has no effect) is unfortunately present in
  a few packages and is now supported.
• Destructing const is now supported,
  eg const x, (; y, z) = 1, (; y=2, z=3) or const x, y..., z = foo().
• The lowered Expr(:const, ...) form has been removed, and replaced with the
  Core.declare_const builtin.
  • JuliaLowering can now lower the secret one-argument surface
    Expr(:const, x), where x is a symbol or GlobalRef. It would be nice to
    deprecate and remove this eventually.
  • constdecl is now a lowering-only kind.
• The lowered Expr(:globaldecl, ...) and Expr(:global, ...) have been
  removed and replaced with Core.declare_global. This is notable because it
  is an explicit side effect that is hoisted to the top level, like closure
  definitions, while globaldecl and global are evaluated by walking the
  lowered code in jl_method_set_source.
  • The lambda kind now has a toplevel_pure attribute, analogous to the
    toplevel-pure Expr head used in flisp.
  • The surface Expr(:global, x) with x being a symbol or GlobalRef is also
    supported now (like the const version, it should eventually be removed in
    favour of calling Core.declare_global directly, if we make it public).
• I adopted the unused_only lowering-only kind previously discussed in Fix decls in value position without assignment to return nothing #67
  since global no longer survives to linearize_ir.

Limitations and TODOs

• JuliaLowering generally evaluates the entire RHS of a destructuring assignment
  before doing any assignments, but this is restricted to one non-nested LHS
  pattern. For example:

  f() = (1, 2)
  const x, y, z = f()
  # With flisp lowering: defines x=1, y=2 before throwing exception
  # With JuliaLowering:  no constants declared
  
  const x, (y, z) = f()
  # With flisp lowering: defines x=1, y=2 before throwing exception
  # With JuliaLowering:  defines x=1 before throwing exception

  It would be nice to extend this to a general guarantee that if the RHS throws
  or is evaluated and determined not to satisfy the LHS pattern, we don't do any
  of the side effects.

• We emit redundant calls to Core.declare_global. They are benign, because
  they are guaranteed to be emitted in order of increasing strength: weak global
  declaration, then strong but with no declared type, then strong with explicit
  type. It would be nice to collapse these into a single call with the
  strongest effect.

• JuliaSyntax converts f() = ... into a function node with the
  SHORT_FORM_FUNCTION_FLAG flag, but doesn't do this for functions defined
  with the destructuring syntax: f(), g() = 1, 2. This "feature" allows you
  to define methods but gives you no way of accessing the arguments (similarly,
  curly assignment in an LHS pattern doesn't let you use the type variables).
  This shouldn't really be allowed, but depending on what PkgEval says we may
  have to hack it in.

  T{U}, f(x::String)::Vector{DataType}..., v::DataType = [Val{i} for i in 1:10]
  # Defines a constant `T`, a method `f(::String)` that returns a Vector of Val
  # datatypes, and a typed global variable `v`, all at once!  : )

[mlechu]

I think you can also remove 1.12 from the CI configuration files given the significant IR changes. (cc @c42f: @aviatesk is OK with dropping 1.12 support so we can focus on full integration with the compiler.)

[mlechu]

@c42f I remember the extra const-specific parameters being done and undone in #10, but I don't think there's any better way to support (const (= (tuple ...) ...)) that doesn't require a lot of refactoring to desugaring, so I'm good with this for now.

[mlechu]

Most world age increments have been moved to linearization in JuliaLowering; move this if you can

[mlechu]

Realizing I mixed up which file I was commenting on, sorry. I was worried about emitting (globaldecl ...) (latestworld) in desugaring and sending that through the woodchipper (closure conversion)

[xal-0]

JuliaLowering doesn't suffer from JuliaLang/julia#59755 because of a quirk in how toplevel statements are hoisted, so this can be merged as-is. I should get around to documenting when "definition time" side effects happen, when you can observe the result of an import, etc...

[mlechu]

This looks good, and people (including me) want JuliaLowering working on 1.13 again. Thanks for porting this!

[c42f]

No need for the nested @ast here :)

[c42f]

This if isn't required - children of type Nothing will be elided from @ast child list - we're allowed to do this without ambiguity because AST fragments must always be of type SyntaxTree.

[c42f]

"nothing"::K"core" is always removable as is access to other core bindings which we know cannot have side effects - so it's not necessary to wrap this.