Lowering API: Allow for switching to JuliaLowering

[mlechu]

This PR along with the associated JuliaLowering PR allow us to experiment
with JuliaLowering as the core lowering implementation, mainly for JETLS
development at the moment (cc @aviatesk, @c42f).

Changes

JuliaLowering works with JuliaLowering.SyntaxTree input instead of Expr. This
change allowsSyntaxTrees out of parsing and into lowering, hopefully without
disturbing existing code expecting Expr. The current design wraps
x::SyntaxTree like Expr(:syntaxtree, x) because otherwise
toplevel_eval_flex wouldn't know what x is, but once we're past the
experimental phase, it would only make sense for the compiler to actually know
about the types it's operating on.

New: Core._lower

It (a julia entrypoint), jl_lower (C entrypoint), jl_fl_lower, and
fl_lower all mirror the existing parsing API which was added in #35243.

Click for pre-PR call graph

After this change:

Like Core._parse, Core._lower returns an svec with the first element as
the expected result, and the remaining elements can hold extra information like
the lowered SyntaxTree with provenance info (I've left the specification of
extra return values for later, when JuliaLowering is more stable and JETLS knows
what it needs, because we may opt to pack provenance into the lowered CodeInfo
object instead). Parsing only uses the svec to return an offset, but it could
actually make use of the the flexibility to return diagnostics in the future.

It's unfortunately not clear what types we can expect to go into and come out of
lowering (I got Expr, LineNumberNode, String, Symbol, and Nothing in testing.)

Remove jl_lower_expr_mod

I realize this isn't being called from C, and Core._lower is a more sensible
entry point from julia. We could probably also remove some uncalled parse and
jl_load functions.

Limitations

These belong to JuliaLowering, but are listed here as known issues for anyone
who wants to try it out.

• (module ...) forms don't go through JuliaLowering yet. Modules are lowered
  to a call to JuliaLowering.eval_module(parentmod, modname, syntaxtree),
  which is currently a stub function that converts to Expr for lowering and
  evaluation, which JuliaLowering can't use.
• Macros work differently. Defining and calling macros should work, but
  invoking normal macros defined pre-JuliaLowering will not.
• Compilation is currently slow (~30 seconds for me).

[oscardssmith]

Should #58055 be merged first?

[mlechu]

I tried converting this to a Core._lower call, which gave me a wide variety of test failures seemingly due to fl_lower being defined after some generated functions are. Fixed by putting the ccall back, but local tests also pass if I keep _lower and move include("flfrontend.jl") earlier in Base_compiler.jl (I don't think it matters, and JuliaLowering doesn't use our generated_body_to_codeinfo anyway).

[mlechu]

Should #58055 be merged first?

Nope, this (and the lowering API refactor) should supersede it

[topolarity]

@JeffBezanson @vtjnash can we get a quick review before merging?

Would be good to have one more set of eyes for the "big picture" direction here

[topolarity]

I'll merge this today if there's no new feedback.

[topolarity]

Thanks @mlechu !