bpart: Start enforcing min_world for global variable definitions #57150

Keno · 2025-01-24T03:36:26Z

This is the analog of #57102 for global variables. Unlike for consants, there is no automatic global backdate mechanism. The reasoning for this is that global variables can be declared at any time, unlike constants which can only be decalared once their value is available. As a result code patterns using Core.eval to declare globals are rarer and likely incorrect.

The internal function `jl_set_const` is allowed during bootstrap only and ignores world age partition. This would give incorrect results after JuliaLang/julia#57150. Just eval the constant definition directly, which has well defined semantics.

Keno · 2025-01-25T04:57:56Z

Distributed test failure is JuliaLang/Distributed.jl#125.

The internal function `jl_set_const` is allowed during bootstrap only and ignores world age partition. This would give incorrect results after JuliaLang/julia#57150. Just eval the constant definition directly, which has well defined semantics.

DilumAluthge · 2025-01-27T18:36:09Z

Needs #57169 I presume.

DilumAluthge · 2025-01-27T21:39:01Z

@Keno Here's the test failure now:

Error in testset rebinding:
Error During Test at /cache/build/tester-amdci5-13/julialang/julia-master/julia-7f6d503f8b/share/julia/test/testdefs.jl:24
  Got exception outside of a @test
  LoadError: invalid redefinition of constant Rebinding.Foo
  Stacktrace:

This is the analog of #57102 for global variables. Unlike for consants, there is no automatic global backdate mechanism. The reasoning for this is that global variables can be declared at any time, unlike constants which can only be decalared once their value is available. As a result code patterns using `Core.eval` to declare globals are rarer and likely incorrect.

IanButterworth · 2025-01-30T19:08:33Z

I assume this test noise from the docs tests was introduced here

      From worker 6:	WARNING: Detected access to binding `NoDocStrings.##meta#60` in a world prior to its definition world.
      From worker 6:	  Julia 1.12 has introduced more strict world age semantics for global bindings.
      From worker 6:	  !!! This code may malfunction under Revise.
      From worker 6:	  !!! This code will error in future versions of Julia.
      From worker 6:	Hint: Add an appropriate `invokelatest` around the access to this binding.
      From worker 6:	WARNING: Detected access to binding `DocRefTests.##meta#60` in a world prior to its definition world.
      From worker 6:	  Julia 1.12 has introduced more strict world age semantics for global bindings.
      From worker 6:	  !!! This code may malfunction under Revise.
      From worker 6:	  !!! This code will error in future versions of Julia.
      From worker 6:	Hint: Add an appropriate `invokelatest` around the access to this binding.

The internal function `jl_set_const` is allowed during bootstrap only and ignores world age partition. This would give incorrect results after JuliaLang/julia#57150. Just eval the constant definition directly, which has well defined semantics.

For method defs, `latestworld` is produced in desugaring rather than closure conversion for now (our closure conversion doesn't seem to cover the same cases as lisp lowering yet). Covers JuliaLang/julia#56523, JuliaLang/julia#56509, JuliaLang/julia#57299. Also includes changes from JuliaLang/julia#57102 (bpart: Start enforcing minimum world age for const bparts) and JuliaLang/julia#57150 (bpart: Start enforcing min_world for global variable definitions) since the lowering changes from those appear to be amendments to the changes above.

…iaLang#57150) This is the analog of JuliaLang#57102 for global variables. Unlike for consants, there is no automatic global backdate mechanism. The reasoning for this is that global variables can be declared at any time, unlike constants which can only be decalared once their value is available. As a result code patterns using `Core.eval` to declare globals are rarer and likely incorrect.

For method defs, `latestworld` is produced in desugaring rather than closure conversion for now (our closure conversion doesn't seem to cover the same cases as lisp lowering yet). Covers JuliaLang/julia#56523, JuliaLang/julia#56509, JuliaLang/julia#57299. Also includes changes from JuliaLang/julia#57102 (bpart: Start enforcing minimum world age for const bparts) and JuliaLang/julia#57150 (bpart: Start enforcing min_world for global variable definitions) since the lowering changes from those appear to be amendments to the changes above.

For method defs, `latestworld` is produced in desugaring rather than closure conversion for now (our closure conversion doesn't seem to cover the same cases as lisp lowering yet). Covers JuliaLang/julia#56523, JuliaLang/julia#56509, JuliaLang/julia#57299. Also includes changes from JuliaLang/julia#57102 (bpart: Start enforcing minimum world age for const bparts) and JuliaLang/julia#57150 (bpart: Start enforcing min_world for global variable definitions) since the lowering changes from those appear to be amendments to the changes above (missing world age increments). Co-authored-by: Claire Foster <[email protected]>

The internal function `jl_set_const` is allowed during bootstrap only and ignores world age partition. This would give incorrect results after JuliaLang/julia#57150. Just eval the constant definition directly, which has well defined semantics.

@aviatesk

* Update CodeInfo struct and handling Co-authored-by: Claire Foster <[email protected]> * Don't produce raw symbol from globalref This used to implicitly refer to a module-level name, but lowering is now expected to wrap it in a `globalref`. Part of JuliaLang/julia#54772 * Updates to const and global lowering; add K"constdecl"; omit `wrap` JuliaLang/julia#54773, JuliaLang/julia#56713, JuliaLang/julia#57470. Some changes omitted from `expand-decls` and `expand-assignment`. Note that the two-argument IR "const" is K"constdecl", whereas the one-argument K"const" only appears in the AST. Also note that the `wrap` parameter is omitted throughout assignment desugaring. As far as I'm aware, all this plumbing was just to support `const a,b,c = 1,2,3` having `b` and `c` inherit the `const`. TODO: find a better way of doing the same thing (a ScopedValue might be a clean solution; we currently throw an error). The check for `let; const x = 1; end`, (which should throw) is in scope analysis (lisp has it in `compile`). Co-authored-by: Claire Foster <[email protected]> * Add `isdefinedglobal` builtin JuliaLang/julia#54999, JuliaLang/julia#56985 * :global no longer valid_ir_argument; rm `is_defined_nothrow_global` JuliaLang/julia#56746. Also call :slot and :static_parameter valid (for now) * Fix `is_defined_and_owned_global` (Core.Binding changes) Adapt to bpart changes in JuliaLang/julia#54788 * Struct desugaring: "Undo decision to publish incomplete types..." JuliaLang/julia#56497; Add self-referencing struct shim I have doubts about how long this solution will stay in the base repository, and how complete it is (doesn't seem to work with M1.M2.S), but we are testing for it here. Also change the expected value of a test changed in the same PR. * Emit `latestworld` world age increments For method defs, `latestworld` is produced in desugaring rather than closure conversion for now (our closure conversion doesn't seem to cover the same cases as lisp lowering yet). Covers JuliaLang/julia#56523, JuliaLang/julia#56509, JuliaLang/julia#57299. Also includes changes from JuliaLang/julia#57102 (bpart: Start enforcing minimum world age for const bparts) and JuliaLang/julia#57150 (bpart: Start enforcing min_world for global variable definitions) since the lowering changes from those appear to be amendments to the changes above (missing world age increments). Co-authored-by: Claire Foster <[email protected]> * bpart changes: `Core._typebody!` signature `Core._typebody!` now takes a new "prev" argument, which we don't use yet here. Changes from JuliaLang/julia#57253 * bpart changes: struct desugaring Changes from JuliaLang/julia#57253 (bpart: Fully switch to partitioned semantics). This fixes one failing test and realigns struct desugaring to match lisp for now. Also changed: the expected result of redefining a primitive type (now allowed). * Additional argument in `new_opaque_closure` Fix segfaulting test. Thanks for the TODO * Adapt to different `GeneratedFunctionStub` signature Signature changed in JuliaLang/julia#57230. Thanks @aviatesk for the help! * Fix `public` and `export` As of JuliaLang/julia#57765, `jl_module_public` is no longer exported. Change our runtime to handle it like `public` and `export` like we handle `import` or `using` for now * Fix modules.jl test I believe this was a world age issue * Regenerate IR tests Too many to count. * Update README to known-good julia, JuliaSyntax versions Latest julia works. Changes are needed to work with the latest JuliaSyntax, but that isn't in base julia yet, and more changes are likely to come. * Fix small bug from #16 so tests pass The change lifted the scope of `note`, so it was being changed in the loop * Changes from code review: const/global lowering Ping me if you'd like this squashed into the original const/global commit! Co-authored-by: Claire Foster <[email protected]> * Remove a special case No longer needed since we no longer put `global` or `local` forms back into the expand_forms machine. Some error messages change slightly as a result. * Changes from code review Co-authored-by: Claire Foster <[email protected]> * Fix + test for assignment in value but not tail position * Disallow `static_parameter` as `valid_ir_argument` See added comment, and discussion at #10 (comment) Co-authored-by: Claire Foster <[email protected]> * Change printing of `K"latestworld"` Parens are nice, but it wasn't consistent. Also make it a leaf (remaining non-leaves are deleted in the next commit.) * Move most `latestworld`s to linearization From the docs: ``` The following statements raise the current world age: 1. An explicit invocation of Core.@latestworld 2. The start of every top-level statement 3. The start of every REPL prompt 4. Any type or struct definition 5. Any method definition 6. Any constant declaration 7. Any global variable declaration (but not a global variable assignment) 8. Any using, import, export or public statement 9. Certain other macros like eval (depends on the macro implementation) ``` This commit handles each case as follows: ``` 1. = 9 2. I'm not sure this actually happens (or needs to happen, unless we're being defensive? Doing it after each world-changing operation should suffice). But if we need it, this would just be emitting once at the beginning of every lowered output. 3. = 2 4. = 6 5. Emit seeing `method` in linearize 6. Emit seeing `constdecl` in linearize 7. Emit seeing `global` or `globaldecl` in linearize 8. We just defer to `eval`, but should probably go in desugaring later - using/import recently became builtin calls, and I haven't updated JL to use them yet. Base._import_using has an expr-based API that may change, and our importpath destructuring is worth keeping. - export and public (special forms) are handled in toplevel.c 9. Done for us ``` Other quirks: - `JuliaLowering.eval_closure_type` calls eval to assign a const, so we still need to deal with that in closure conversion. - The `include` hack isn't mentioned in the docs, but can stay in desugaring. I'm not certain why we don't do the same for non-macro `eval`. --------- Co-authored-by: Claire Foster <[email protected]>

The internal function `jl_set_const` is allowed during bootstrap only and ignores world age partition. This would give incorrect results after JuliaLang/julia#57150. Just eval the constant definition directly, which has well defined semantics.

Keno force-pushed the kf/globalminworld branch from a06285c to b53c87e Compare January 24, 2025 03:41

Base automatically changed from kf/backdatedconstwarn to master January 25, 2025 02:50

Keno mentioned this pull request Jan 25, 2025

Don't use internal jl_set_const to create constants JuliaLang/Distributed.jl#125

Merged

Keno force-pushed the kf/globalminworld branch from b53c87e to dfa20f4 Compare January 25, 2025 04:57

Keno force-pushed the kf/globalminworld branch from 7f6d503 to 036bc6f Compare January 28, 2025 01:24

Keno force-pushed the kf/globalminworld branch from 036bc6f to e7b83ba Compare January 28, 2025 22:51

Keno merged commit f209eba into master Jan 29, 2025
7 checks passed

Keno deleted the kf/globalminworld branch January 29, 2025 02:08

giordano mentioned this pull request Mar 7, 2025

Code coverage dropped to ~20% after bpart changes #57668

Closed

nickrobinson251 mentioned this pull request Sep 1, 2025

Julia 1.12.0-rc1 error: too many arguments to function 'jl_get_binding_wr' JuliaLang/PackageCompiler.jl#1058

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

bpart: Start enforcing min_world for global variable definitions #57150

bpart: Start enforcing min_world for global variable definitions #57150

Uh oh!

Keno commented Jan 24, 2025

Uh oh!

Keno commented Jan 25, 2025

Uh oh!

DilumAluthge commented Jan 27, 2025

Uh oh!

DilumAluthge commented Jan 27, 2025

Uh oh!

Uh oh!

IanButterworth commented Jan 30, 2025

Uh oh!

Uh oh!

Uh oh!

bpart: Start enforcing min_world for global variable definitions #57150

bpart: Start enforcing min_world for global variable definitions #57150

Uh oh!

Conversation

Keno commented Jan 24, 2025

Uh oh!

Keno commented Jan 25, 2025

Uh oh!

DilumAluthge commented Jan 27, 2025

Uh oh!

DilumAluthge commented Jan 27, 2025

Uh oh!

Uh oh!

IanButterworth commented Jan 30, 2025

Uh oh!

Uh oh!