Add K"static_eval" for cfunction/ccall/cglobal

[c42f]

A few special forms have a kind of "deferred top level evaluation" semantics for some of their children:

• @cfunction - the function name and types
• ccall / foreigncall / @ccall - the type arguments and sometimes the function name
• cglobal - the function name
• @generated - the expression defining the generated function stub

For example, in @ccall f()::Int, the Int means "the symbol Int as looked up in global scope in the module", and should fail if Int refers to a local variable. Currently all three of these cases are handled through different mechanisms with varying levels of hygiene inconsistency and ability to warn about access to local variables.

To fix this problem, introduce the new K"deferred_toplevel_eval"K"static_eval" form which wraps an expression and preserves it as a piece of AST in the output (rather than producing IR), but still resolves scope and hygiene.

Use this new form to remove all the special case child-index-dependent handling of these disparate forms from the IR.

Also fixes bugs in Base.@cfunction hygiene where the function name might be resolved to a global symbol in the wrong module.

Also move demo @ccall implementation into JuliaLowering, clean up and fix a few things which were broken and implement the gc_safe option from Base.@ccall. Makes use of deferred_toplevel_eval kind for more precise diagnostics.

Needed for #33 which makes the issues with @cfunction hygiene more obvious

[mlechu]

Linking JuliaLang/julia#57931 as a relevant discussion (though mostly about the shared library argument to ccall) and in case you have opinions to add

[topolarity]

Does this still support get_item(::Type{T}) where T = @ccall foo()::Ref{T} ?

IIRC the ccall lowering is an awkward "local 'static' TypeVars + top-level evaluation" where the local static parameters have to be preserved as TypeVars in the resulting expression

[c42f]

Does this still support get_item(::Type{T}) where T = @ccall foo()::Ref{T} ?

No, good catch! I think I've made it slightly too restrictive right now. Do the exact same rules apply to cglobal and cfunction?

I can easily add :static_parameter as allowed use of bindings, alongside :global. I should rename the kind to ... something. K"static_eval" might be good enough for now (more precise names welcome)

[c42f]

@topolarity wow, the cases where static parameters are allowed are quite limited in the existing runtime's interpretation of ccall parameters because they're checked quite early. I didn't expect this:

julia> get_item(::Type{T}) where T = @ccall foo()::Ref{T}
get_item (generic function with 1 method)

julia> get_item(::Type{T}) where T = @ccall foo()::T
ERROR: TypeError: in ccall method definition, expected Type, got a value of type TypeVar

I've updated things here to allow any use of static parameters because it doesn't seem like lowering can make the decision about which uses of static parameters are valid.

[topolarity]

Do the exact same rules apply to cglobal and cfunction?

I believe so!

Here @cfunction works the same:

julia> plus(x::T) where T = @cfunction($((y)->x+y), Ref{T}, (Ref{T},))
julia> ccall(plus(1).ptr, Ref{Int}, (Ref{Int},), 2)
3
julia> ccall(plus(1 + 3im).ptr, Ref{Complex{Int}}, (Ref{Complex{Int}},), 2 - 2im)
3 + 1im

and it looks like cglobal does too:

julia> find_sym(::Val{symbol}) where symbol = cglobal(symbol)
julia> find_sym(Val{:malloc}())
Ptr{Nothing} @0x00007bcdf5aad650

they're checked quite early

Yeah, that's true. That error message is arguably a defect of the type-checking in method.c (https://github.com/JuliaLang/julia/blob/44fdede182dbc8e1e6f0e40c5af0470060e5063b/src/method.c#L194-L204), but in general we do require that the ccall etc. has a static C ABI at method definition time, even though the underlying Julia types might be dynamic. I think we do that so that we can still successfully codegen the de-specialized method (without libffi and other shenanigans to handle fully-dynamic C ABIs).

That error message should probably be more like this one:

julia> struct Foo{T}; val::T; end
julia> foo(x::T) where T = @cfunction($((y)->[x] .+ y), Foo{T}, (Foo{T},))
ERROR: cfunction method definition: return type doesn't correspond to a C type

[c42f]

I think we do that so that we can still successfully codegen the de-specialized method

Makes sense, that was also my guess after I got over the surprise :-)

That error message should probably be more like this one
ERROR: cfunction method definition: return type doesn't correspond to a C type

From the user's high level point of view they may very well be trying to use ccall with a C type (eg, Cint), but the TypeVar hasn't been resolved by the compiler at that early point in compilation (and may never be known statically if we compile a nospecialize version of the function). So it probably should hedge a bit ... like "return type couldn't be statically resolved to a C type" or something.

[c42f]

Ok, I think this will do. Thanks for the comments!