Precompilation takes forever

[xgdgsc]

I guess it' s because you use

gRPCClient.jl/src/gRPCClient.jl

Line 117 in ad3fd46

TEST_HOST = "172.238.177.88"

which my machine cannot access. Might just remove precompilation and let user packages do it by themself. Or because of something similar to https://discourse.julialang.org/t/precompiletools-hangs/134397.

[csvance]

I expected this could be an issue in air gapped environments and potentially other countries beyond what I mentioned with corporate firewalls.

If possible I would like for precompile to work since I went through so much effort to make the core code not require additional specialization.

What is curious is we should be getting an exception if we cannot connect to the server. I will look into trying to reproduce this and make it fail gracefully, allowing precompile to finish even if it cannot connect.

[csvance]

I was able to reproduce this by doing sudo ufw deny out from any to 172.238.177.88 and then running precompile. Working on a fix now.

[csvance]

@xgdgsc I'm adding several options which resolve this for you (and obviously fixing the connection hang bug):

• Set the GRPC_PRECOMPILE_DISABLE environment variable to disable gRPCClient.jl precompiliation
• Run your own instance of the Go gRPC test server in public mode ./grpc_test_server -public and set GRPC_PRECOMPILE_SERVER_HOST and GRPC_PRECOMPILE_SERVER_PORT environment variables to point to it
• Allow precompile to gracefully timeout after 10 seconds

Let me know if this resolves the issue for you.

[xgdgsc]

Or pack the server with something like https://github.com/AtelierArith/RustToolChains.jl ?

[xgdgsc]

It doesn' t timeout after 10 seconds. Setting GRPC_PRECOMPILE_DISABLE works.

[csvance]

@xgdgsc what version of Julia / OS? At least on Linux + Julia 1.12.2 the timeout works for me.

[xgdgsc]

Ubuntu 24.04+Julia 1.12.3

[xgdgsc]

Now this fails with err:

PkgPrecompileError: The following 1 direct dependency failed to precompile:

gRPCClient 

Failed to precompile gRPCClient [aaca4a50-36af-4a1d-b878-4c443f2061ad] to "/home/user1/.julia/compiled/v1.12/gRPCClient/jl_cp2ggU".
ERROR: LoadError: FieldError: type InvalidStateException has no field `code`, available fields: `msg`, `state`
Stacktrace:
  [1] getproperty(x::InvalidStateException, f::Symbol)
    @ Base ./Base_compiler.jl:54
  [2] macro expansion
    @ ~/.julia/packages/gRPCClient/QDgtc/src/gRPCClient.jl:194 [inlined]
  [3] macro expansion
    @ ~/.julia/packages/PrecompileTools/gn08A/src/workloads.jl:73 [inlined]
  [4] macro expansion
    @ ~/.julia/packages/gRPCClient/QDgtc/src/gRPCClient.jl:136 [inlined]
  [5] macro expansion
    @ ~/.julia/packages/PrecompileTools/gn08A/src/workloads.jl:121 [inlined]
  [6] top-level scope
    @ ~/.julia/packages/gRPCClient/QDgtc/src/gRPCClient.jl:115
  [7] include(mod::Module, _path::String)
    @ Base ./Base.jl:306
  [8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2996
  [9] top-level scope
    @ stdin:5
 [10] eval(m::Module, e::Any)
    @ Core ./boot.jl:489
 [11] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
    @ Base ./loading.jl:2842
 [12] include_string
    @ ./loading.jl:2852 [inlined]
 [13] exec_options(opts::Base.JLOptions)
    @ Base ./client.jl:315
 [14] _start()
    @ Base ./client.jl:550
in expression starting at /home/user1/.julia/packages/gRPCClient/QDgtc/src/gRPCClient.jl:1
in expression starting at stdin:5

caused by: InvalidStateException: Channel is closed.
Stacktrace:
  [1] try_yieldto(undo::typeof(Base.ensure_rescheduled))
    @ Base ./task.jl:1128
  [2] wait()
    @ Base ./task.jl:1200
  [3] wait(c::Base.GenericCondition{ReentrantLock}; first::Bool)
    @ Base ./condition.jl:141
  [4] wait
    @ ./condition.jl:136 [inlined]
  [5] take_buffered(c::Channel{gRPCClient.TestResponse})
    @ Base ./channels.jl:532
  [6] take!(c::Channel{gRPCClient.TestResponse})
    @ Base ./channels.jl:526
  [7] macro expansion
    @ ~/.julia/packages/gRPCClient/QDgtc/src/gRPCClient.jl:179 [inlined]
  [8] macro expansion
    @ ~/.julia/packages/PrecompileTools/gn08A/src/workloads.jl:73 [inlined]
  [9] macro expansion
    @ ~/.julia/packages/gRPCClient/QDgtc/src/gRPCClient.jl:136 [inlined]
 [10] macro expansion
    @ ~/.julia/packages/PrecompileTools/gn08A/src/workloads.jl:121 [inlined]
 [11] top-level scope
    @ ~/.julia/packages/gRPCClient/QDgtc/src/gRPCClient.jl:115
 [12] include(mod::Module, _path::String)
    @ Base ./Base.jl:306
 [13] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:2996
 [14] top-level scope
    @ stdin:5
 [15] eval(m::Module, e::Any)
    @ Core ./boot.jl:489
 [16] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
    @ Base ./loading.jl:2842
 [17] include_string
    @ ./loading.jl:2852 [inlined]
 [18] exec_options(opts::Base.JLOptions)
    @ Base ./client.jl:315
 [19] _start()
    @ Base ./client.jl:550

at least not hanging.

[csvance]

@xgdgsc should be fixed now, sorry about that. You should get this message when it times out now:

            @warn """
            DEADLINE_EXCEEDED during gRPCClient.jl precompile

            A dedicated server for allowing this package to precompile is provided to the public but is not accessible from this network.
            If you care about this you can consider the following options:
            - Request that your network administrator allow connections to this address on TCP: $(TEST_HOST):$(TEST_PORT)
            - Add a precompile block to your own package which calls an accessible gRPC server
            - Run your own instance of the Go gRPC test server in public mode `./grpc_test_server -public` and set GRPC_PRECOMPILE_SERVER_HOST and GRPC_PRECOMPILE_SERVER_PORT environment variables to point to it
            - Set the GRPC_PRECOMPILE_DISABLE environment variable to disable gRPCClient.jl precompiliation and this message
            """

[xgdgsc]

I get

Failed to precompile gRPCClient [aaca4a50-36af-4a1d-b878-4c443f2061ad] to "/Users/gaoshichao/.julia/compiled/v1.12/gRPCClient/jl_OQyvWn".
ERROR: LoadError: InvalidStateException: Channel is closed.
Stacktrace:
  [1] try_yieldto(undo::typeof(Base.ensure_rescheduled))
    @ Base ./task.jl:1157
  [2] wait()
    @ Base ./task.jl:1229
  [3] wait(c::Base.GenericCondition{ReentrantLock}; first::Bool)
    @ Base ./condition.jl:141
  [4] wait
    @ ./condition.jl:136 [inlined]
  [5] take_buffered(c::Channel{gRPCClient.TestResponse})
    @ Base ./channels.jl:532
  [6] take!(c::Channel{gRPCClient.TestResponse})
    @ Base ./channels.jl:526
  [7] macro expansion
    @ ~/.julia/packages/gRPCClient/qij37/src/gRPCClient.jl:179 [inlined]
  [8] macro expansion
    @ ~/.julia/packages/PrecompileTools/gn08A/src/workloads.jl:73 [inlined]
  [9] macro expansion
    @ ~/.julia/packages/gRPCClient/qij37/src/gRPCClient.jl:136 [inlined]
 [10] macro expansion
    @ ~/.julia/packages/PrecompileTools/gn08A/src/workloads.jl:121 [inlined]
 [11] top-level scope
    @ ~/.julia/packages/gRPCClient/qij37/src/gRPCClient.jl:115
 [12] include(mod::Module, _path::String)
    @ Base ./Base.jl:306
 [13] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:3024
 [14] top-level scope
    @ stdin:5
 [15] eval(m::Module, e::Any)
    @ Core ./boot.jl:489
 [16] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
    @ Base ./loading.jl:2870
 [17] include_string
    @ ./loading.jl:2880 [inlined]
 [18] exec_options(opts::Base.JLOptions)
    @ Base ./client.jl:315
 [19] _start()
    @ Base ./client.jl:550
in expression starting at /Users/gaoshichao/.julia/packages/gRPCClient/qij37/src/gRPCClient.jl:1
in expression starting at stdin: