Skip to content

Conversation

digital-carver
Copy link

As discussed elsewhere, this adds a Warning message to let Windows users that installation failure is a known issue, and links them to the descriptive list of underlying packages.

@digital-carver digital-carver changed the title Add warning about Windows installation failure Add warning about Windows installation failure to README Feb 11, 2025
@drizk1
Copy link
Member

drizk1 commented Feb 13, 2025

@digital-carver

looks like they just bumped the julia version - i havent been able to downloaded it yet because im not sure its on the registry quite yet... but when it does, can you try to see if you still experience the build issue on a windows machine (im on a mac)?

if you dont, i can change the tidierdb compats to only included 1.2

@digital-carver
Copy link
Author

Unfortunately, the issue remains the same.

Error with DuckDB v1.2.0
(@tidy) pkg> add DuckDB
   Resolving package versions...
   Installed DuckDB_jll ─ v1.2.0+0
   Installed DuckDB ───── v1.2.0
  Downloaded artifact: DuckDB
    Updating `C:\Users\Sundar\.julia\environments\tidy\Project.toml`
  [d2f5444f] + DuckDB v1.2.0
    Updating `C:\Users\Sundar\.julia\environments\tidy\Manifest.toml`
  [c3b6d118] + BitIntegers v0.3.2
  [a10d1c49] + DBInterface v2.6.1
  [d2f5444f] + DuckDB v1.2.0
⌅ [fb4d412d] + FixedPointDecimals v0.5.6
  [ea10d353] + WeakRefStrings v1.4.2
  [2cbbab25] + DuckDB_jll v1.2.0+0
        Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m`
Precompiling project...
  ✗ DuckDB
  180 dependencies successfully precompiled in 252 seconds. 142 already precompiled.
  1 dependency errored.
  For a report of the errors see `julia> err`. To retry use `pkg> precompile`

julia> err
PkgPrecompileError: The following 1 direct dependency failed to precompile:

DuckDB

Failed to precompile DuckDB [d2f5444f-75bc-4fdf-ac35-56f514c445e1] to "C:\\Users\\Sundar\\.julia\\compiled\\v1.11\\DuckDB\\jl_6E0D.tmp".
ERROR: LoadError: could not load symbol "duckdb_vector_size":
The specified procedure could not be found.
Stacktrace:
 [1] duckdb_vector_size()
   @ DuckDB C:\Users\Sundar\.julia\packages\DuckDB\5Gcwk\src\api.jl:1296
 [2] top-level scope
   @ C:\Users\Sundar\.julia\packages\DuckDB\5Gcwk\src\database.jl:101
 [3] include(mod::Module, _path::String)
   @ Base .\Base.jl:557
 [4] include(x::String)
   @ DuckDB C:\Users\Sundar\.julia\packages\DuckDB\5Gcwk\src\DuckDB.jl:1
 [5] top-level scope
   @ C:\Users\Sundar\.julia\packages\DuckDB\5Gcwk\src\DuckDB.jl:29
 [6] include
   @ .\Base.jl:557 [inlined]
 [7] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String},
 concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
   @ Base .\loading.jl:2881
 [8] top-level scope
   @ stdin:6

(The same error happens if I add Tidier too, the output is just longer and messier.)

@drizk1
Copy link
Member

drizk1 commented Mar 6, 2025

@digital-carver From what I've read in the prs, I think this is now solved with the 1.2.1 duckdb_jll released this morning

Was wondering if you could try locally to see if this is the case for you?

@digital-carver
Copy link
Author

Well, my celebration emoji above may have been a bit premature 😕

There's no longer an installation failure, so that part of it is technically solved. Adding Tidier or TidierDB to an environment seems to work fine, no precompilation issues or errors.

After adding them though, I tried a small test to make sure it actually works.

  1. In a temporary environment, I added just Tidier, and ran db = DB.connect(DB.duckdb())
  2. In a different temporary environment, I added just TidierDB, and ran db = TidierDB.connect(TidierDB.duckdb()) (I did this since Tidier depends on an older TidierDB version, so I wanted to make sure the same issue occurs with the current release of TidierDB as well.)

In both cases, the Julia process crashes with an EXCEPTION_ACCESS_VIOLATION message.

Full `EXCEPTION_ACCESS_VIOLATION` when trying `DB.connect(DB.duckdb())` from Tidier julia> using Tidier

julia> db = DB.connect(DB.duckdb())

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ff9b45c71d6 -- RtlFreeHeap at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
in expression starting at none:1
RtlFreeHeap at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
free_base at C:\WINDOWS\System32\ucrtbase.dll (unknown line)
_ZNSt10_HashtableINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4pairIKS5_N6duckdb15ExtensionOptionEESaISA_ENSt8__detail10_Select1stENS8_29CaseInsensitiveStringEqualityENS8_33CaseInsensitiveStringHashFunctionENSC_18_Mod_range_hashingENSC_20_Default_ranged_hashENSC_20_Prime_rehash_policyENSC_17_Hashtable_traitsILb1ELb0ELb1EEEE9_M_rehashEyRKy at C:\Users\Sundar.duckdb\extensions\v1.2.1\windows_amd64_mingw\httpfs.duckdb_extension (unknown line)
.text$_ZN6duckdb8DBConfig18AddExtensionOptionERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES6_NS_11LogicalTypeERKNS_5ValueEPFvRNS_13ClientContextENS_8SetScopeERSA_E at C:\Users\Sundar.duckdb\extensions\v1.2.1\windows_amd64_mingw\httpfs.duckdb_extension (unknown line)
.text$_ZN6duckdbL12LoadInternalERNS_16DatabaseInstanceE at C:\Users\Sundar.duckdb\extensions\v1.2.1\windows_amd64_mingw\httpfs.duckdb_extension (unknown line)
_ZN6duckdb15ExtensionHelper21LoadExternalExtensionERNS_16DatabaseInstanceERNS_10FileSystemERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
_ZNK6duckdb12PhysicalLoad7GetDataERNS_16ExecutionContextERNS_9DataChunkERNS_19OperatorSourceInputE at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
_ZN6duckdb16PipelineExecutor15FetchFromSourceERNS_9DataChunkE at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
_ZN6duckdb16PipelineExecutor7ExecuteEy at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
_ZN6duckdb12PipelineTask11ExecuteTaskENS_17TaskExecutionModeE at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
_ZN6duckdb12ExecutorTask7ExecuteENS_17TaskExecutionModeE at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
_ZN6duckdb8Executor11ExecuteTaskEb at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
_ZN6duckdb13ClientContext19ExecuteTaskInternalERNS_17ClientContextLockERNS_15BaseQueryResultEb at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
_ZN6duckdb18PendingQueryResult11ExecuteTaskEv at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
duckdb_pending_execute_task at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
duckdb_pending_execute_task at C:\Users\Sundar.julia\packages\DuckDB\5Gcwk\src\api.jl:2689 [inlined]
pending_execute_tasks at C:\Users\Sundar.julia\packages\DuckDB\5Gcwk\src\result.jl:624
execute_singlethreaded at C:\Users\Sundar.julia\packages\DuckDB\5Gcwk\src\result.jl:676
execute at C:\Users\Sundar.julia\packages\DuckDB\5Gcwk\src\result.jl:719
execute at C:\Users\Sundar.julia\packages\DuckDB\5Gcwk\src\result.jl:872 [inlined]
execute at C:\Users\Sundar.julia\packages\DBInterface\nQcsk\src\DBInterface.jl:130 [inlined]
#execute#2 at C:\Users\Sundar.julia\packages\DBInterface\nQcsk\src\DBInterface.jl:152 [inlined]
execute at C:\Users\Sundar.julia\packages\DBInterface\nQcsk\src\DBInterface.jl:152 [inlined]
#connect#185 at C:\Users\Sundar.julia\packages\TidierDB\J2fSt\src\TidierDB.jl:430
connect at C:\Users\Sundar.julia\packages\TidierDB\J2fSt\src\TidierDB.jl:422
unknown function (ip: 0000028cc885e6d4)
jl_apply at C:/workdir/src\julia.h:2157 [inlined]
do_call at C:/workdir/src\interpreter.c:126
eval_value at C:/workdir/src\interpreter.c:223
eval_stmt_value at C:/workdir/src\interpreter.c:174 [inlined]
eval_body at C:/workdir/src\interpreter.c:681
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:821
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:943
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:886
ijl_toplevel_eval at C:/workdir/src\toplevel.c:952 [inlined]
ijl_toplevel_eval_in at C:/workdir/src\toplevel.c:994
eval at .\boot.jl:430 [inlined]
eval_user_input at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPL.jl:245
repl_backend_loop at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPL.jl:342
#start_repl_backend#59 at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPL.jl:327
start_repl_backend at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPL.jl:324
#run_repl#72 at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPL.jl:483
run_repl at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPL.jl:469
jfptr_run_repl_10359.1 at C:\Users\Sundar.julia\juliaup\julia-1.11.3+0.x64.w64.mingw32\share\julia\compiled\v1.11\REPL\u0gqU_hz07T.dll (unknown line)
#1150 at .\client.jl:446
jfptr_YY.1150_15093.1 at C:\Users\Sundar.julia\juliaup\julia-1.11.3+0.x64.w64.mingw32\share\julia\compiled\v1.11\REPL\u0gqU_hz07T.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:2157 [inlined]
jl_f__call_latest at C:/workdir/src\builtins.c:875
#invokelatest#2 at .\essentials.jl:1055 [inlined]
invokelatest at .\essentials.jl:1052 [inlined]
run_main_repl at .\client.jl:430
repl_main at .\client.jl:567 [inlined]
_start at .\client.jl:541
jfptr__start_75336.1 at C:\Users\Sundar.julia\juliaup\julia-1.11.3+0.x64.w64.mingw32\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:2157 [inlined]
true_main at C:/workdir/src\jlapi.c:900
jl_repl_entrypoint at C:/workdir/src\jlapi.c:1059
mainCRTStartup at C:/workdir/cli\loader_exe.c:58
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Allocations: 26277164 (Pool: 26274649; Big: 2515); GC: 23

I'm not sure if this is the same issue reoccurring in a different form, or if fixing the original issue exposed this new issue with DuckDB. If there are other tests you want me to run, or some other place to report this issue at, please let me know.

@drizk1
Copy link
Member

drizk1 commented Mar 6, 2025

Hmm this may have to do with the old version of tiderdb in tidier not being changed.

Can you try in a dif env adding the TidierDB#compats (you prob won't be able to have tidier and that coexist because I think it's tagged as .8) branch and making sure that works?

@digital-carver
Copy link
Author

Nope, same issue unfortunately.

Details (@v1.11) pkg> activate --temp Activating new project at `C:\Users\Sundar\AppData\Local\Temp\jl_0vLCEE`

(jl_0vLCEE) pkg> add TidierDB#compats
Cloning git-repo https://github.com/TidierOrg/TidierDB.jl.git
Updating git-repo https://github.com/TidierOrg/TidierDB.jl.git
Resolving package versions...
Updating C:\Users\Sundar\AppData\Local\Temp\jl_0vLCEE\Project.toml
[86993f9b] + TidierDB v0.8.0 https://github.com/TidierOrg/TidierDB.jl.git#compats
Updating C:\Users\Sundar\AppData\Local\Temp\jl_0vLCEE\Manifest.toml
[...trimmed very long manifest output...]
Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use status --outdated -m
Precompiling project...
1 dependency successfully precompiled in 4 seconds. 95 already precompiled.

julia> using TidierDB

julia> db = TidierDB.connect(TidierDB.duckdb())

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x7ff9b45c6a08 -- RtlFreeHeap at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
in expression starting at REPL[4]:1
RtlFreeHeap at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
free_base at C:\WINDOWS\System32\ucrtbase.dll (unknown line)
_ZNSt10_HashtableINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4pairIKS5_N6duckdb15ExtensionOptionEESaISA_ENSt8__detail10_Select1stENS8_29CaseInsensitiveStringEqualityENS8_33CaseInsensitiveStringHashFunctionENSC_18_Mod_range_hashingENSC_20_Default_ranged_hashENSC_20_Prime_rehash_policyENSC_17_Hashtable_traitsILb1ELb0ELb1EEEE9_M_rehashEyRKy at C:\Users\Sundar.duckdb\extensions\v1.2.1\windows_amd64_mingw\httpfs.duckdb_extension (unknown line)
.text$_ZN6duckdb8DBConfig18AddExtensionOptionERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES6_NS_11LogicalTypeERKNS_5ValueEPFvRNS_13ClientContextENS_8SetScopeERSA_E at C:\Users\Sundar.duckdb\extensions\v1.2.1\windows_amd64_mingw\httpfs.duckdb_extension (unknown line)
.text$_ZN6duckdbL12LoadInternalERNS_16DatabaseInstanceE at C:\Users\Sundar.duckdb\extensions\v1.2.1\windows_amd64_mingw\httpfs.duckdb_extension (unknown line)
_ZN6duckdb15ExtensionHelper21LoadExternalExtensionERNS_16DatabaseInstanceERNS_10FileSystemERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
_ZNK6duckdb12PhysicalLoad7GetDataERNS_16ExecutionContextERNS_9DataChunkERNS_19OperatorSourceInputE at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
_ZN6duckdb16PipelineExecutor15FetchFromSourceERNS_9DataChunkE at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
_ZN6duckdb16PipelineExecutor7ExecuteEy at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
_ZN6duckdb12PipelineTask11ExecuteTaskENS_17TaskExecutionModeE at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
_ZN6duckdb12ExecutorTask7ExecuteENS_17TaskExecutionModeE at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
_ZN6duckdb8Executor11ExecuteTaskEb at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
_ZN6duckdb13ClientContext19ExecuteTaskInternalERNS_17ClientContextLockERNS_15BaseQueryResultEb at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
_ZN6duckdb18PendingQueryResult11ExecuteTaskEv at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
duckdb_pending_execute_task at C:\Users\Sundar.julia\artifacts\0a92af888175ef893a42281482aaff9c937f951d\bin\libduckdb.dll (unknown line)
duckdb_pending_execute_task at C:\Users\Sundar.julia\packages\DuckDB\5Gcwk\src\api.jl:2689 [inlined]
pending_execute_tasks at C:\Users\Sundar.julia\packages\DuckDB\5Gcwk\src\result.jl:624
execute_singlethreaded at C:\Users\Sundar.julia\packages\DuckDB\5Gcwk\src\result.jl:676
execute at C:\Users\Sundar.julia\packages\DuckDB\5Gcwk\src\result.jl:719
execute at C:\Users\Sundar.julia\packages\DuckDB\5Gcwk\src\result.jl:872 [inlined]
execute at C:\Users\Sundar.julia\packages\DBInterface\nQcsk\src\DBInterface.jl:130 [inlined]
#execute#2 at C:\Users\Sundar.julia\packages\DBInterface\nQcsk\src\DBInterface.jl:152 [inlined]
execute at C:\Users\Sundar.julia\packages\DBInterface\nQcsk\src\DBInterface.jl:152 [inlined]
#connect#199 at C:\Users\Sundar.julia\packages\TidierDB\x0GbK\src\TidierDB.jl:445
connect at C:\Users\Sundar.julia\packages\TidierDB\x0GbK\src\TidierDB.jl:437
unknown function (ip: 000001fd7cb96cf4)
jl_apply at C:/workdir/src\julia.h:2157 [inlined]
do_call at C:/workdir/src\interpreter.c:126
eval_value at C:/workdir/src\interpreter.c:223
eval_stmt_value at C:/workdir/src\interpreter.c:174 [inlined]
eval_body at C:/workdir/src\interpreter.c:681
jl_interpret_toplevel_thunk at C:/workdir/src\interpreter.c:821
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:943
jl_toplevel_eval_flex at C:/workdir/src\toplevel.c:886
ijl_toplevel_eval at C:/workdir/src\toplevel.c:952 [inlined]
ijl_toplevel_eval_in at C:/workdir/src\toplevel.c:994
eval at .\boot.jl:430 [inlined]
eval_user_input at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPL.jl:245
repl_backend_loop at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPL.jl:342
#start_repl_backend#59 at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPL.jl:327
start_repl_backend at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPL.jl:324
#run_repl#72 at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPL.jl:483
run_repl at C:\workdir\usr\share\julia\stdlib\v1.11\REPL\src\REPL.jl:469
jfptr_run_repl_10359.1 at C:\Users\Sundar.julia\juliaup\julia-1.11.3+0.x64.w64.mingw32\share\julia\compiled\v1.11\REPL\u0gqU_hz07T.dll (unknown line)
#1150 at .\client.jl:446
jfptr_YY.1150_15093.1 at C:\Users\Sundar.julia\juliaup\julia-1.11.3+0.x64.w64.mingw32\share\julia\compiled\v1.11\REPL\u0gqU_hz07T.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:2157 [inlined]
jl_f__call_latest at C:/workdir/src\builtins.c:875
#invokelatest#2 at .\essentials.jl:1055 [inlined]
invokelatest at .\essentials.jl:1052 [inlined]
run_main_repl at .\client.jl:430
repl_main at .\client.jl:567 [inlined]
_start at .\client.jl:541
jfptr__start_75336.1 at C:\Users\Sundar.julia\juliaup\julia-1.11.3+0.x64.w64.mingw32\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:2157 [inlined]
true_main at C:/workdir/src\jlapi.c:900
jl_repl_entrypoint at C:/workdir/src\jlapi.c:1059
mainCRTStartup at C:/workdir/cli\loader_exe.c:58
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Allocations: 9824320 (Pool: 9822506; Big: 1814); GC: 12

@drizk1
Copy link
Member

drizk1 commented Mar 6, 2025

That's unfortunate.

I am wondering if we will have to wait for the update julia package since the compat still has 1.2.0

DBInterface = "2.5"
DuckDB_jll = "1.2.0"
FixedPointDecimals = "0.4, 0.5, 0.6"
Tables = "1.7"
WeakRefStrings = "1.4"
julia = "1.6"

@digital-carver i just tried to add the compats branch locally and it failed. I think for the above reason. Hopefully, when the 1.2.1 package version is released it will work w the binary

(@v1.10) pkg> add TidierDB#compats
    Updating git-repo `https://github.com/TidierOrg/TidierDB.jl`
   Resolving package versions...
ERROR: Unsatisfiable requirements detected for package DuckDB_jll [2cbbab25]:
 DuckDB_jll [2cbbab25] log:
 ├─possible versions are: 0.2.1-1.2.0 or uninstalled
 └─restricted to versions 1.2.1-1 by TidierDB [86993f9b] — no versions left
   └─TidierDB [86993f9b] log:
     ├─possible versions are: 0.8.0 or uninstalled
     └─TidierDB [86993f9b] is fixed to version 0.8.0

@digital-carver
Copy link
Author

wait for the update julia package since the compat still has 1.2.0

By "julia package" here I assume you mean the DuckDB package? It does seem to list the compat as 1.2.0, but according to Julia's version specifier format, that means any 1.x.y version is allowed, right?

Matching that interpretation, in all three environments (the one with Tidier.jl, the one with TidierDB.jl, and the one with TidierDB#compats), the Manifest listing shows:

   [2cbbab25] + DuckDB_jll v1.2.1+0

so it seems 1.2.1 is already being pulled in. Checking why shows:

jl_MbVRdQ) pkg> why DuckDB_jll
  TidierDB → DuckDB → DuckDB_jll

@digital-carver
Copy link
Author

Searching for EXCEPTION_ACCESS_VIOLATION in the duckdb repo took me to duckdb/duckdb#9419 which is a previous instance of this error on Julia 1.8 and 1.9. It seems to have gone away somehow with Julia 1.10, so it was never figured out why exactly it was happening. (Also, the error in that case was happening on an execute call, not a connect call, so it may have been a different issue that just happened to cause the same error.)

@drizk1
Copy link
Member

drizk1 commented Mar 7, 2025

I did mean the DuckDB.jl package above, yes.

The compat bounds always trip me up. Based on the issues I am running into, it would seem that having it as 1.2

When I have

# in TidierDB toml
DuckDB_jll = "1.2.1" 

I cannot install the compats branch at all.

(@v1.10) pkg> add TidierDB#compats
     Cloning git-repo `https://github.com/TidierOrg/TidierDB.jl.git`
    Updating git-repo `https://github.com/TidierOrg/TidierDB.jl.git`
   Resolving package versions...
ERROR: Unsatisfiable requirements detected for package DuckDB_jll [2cbbab25]:
 DuckDB_jll [2cbbab25] log:
 ├─possible versions are: 0.2.1-1.2.0 or uninstalled
 └─restricted to versions 1.2.1-1 by TidierDB [86993f9b] — no versions left
   └─TidierDB [86993f9b] log:
     ├─possible versions are: 0.8.0 or uninstalled
     └─TidierDB [86993f9b] is fixed to version 0.8.0

When I change it to DuckDB_jll = "1.2.1" , it works, but I cant seem to install 1.2.1 jll

# It installs
(@v1.10) pkg> why DuckDB_jll
  DuckDB_jll
  TidierDB → DuckDB → DuckDB_jll
  TidierDB → DuckDB_jll

(@v1.10) pkg> status DuckDB_jll
Status `~/.julia/environments/v1.10/Project.toml`
  [2cbbab25] DuckDB_jll v1.2.0+0

but then i cant seem to install [email protected]

(@v1.10) pkg> add [email protected]
   Resolving package versions...
ERROR: Unsatisfiable requirements detected for package DuckDB_jll [2cbbab25]:
 DuckDB_jll [2cbbab25] log:
 ├─possible versions are: 0.2.1-1.2.0 or uninstalled
 ├─restricted to versions 1.2.0-1 by TidierDB [86993f9b], leaving only versions: 1.2.0
 │ └─TidierDB [86993f9b] log:
 │   ├─possible versions are: 0.8.0 or uninstalled
 │   └─TidierDB [86993f9b] is fixed to version 0.8.0
 └─restricted to versions 1.2.1 by an explicit requirement — no versions left

but is that DuckDB_jll v1.2.0+0 sufficient?

also did the issue you experienced go away when you used julia 1.10 and above?

ALSO: huge thank you to you for working through this with me. I really appreciate it.

@digital-carver
Copy link
Author

also did the issue you experienced go away when you used julia 1.10 and above?

All the above outputs have been from Julia 1.11. It seems that your results are from 1.10, maybe that's somehow be the reason for [email protected] failing to install? As mentioned in my comment above, DuckDB_jll v1.2.1 does get installed for me automatically (on 1.11). It's when I try to use it that the issues arise.

Although, DuckDB_jll itself lists only Julia 1.6 (and above) as its requirement, so maybe it's not Julia's version itself, but one of the other packages in your environment that's restricting DuckDB_jll - could you try installing just TidierDB and [email protected] in a new temporary environment (eg. after activate --temp), and check whether that works? I wish the

  ├─possible versions are: 0.2.1-1.2.0 or uninstalled

message was more informative about why only those are the possible versions!

ALSO: huge thank you to you for working through this with me. I really appreciate it.

I'm glad to be of help. It's the care and effort put into the Tidier ecosystem, by you and others, that keeps me motivated to try and get this working!

@drizk1
Copy link
Member

drizk1 commented Mar 8, 2025

So think I figured out the reason it was getting restricted because I was specifying the version to DuckDB_jll = "1.2.1" on branch i was experimenting with.

Interesting when I installed TidierDB v.6.3 with [email protected] on google collab, DuckDB_jll v1.2.1+0 installed automatically.
The TidierDB v.6.3 only DuckDB compat is DuckDB = "1 - 2" For that version I did not have a _jll compat listed in teh toml.

(@v1.10) pkg>  activate --temp
  Activating new project at `/var/folders/vq/k_j88dsn7b130v0qvtnb3zh80000gn/T/jl_V73JfR`

(jl_V73JfR) pkg> add TidierDB
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
   Installed Zstd_jll ────── v1.5.7+1
   Installed DuckDB_jll ──── v1.2.1+0
   Installed FilePathsBase ─ v0.9.24
   Installed TimeZones ───── v1.21.3
   Installed TidierDB ────── v0.7.1
  Downloaded artifact: DuckDB
  Downloaded artifact: Zstd
    Updating `/private/var/folders/vq/k_j88dsn7b130v0qvtnb3zh80000gn/T/jl_V73JfR/Project.toml`

I was then able to open up a db, so unable to recreate the error you experienced above with Julia 1.10

julia> import TidierDB as DB

julia> db = DB.connect(DB.duckdb());

julia> DB.show_tables(db)
0×6 DataFrame
 Row │ database  schema  name    column_names  column_types  temporary 
     │ String    String  String  Array…        Array…        Bool      
─────┴─────────────────────────────────────────────────────────────────

@drizk1
Copy link
Member

drizk1 commented Mar 8, 2025

Do you think you would be able to try the main branch again? I reverted back to the 1.0 and 1.0 fur duckdb and duckdb_jll AND the docs with UDFs still builds ...

so if the main page gets rid of connection issue you had and the prior non compile issue, then i think its a happy medium to make sure the dust settle and TidierDB does cause install headaches for Tidier users

@digital-carver
Copy link
Author

I was then able to open up a db, so unable to recreate the error you experienced above with Julia 1.10

If this is on Google Colab, that's to be expected isn't it, since Colab uses a Linux OS underneath? This seems to remain a Windows-only issue still.

Do you think you would be able to try the main branch again?

I did add TidierDB#main in a new temp environment and tried the same TidierDB.connect(TidierDB.duckdb()) command. This still makes the Julia process crash and exit, the only difference is that it doesn't show the long EXCEPTION_ACCESS_VIOLATION error message. I then checked the Windows Event Viewer for the log of this crash, and it shows Exception code: 0xc0000374 which seems to be the exception code for STATUS_HEAP_CORRUPTION. So it's still a memory-related error that makes it crash, but with a different exception code.

I reverted back to the 1.0 and 1.0 fur duckdb and duckdb_jll

From my understanding, the

DuckDB = "1.0"
DuckDB_jll = "1.0"

lines would still mean [1.x - 2.0) i.e. it still allows any DuckDB and DuckDB_jll version that's less than 2.0, as we discussed before. And the Manifest output when installing TidierDB#main does show that it's DuckDB_jll 1.2.1+0 that's getting installed even when using the main branch. To restrict it to exactly 1.0 version, I think you'd have to use

DuckDB = "=1.0"
DuckDB_jll = "=1.0"

But instead of doing that and restricting users of all OSs to an older DuckDB, we can maybe ask Windows users to manually restrict DuckDB to an older working version themselves. So, for now, please leave the main branch as it is, allowing any [1.x - 2.0) version of DuckDB, and I'll try manually restricting DuckDB to versions less than 1.20 myself and see what works. (This idea just occurred to me while writing this post, and it's post-3AM here, so I'll try this experiment at some point tomorrow and reply back with the results!)

@drizk1
Copy link
Member

drizk1 commented Mar 9, 2025

Hmmm this is tricky. Thank you for all the details. I discovered this compat rule after the the push to main but this was good to know regardless .

Were you able to load tidier at all ? With
using Tidier and then actually use tidier or was this failing as well.

I'm wondering if this is a local issue for you? Or broader. I'm curious if people who originally experienced the failure are experiencing this new issue as well.

@digital-carver
Copy link
Author

Were you able to load tidier at all ? With
using Tidier and then actually use tidier or was this failing as well.

I'll try this when I'm at my computer again.

I'm wondering if this is a local issue for you? Or broader. I'm curious if people who originally experienced the failure are experiencing this new issue as well.

I was wondering this as well. My Windows setup isn't anything unique or custom, so I can't readily think of a reason this would be specific to me, but it would still be good to know. If I understand https://github.com/TidierOrg/TidierDB.jl/blob/main/.github/workflows/CI.yml correctly, the CI for TidierDB runs only on Ubuntu Linux. If it's feasible to add Windows also as a CI target platform, the runtest action should be informative in this regard.

@drizk1
Copy link
Member

drizk1 commented Mar 10, 2025

If you can run tidier successfully, I think that would definitely make me inclined to keep 1.2.1 with a spot in the docs somewhere to help folks who are having duckdb troubles. (The other library Tulipia I learned the workaround from just moved to 1.2.1 )

That being said, would be happy about the ci test. CIs are very much not my skillset, so if you have the bandwidth or knowledge, would you be able to put in a PR on TidierDB with that change and we can run the test and see ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants