Describe the bug
When using produce_or_load with a parameter dictionary that produces a long savename, the save silently fails on Windows with a misleading error:
SystemError: opening file "C:\...\knapsack_maxit=10_method=dd_n_features=4_...jld2": No such file or directory
This error gives no indication of the actual cause. The directory exists and is already populated with results from previous runs that had shorter filenames. The failure occurs because the generated file path exceeds Windows' MAX_PATH limit of 260 characters. Luckily an LLM was able to identify the error for me, otherwise I would have been stuck for a long time trying to figure it out.
Minimal Working Example
using DrWatson
@quickactivate
params = Dict(
"very_long_parameter_name_1" => "value_1",
"very_long_parameter_name_2" => "value_2",
"very_long_parameter_name_3" => true,
"very_long_parameter_name_4" => false,
"very_long_parameter_name_5" => 100,
"very_long_parameter_name_6" => 10,
"very_long_parameter_name_7" => 20,
"very_long_parameter_name_8" => 50,
"very_long_parameter_name_9" => 1234,
"very_long_parameter_name_10" => 12345,
)
# On Windows with a deep base path (e.g. OneDrive), this can exceed 260 characters
path = datadir("simulations", "my_experiment_name", savename(params, "jld2"))
println("Path length: ", length(path))
println(path)
# This will throw "SystemError: No such file or directory" if length > 260
produce_or_load(p -> Dict("result" => 42), params, datadir("simulations", "my_experiment_name"))
Describe the bug
When using produce_or_load with a parameter dictionary that produces a long savename, the save silently fails on Windows with a misleading error:
This error gives no indication of the actual cause. The directory exists and is already populated with results from previous runs that had shorter filenames. The failure occurs because the generated file path exceeds Windows' MAX_PATH limit of 260 characters. Luckily an LLM was able to identify the error for me, otherwise I would have been stuck for a long time trying to figure it out.
Minimal Working Example