move writing to REPL history file to behind a PID lock #45450

KristofferC · 2022-05-25T07:21:55Z

Should fix #37015

KristofferC · 2022-05-25T08:25:44Z

Looks like PID files are not getting deleted on windows.

ERROR: The working directory is dirty.
Output of git status:
On branch refs/pull/45450/merge
Untracked files:
  (use "git add <file>..." to include in what will be committed)
	base/$16NCnul.pid.deleted
	base/$1ARWnul.pid.deleted
	base/$3OHYnul.pid.deleted
	base/$5I78nul.pid.deleted
	base/$5IRKnul.pid.deleted
	base/$5MBOnul.pid.deleted
	base/$5UBGnul.pid.deleted
	base/$74L2nul.pid.deleted
	base/$9AB4nul.pid.deleted
	base/$9AJCnul.pid.deleted
	base/$BS1Anul.pid.deleted
	base/$D2NGnul.pid.deleted
	base/$H2RWnul.pid.deleted
	base/$JC5Ynul.pid.deleted
	base/$LAB4nul.pid.deleted
	base/$PUV4nul.pid.deleted
	base/$RWHAnul.pid.deleted
	base/$XANOnul.pid.deleted
	base/$XI30nul.pid.deleted
	base/$XY3Knul.pid.deleted

Also:

┌ Error: Error in the keymap
│   exception =
│    IOError: open("/dev/null.pid", 194, 292): permission denied (EACCES)
│    Stacktrace:

KristofferC · 2022-05-25T12:26:56Z

I don't get these .deleted files building it locally with cygwin.

vtjnash · 2022-05-25T14:05:42Z

This seems to be a bug in our rename function. Julia has no way to do such a fundamental operation as just renaming a file, and always does other junk with it. So when someone make a pidfile on the illegal file name nul.pid, we make a copy of the empty nul file instead.

KristofferC · 2022-08-31T13:21:11Z

@vtjnash, does this look ok now?

vtjnash · 2022-08-31T14:43:40Z

SGTM, though looks mildly concerning that the new test segfaulted the windows worker and causes temp_cleanup_purge to crash

KristofferC · 2022-08-31T15:39:08Z

Yeah, that's not great...

       From worker 8:	Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
      From worker 8:	Exception: EXCEPTION_ACCESS_VIOLATION at 0x6d37d1ba -- ijl_load_dynamic_library at /cygdrive/c/buildbot/worker/package_win32/build/src\dlload.c:230
      From worker 8:	in expression starting at C:\buildbot\worker-tabularasa\tester_win32\build\share\julia\stdlib\v1.9\REPL\test\repl.jl:1011
      From worker 8:	ijl_load_dynamic_library at /cygdrive/c/buildbot/worker/package_win32/build/src\dlload.c:230
      From worker 8:	jl_get_library_ at /cygdrive/c/buildbot/worker/package_win32/build/src\runtime_ccall.cpp:48
      From worker 8:	jl_get_library_ at /cygdrive/c/buildbot/worker/package_win32/build/src\runtime_ccall.cpp:39 [inlined]
      From worker 8:	ijl_load_and_lookup at /cygdrive/c/buildbot/worker/package_win32/build/src\runtime_ccall.cpp:61
      From worker 8:	jlplt_gethostname_59271.clone_1 at C:\buildbot\worker-tabularasa\tester_win32\build\lib\julia\sys.dll (unknown line)
      From worker 8:	write_pidfile at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\FileWatching\src\pidfile.jl:113
      From worker 8:	mkpidlock#1 at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\FileWatching\src\pidfile.jl:59
      From worker 8:	mkpidlock##kw at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\FileWatching\src\pidfile.jl:51 [inlined]
      From worker 8:	#mkpidlock#7 at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\FileWatching\src\pidfile.jl:80
      From worker 8:	mkpidlock##kw at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\FileWatching\src\pidfile.jl:79 [inlined]
      From worker 8:	#mkpidlock#6 at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\FileWatching\src\pidfile.jl:77 [inlined]
      From worker 8:	mkpidlock##kw at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\FileWatching\src\pidfile.jl:77 [inlined]
      From worker 8:	add_history at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\REPL\src\REPL.jl:629

KristofferC · 2022-09-02T06:09:35Z

@vtjnash, any idea why it would crash in gethostname on win32?

vtjnash · 2022-09-02T16:24:13Z

No, that is very strange. It is supposed to be stack memory there, or the constant string "ws2_32", depending on where it is failing at that line. Or the stack is smashed by that code there (missing a \0 perhaps), and the stacktrace is bogus?

mbauman · 2023-08-10T19:48:22Z

Is the situation here better now that we have similar PID locking for Manifest in Pkg? Cf JuliaLang/Pkg.jl#2793

mbauman · 2023-09-15T18:56:15Z

I guess let's just try CI again with a rebase

KristofferC · 2023-09-15T20:30:29Z

I don't think Pkg is relevant here and I am not sure the pid file code has changed much but still interesting to rebase this.

paciorek · 2025-02-26T23:33:42Z

Is there any energy to come back to this issue and put a fix in place? I'm having frequent REPL history file contamination when having multiple Julia sessions on a Linux cluster with a shared home directory.

StefanKarpinski · 2025-03-21T17:58:03Z

Yes, I think that we should go ahead with this.

mbauman · 2025-03-21T18:15:40Z

Looks like the tests now pass on Windows!? The only failure was an OOM on 32-bit linux, but that was back in February.

TorkelE · 2025-04-25T16:18:40Z

Someone referenced me here as I am getting increasing number of crashes on Julia starts due to REPL problems (not might happen multiple times a day). Happy to provide more info if useful, this is an error message I have gotten when trying to start julia (deleting the REPL history file fixes this, but annoying to have to do so frequently).

:~$ julia
Precompiling JuliaFormatter...
  2 dependencies successfully precompiled in 27 seconds. 9 already precompiled.
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org/
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.11.5 (2025-04-14)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

UndefRefError: access to undefined reference
UndefRefError: access to undefined reference
Unhandled Task ERROR: UndefRefError: access to undefined reference
Stacktrace:
 [1] getproperty
   @ ./Base.jl:49 [inlined]
 [2] backend
   @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:997 [inlined]
 [3] (::REPL.var"#70#71"{REPL.LineEditREPL, Pair{Any, Bool}, Bool, Bool})(io::Any)
   @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:405
 [4] with_repl_linfo(f::Any, repl::REPL.LineEditREPL)
   @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:678
 [5] print_response
   @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:403 [inlined]
 [6] setup_interface(repl::REPL.LineEditREPL, hascolor::Bool, extra_repl_keymap::Any)
   @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:1222
 [7] setup_interface
   @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:1090 [inlined]
 [8] run_frontend(repl::REPL.LineEditREPL, backend::REPL.REPLBackendRef)
   @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:1500
 [9] (::REPL.var"#79#85"{REPL.LineEditREPL, REPL.REPLBackendRef})()
   @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:497

caused by: Invalid history file (/home/loman/.julia/logs/repl_history.jl) format:
If you have a history file left over from an older version of Julia,
try renaming or deleting it.
Invalid character: '\0' at line 148
Stacktrace:
 [1] error(::String, ::String, ::String, ::Int64)
   @ Base ./error.jl:44
 [2] hist_from_file(hp::REPL.REPLHistoryProvider, path::String)
   @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:727
 [3] setup_interface(repl::REPL.LineEditREPL, hascolor::Bool, extra_repl_keymap::Any)
   @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:1219
 [4] setup_interface
   @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:1090 [inlined]
 [5] run_frontend(repl::REPL.LineEditREPL, backend::REPL.REPLBackendRef)
   @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:1500
 [6] (::REPL.var"#79#85"{REPL.LineEditREPL, REPL.REPLBackendRef})()
   @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:497
ERROR: TaskFailedException
Stacktrace:
  [1] try_yieldto(undo::typeof(Base.ensure_rescheduled))
    @ Base ./task.jl:958
  [2] wait()
    @ Base ./task.jl:1022
  [3] wait(c::Base.GenericCondition{ReentrantLock}; first::Bool)
    @ Base ./condition.jl:130
  [4] wait
    @ ./condition.jl:125 [inlined]
  [5] take_buffered(c::Channel{Any})
    @ Base ./channels.jl:493
  [6] take!
    @ ./channels.jl:487 [inlined]
  [7] repl_backend_loop(backend::REPL.REPLBackend, get_module::Function)
    @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:353
  [8] start_repl_backend(backend::REPL.REPLBackend, consumer::Any; get_module::Function)
    @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:343
  [9] run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool, backend::Any)
    @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:500
 [10] run_repl(repl::REPL.AbstractREPL, consumer::Any)
    @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:486
 [11] (::Base.var"#1150#1152"{Bool, Symbol, Bool})(REPL::Module)
    @ Base ./client.jl:446
 [12] #invokelatest#2
    @ ./essentials.jl:1055 [inlined]
 [13] invokelatest
    @ ./essentials.jl:1052 [inlined]
 [14] run_main_repl(interactive::Bool, quiet::Bool, banner::Symbol, history_file::Bool, color_set::Bool)
    @ Base ./client.jl:430
 [15] repl_main
    @ ./client.jl:567 [inlined]
 [16] _start()
    @ Base ./client.jl:541

    nested task error: UndefRefError: access to undefined reference
    Stacktrace:
     [1] getproperty
       @ ./Base.jl:49 [inlined]
     [2] backend
       @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:997 [inlined]
     [3] (::REPL.var"#70#71"{REPL.LineEditREPL, Pair{Any, Bool}, Bool, Bool})(io::Any)
       @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:405
     [4] with_repl_linfo(f::Any, repl::REPL.LineEditREPL)
       @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:678
     [5] print_response
       @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:403 [inlined]
     [6] setup_interface(repl::REPL.LineEditREPL, hascolor::Bool, extra_repl_keymap::Any)
       @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:1222
     [7] setup_interface
       @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:1090 [inlined]
     [8] run_frontend(repl::REPL.LineEditREPL, backend::REPL.REPLBackendRef)
       @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:1500
     [9] (::REPL.var"#79#85"{REPL.LineEditREPL, REPL.REPLBackendRef})()
       @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:497
    
    caused by: Invalid history file (/home/loman/.julia/logs/repl_history.jl) format:
    If you have a history file left over from an older version of Julia,
    try renaming or deleting it.
    Invalid character: '\0' at line 148
    Stacktrace:
     [1] error(::String, ::String, ::String, ::Int64)
       @ Base ./error.jl:44
     [2] hist_from_file(hp::REPL.REPLHistoryProvider, path::String)
       @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:727
     [3] setup_interface(repl::REPL.LineEditREPL, hascolor::Bool, extra_repl_keymap::Any)
       @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:1219
     [4] setup_interface
       @ ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:1090 [inlined]
     [5] run_frontend(repl::REPL.LineEditREPL, backend::REPL.REPLBackendRef)
       @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:1500
     [6] (::REPL.var"#79#85"{REPL.LineEditREPL, REPL.REPLBackendRef})()
       @ REPL ~/.julia/juliaup/julia-1.11.5+0.x64.linux.gnu/share/julia/stdlib/v1.11/REPL/src/REPL.jl:497

Should fix JuliaLang#37015 --------- Co-authored-by: KristofferC <[email protected]> Co-authored-by: Ian Butterworth <[email protected]> Co-authored-by: Matt Bauman <[email protected]>

KristofferC requested a review from vtjnash May 25, 2022 07:21

pfitzseb mentioned this pull request May 25, 2022

Lock history file if possible julia-vscode/julia-vscode#2900

Open

giordano added the REPL Julia's REPL (Read Eval Print Loop) label Aug 31, 2022

DilumAluthge force-pushed the kc/pidlock_repl_history branch from 63efded to dd4b8c4 Compare August 31, 2022 16:32

KristofferC and others added 3 commits September 15, 2023 14:55

move writing to REPL history file to behind a PID lock

bc1c7bb

tweak lock for non files

a5a3f82

trailing whitespace

e8a2c11

mbauman force-pushed the kc/pidlock_repl_history branch from dd4b8c4 to e8a2c11 Compare September 15, 2023 18:55

Merge branch 'master' into kc/pidlock_repl_history

6374135

Merge branch 'master' into kc/pidlock_repl_history

0b205e1

mbauman merged commit 7487b8f into master Apr 25, 2025
6 of 7 checks passed

mbauman deleted the kc/pidlock_repl_history branch April 25, 2025 16:24

Uh oh!

move writing to REPL history file to behind a PID lock #45450

move writing to REPL history file to behind a PID lock #45450

Uh oh!

Conversation

KristofferC commented May 25, 2022

Uh oh!

KristofferC commented May 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KristofferC commented May 25, 2022

Uh oh!

vtjnash commented May 25, 2022

Uh oh!

KristofferC commented Aug 31, 2022

Uh oh!

vtjnash commented Aug 31, 2022

Uh oh!

KristofferC commented Aug 31, 2022

Uh oh!

KristofferC commented Sep 2, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vtjnash commented Sep 2, 2022

Uh oh!

mbauman commented Aug 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mbauman commented Sep 15, 2023

Uh oh!

KristofferC commented Sep 15, 2023

Uh oh!

paciorek commented Feb 26, 2025

Uh oh!

StefanKarpinski commented Mar 21, 2025

Uh oh!

mbauman commented Mar 21, 2025

Uh oh!

TorkelE commented Apr 25, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

KristofferC commented May 25, 2022 •

edited

Loading

KristofferC commented Sep 2, 2022 •

edited

Loading

mbauman commented Aug 10, 2023 •

edited

Loading