[migration] Nvim 0.9 Support

[Jint-lzxy]

Feature description

Nvim v0.9.0 was released yesterday. This issue will be used to collect & discuss mandatory changes as well as new features during migration.

Git branch for migration: 0.9.

Additional information

• MANDATORY CHANGES

  • Update install scripts and issue templates to support v0.9. <==> feat(scripts): support v0.9 #630
  • Consider dropping v0.7 support as it doesn't include version control. <==> feat(scripts): support v0.9 #630
  • Update hl_to_rgb to use builtin nvim_get_hl instead. <==> feat(utils): use neovim builtin to get highlights #632
  • Remove legacy options (:h news) <==> feat(options): support new options shipped with v0.9 #631
  • Update all FUNC_API_DEPRECATED_SINCE(9) API functions.
  • ! Refactor Treesitter highlights <==> refactor(catppuccin)!: reimplement basic hl groups #676 * --> See 0.10
    * Support for Treesitter highlights is currently a mess. The plan is to reimplement basic hl groups and link Treesitter's highlights to these basic hl groups to reduce maintenance pressure.

• NEW FEATURES

  • Support LSP semantic tokens for catppuccin. <==> feat!: add initial support for lsp semantic tokens #633
  • Support new options <==> feat(options): support new options shipped with v0.9 #631

Linked issues

• nvim-treesitter intergration requires neovim 0.8 #568 - wontfix
• Lost file history #612
• Cannot use sudo write #617
• ray-x/cmp-treesitter/pull/10 cause completion system to broke #624

[Jint-lzxy]

Check before merging

Sometimes nvim crashes during exit (:q) (SIGABRT), but it cannot be reliably reproduced. This would need to be tested on more platforms.

However, this issue only occurs after most plugins have been registered & loaded (e.g., a few seconds after opening a file for editing). Exiting from the dashboard will not lead to this issue, and nvim --clean doesn't lead to this issue either.

Backtrace points to nvim trying to close a handle that is either already closed or in a closing state. Blindly guessing, this should be related to a specific plugin (a pipeline created was released too early):

assert(!uv__is_closing(handle)); // evaluates to assert(!true)

Current workaround:

diff --git a/src/nvim/event/stream.c b/src/nvim/event/stream.c
index 0a49186..9150873 100644
--- a/src/nvim/event/stream.c
+++ b/src/nvim/event/stream.c
@@ -139,7 +139,9 @@ void stream_close_handle(Stream *stream)
            (void *)stream,
            uv_stream_get_write_queue_size(stream->uvstream));
     }
-    uv_close((uv_handle_t *)stream->uvstream, close_cb);
+    if (!uv_is_closing((uv_handle_t *)stream->uvstream)) {
+      uv_close((uv_handle_t *)stream->uvstream, close_cb);
+    }
   } else {
     uv_close((uv_handle_t *)&stream->uv.idle, close_cb);
   }

Full stack backtrace (debug):

Process:               nvim [719]
Path:                  /Users/USER/Desktop/*/nvim
Code Type:             X86-64 (Native)
Parent Process:        Exited process [718]
Responsible:           iTerm2 [502]
User ID:               501

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	    0x7ff808b321f2 __pthread_kill + 10
1   libsystem_pthread.dylib       	    0x7ff808b69ee6 pthread_kill + 263
2   libsystem_c.dylib             	    0x7ff808a90b45 abort + 123
3   libsystem_c.dylib             	    0x7ff808a8fe5e __assert_rtn + 314
4   nvim                          	       0x1038a3f74 uv_close + 84 (core.c:117)
5   nvim                          	       0x10350b2bd stream_close_handle + 157 (stream.c:142)
6   nvim                          	       0x10350bd21 write_cb + 209 (wstream.c:145)
7   nvim                          	       0x1038b7e8e uv__write_callbacks + 542 (stream.c:927)
8   nvim                          	       0x1038b7c10 uv__stream_destroy + 336 (stream.c:456)
9   nvim                          	       0x1038a7745 uv__finish_close + 293 (core.c:304)
10  nvim                          	       0x1038a48aa uv__run_closing_handles + 74 (core.c:334)
11  nvim                          	       0x1038a4644 uv_run + 404 (core.c:425)
12  nvim                          	       0x103506117 loop_close + 215 (loop.c:166)
13  nvim                          	       0x1033c6b09 event_teardown + 137 (main.c:170)
14  nvim                          	       0x1033cad83 os_exit + 67 (main.c:662)
15  nvim                          	       0x1033ca7d5 getout + 1493 (main.c:799)
16  nvim                          	       0x103529368 ex_exit + 360 (ex_docmd.c:4877)
17  nvim                          	       0x103521f7e execute_cmd0 + 894 (ex_docmd.c:1620)
18  nvim                          	       0x10351dbf9 do_one_cmd + 4921 (ex_docmd.c:2279)
19  nvim                          	       0x10351b697 do_cmdline + 2887 (ex_docmd.c:578)
20  nvim                          	       0x10361372d nv_colon + 349 (normal.c:3228)
21  nvim                          	       0x1036111a8 normal_execute + 2136 (normal.c:1196)
22  nvim                          	       0x10370d37a state_enter + 538 (state.c:99)
23  nvim                          	       0x10360d185 normal_enter + 165 (normal.c:497)
24  nvim                          	       0x1033c7d1d main + 4045 (main.c:641)
25  dyld                          	    0x7ff80881041f start + 1903

Binary Images:
       0x1033c4000 -        0x103947fff nvim (*) <d547b91c-293a-3321-a609-7c36d55e050b> /Users/USER/Desktop/*/nvim
       0x103b7c000 -        0x103b87fff libintl.8.dylib (*) <146b350b-3118-3f81-aab0-b6792d3d810d> /usr/local/Cellar/gettext/0.21.1/lib/libintl.8.dylib
       0x1043cd000 -        0x1043d8fff lua.so (*) <f3f2e975-b3d2-3058-ad51-d238961a1b1d> /Users/USER/*/lua.so
       0x103d5b000 -        0x103d5efff libfzy-darwin-x86_64.so (*) <e0be7cbb-ce8f-3b19-883f-541e85282aaf> /Users/USER/*/libfzy-darwin-x86_64.so
    0x7ff808b2a000 -     0x7ff808b63ff7 libsystem_kernel.dylib (*) <08606a44-7008-3658-9f00-6c250b80e9c3> /usr/lib/system/libsystem_kernel.dylib
    0x7ff808b64000 -     0x7ff808b6ffff libsystem_pthread.dylib (*) <86dfa543-95fa-36b4-83c6-bf03d01b2aad> /usr/lib/system/libsystem_pthread.dylib
    0x7ff808a11000 -     0x7ff808a98ff7 libsystem_c.dylib (*) <0773ddbc-707e-3b56-ad3e-97aaa9b2c3ed> /usr/lib/system/libsystem_c.dylib
    0x7ff80880a000 -     0x7ff8088a25a7 dyld (*) <afa3518c-143e-3060-bbe8-624d4ca41063> /usr/lib/dyld
    0x7ff808bc9000 -     0x7ff809065fef com.apple.CoreFoundation (6.9) <315a3f65-0954-3635-96dc-2f65c691d074> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation 

cc @ayamir @CharlesChiuGit

[ayamir]

Maybe we can checkout a 0.8 branch before merge changes for 0.9.

[Jint-lzxy]

Maybe we can checkout a 0.8 branch before merge changes for 0.9.

tree#0.8

[ayamir]

Check before merging

Sometimes nvim crashes during exit (:q) (SIGABRT), but it cannot be reliably reproduced. This would need to be tested on more platforms.

However, this issue only occurs after most plugins have been registered & loaded (e.g., a few seconds after opening a file for editing). Exiting from the dashboard will not lead to this issue, and nvim --clean doesn't lead to this issue either.

Backtrace points to nvim trying to close a handle that is either already closed or in a closing state. Blindly guessing, this should be related to a specific plugin (a pipeline created was released too early):

assert(!uv__is_closing(handle)); // evaluates to assert(!true)

Current workaround:

diff --git a/src/nvim/event/stream.c b/src/nvim/event/stream.c
index 0a49186..9150873 100644
--- a/src/nvim/event/stream.c
+++ b/src/nvim/event/stream.c
@@ -139,7 +139,9 @@ void stream_close_handle(Stream *stream)
            (void *)stream,
            uv_stream_get_write_queue_size(stream->uvstream));
     }
-    uv_close((uv_handle_t *)stream->uvstream, close_cb);
+    if (!uv_is_closing((uv_handle_t *)stream->uvstream)) {
+      uv_close((uv_handle_t *)stream->uvstream, close_cb);
+    }
   } else {
     uv_close((uv_handle_t *)&stream->uv.idle, close_cb);
   }

Full stack backtrace (debug):

Process:               nvim [719]
Path:                  /Users/USER/Desktop/*/nvim
Code Type:             X86-64 (Native)
Parent Process:        Exited process [718]
Responsible:           iTerm2 [502]
User ID:               501

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	    0x7ff808b321f2 __pthread_kill + 10
1   libsystem_pthread.dylib       	    0x7ff808b69ee6 pthread_kill + 263
2   libsystem_c.dylib             	    0x7ff808a90b45 abort + 123
3   libsystem_c.dylib             	    0x7ff808a8fe5e __assert_rtn + 314
4   nvim                          	       0x1038a3f74 uv_close + 84 (core.c:117)
5   nvim                          	       0x10350b2bd stream_close_handle + 157 (stream.c:142)
6   nvim                          	       0x10350bd21 write_cb + 209 (wstream.c:145)
7   nvim                          	       0x1038b7e8e uv__write_callbacks + 542 (stream.c:927)
8   nvim                          	       0x1038b7c10 uv__stream_destroy + 336 (stream.c:456)
9   nvim                          	       0x1038a7745 uv__finish_close + 293 (core.c:304)
10  nvim                          	       0x1038a48aa uv__run_closing_handles + 74 (core.c:334)
11  nvim                          	       0x1038a4644 uv_run + 404 (core.c:425)
12  nvim                          	       0x103506117 loop_close + 215 (loop.c:166)
13  nvim                          	       0x1033c6b09 event_teardown + 137 (main.c:170)
14  nvim                          	       0x1033cad83 os_exit + 67 (main.c:662)
15  nvim                          	       0x1033ca7d5 getout + 1493 (main.c:799)
16  nvim                          	       0x103529368 ex_exit + 360 (ex_docmd.c:4877)
17  nvim                          	       0x103521f7e execute_cmd0 + 894 (ex_docmd.c:1620)
18  nvim                          	       0x10351dbf9 do_one_cmd + 4921 (ex_docmd.c:2279)
19  nvim                          	       0x10351b697 do_cmdline + 2887 (ex_docmd.c:578)
20  nvim                          	       0x10361372d nv_colon + 349 (normal.c:3228)
21  nvim                          	       0x1036111a8 normal_execute + 2136 (normal.c:1196)
22  nvim                          	       0x10370d37a state_enter + 538 (state.c:99)
23  nvim                          	       0x10360d185 normal_enter + 165 (normal.c:497)
24  nvim                          	       0x1033c7d1d main + 4045 (main.c:641)
25  dyld                          	    0x7ff80881041f start + 1903

Binary Images:
       0x1033c4000 -        0x103947fff nvim (*) <d547b91c-293a-3321-a609-7c36d55e050b> /Users/USER/Desktop/*/nvim
       0x103b7c000 -        0x103b87fff libintl.8.dylib (*) <146b350b-3118-3f81-aab0-b6792d3d810d> /usr/local/Cellar/gettext/0.21.1/lib/libintl.8.dylib
       0x1043cd000 -        0x1043d8fff lua.so (*) <f3f2e975-b3d2-3058-ad51-d238961a1b1d> /Users/USER/*/lua.so
       0x103d5b000 -        0x103d5efff libfzy-darwin-x86_64.so (*) <e0be7cbb-ce8f-3b19-883f-541e85282aaf> /Users/USER/*/libfzy-darwin-x86_64.so
    0x7ff808b2a000 -     0x7ff808b63ff7 libsystem_kernel.dylib (*) <08606a44-7008-3658-9f00-6c250b80e9c3> /usr/lib/system/libsystem_kernel.dylib
    0x7ff808b64000 -     0x7ff808b6ffff libsystem_pthread.dylib (*) <86dfa543-95fa-36b4-83c6-bf03d01b2aad> /usr/lib/system/libsystem_pthread.dylib
    0x7ff808a11000 -     0x7ff808a98ff7 libsystem_c.dylib (*) <0773ddbc-707e-3b56-ad3e-97aaa9b2c3ed> /usr/lib/system/libsystem_c.dylib
    0x7ff80880a000 -     0x7ff8088a25a7 dyld (*) <afa3518c-143e-3060-bbe8-624d4ca41063> /usr/lib/dyld
    0x7ff808bc9000 -     0x7ff809065fef com.apple.CoreFoundation (6.9) <315a3f65-0954-3635-96dc-2f65c691d074> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation 

cc @ayamir @CharlesChiuGit

I'm waiting for the 0.9 version pr of neovim merged. So it will be tested later.

[ayamir]

Add suda.vim is accpetable to close #617 but whether it is necessary.

[charliie-dev]

Check before merging

Sometimes nvim crashes during exit (:q) (SIGABRT), but it cannot be reliably reproduced. This would need to be tested on more platforms.

However, this issue only occurs after most plugins have been registered & loaded (e.g., a few seconds after opening a file for editing). Exiting from the dashboard will not lead to this issue, and nvim --clean doesn't lead to this issue either.

Backtrace points to nvim trying to close a handle that is either already closed or in a closing state. Blindly guessing, this should be related to a specific plugin (a pipeline created was released too early):

assert(!uv__is_closing(handle)); // evaluates to assert(!true)

Current workaround:

diff --git a/src/nvim/event/stream.c b/src/nvim/event/stream.c
index 0a49186..9150873 100644
--- a/src/nvim/event/stream.c
+++ b/src/nvim/event/stream.c
@@ -139,7 +139,9 @@ void stream_close_handle(Stream *stream)
            (void *)stream,
            uv_stream_get_write_queue_size(stream->uvstream));
     }
-    uv_close((uv_handle_t *)stream->uvstream, close_cb);
+    if (!uv_is_closing((uv_handle_t *)stream->uvstream)) {
+      uv_close((uv_handle_t *)stream->uvstream, close_cb);
+    }
   } else {
     uv_close((uv_handle_t *)&stream->uv.idle, close_cb);
   }

Full stack backtrace (debug):

Process:               nvim [719]
Path:                  /Users/USER/Desktop/*/nvim
Code Type:             X86-64 (Native)
Parent Process:        Exited process [718]
Responsible:           iTerm2 [502]
User ID:               501

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	    0x7ff808b321f2 __pthread_kill + 10
1   libsystem_pthread.dylib       	    0x7ff808b69ee6 pthread_kill + 263
2   libsystem_c.dylib             	    0x7ff808a90b45 abort + 123
3   libsystem_c.dylib             	    0x7ff808a8fe5e __assert_rtn + 314
4   nvim                          	       0x1038a3f74 uv_close + 84 (core.c:117)
5   nvim                          	       0x10350b2bd stream_close_handle + 157 (stream.c:142)
6   nvim                          	       0x10350bd21 write_cb + 209 (wstream.c:145)
7   nvim                          	       0x1038b7e8e uv__write_callbacks + 542 (stream.c:927)
8   nvim                          	       0x1038b7c10 uv__stream_destroy + 336 (stream.c:456)
9   nvim                          	       0x1038a7745 uv__finish_close + 293 (core.c:304)
10  nvim                          	       0x1038a48aa uv__run_closing_handles + 74 (core.c:334)
11  nvim                          	       0x1038a4644 uv_run + 404 (core.c:425)
12  nvim                          	       0x103506117 loop_close + 215 (loop.c:166)
13  nvim                          	       0x1033c6b09 event_teardown + 137 (main.c:170)
14  nvim                          	       0x1033cad83 os_exit + 67 (main.c:662)
15  nvim                          	       0x1033ca7d5 getout + 1493 (main.c:799)
16  nvim                          	       0x103529368 ex_exit + 360 (ex_docmd.c:4877)
17  nvim                          	       0x103521f7e execute_cmd0 + 894 (ex_docmd.c:1620)
18  nvim                          	       0x10351dbf9 do_one_cmd + 4921 (ex_docmd.c:2279)
19  nvim                          	       0x10351b697 do_cmdline + 2887 (ex_docmd.c:578)
20  nvim                          	       0x10361372d nv_colon + 349 (normal.c:3228)
21  nvim                          	       0x1036111a8 normal_execute + 2136 (normal.c:1196)
22  nvim                          	       0x10370d37a state_enter + 538 (state.c:99)
23  nvim                          	       0x10360d185 normal_enter + 165 (normal.c:497)
24  nvim                          	       0x1033c7d1d main + 4045 (main.c:641)
25  dyld                          	    0x7ff80881041f start + 1903

Binary Images:
       0x1033c4000 -        0x103947fff nvim (*) <d547b91c-293a-3321-a609-7c36d55e050b> /Users/USER/Desktop/*/nvim
       0x103b7c000 -        0x103b87fff libintl.8.dylib (*) <146b350b-3118-3f81-aab0-b6792d3d810d> /usr/local/Cellar/gettext/0.21.1/lib/libintl.8.dylib
       0x1043cd000 -        0x1043d8fff lua.so (*) <f3f2e975-b3d2-3058-ad51-d238961a1b1d> /Users/USER/*/lua.so
       0x103d5b000 -        0x103d5efff libfzy-darwin-x86_64.so (*) <e0be7cbb-ce8f-3b19-883f-541e85282aaf> /Users/USER/*/libfzy-darwin-x86_64.so
    0x7ff808b2a000 -     0x7ff808b63ff7 libsystem_kernel.dylib (*) <08606a44-7008-3658-9f00-6c250b80e9c3> /usr/lib/system/libsystem_kernel.dylib
    0x7ff808b64000 -     0x7ff808b6ffff libsystem_pthread.dylib (*) <86dfa543-95fa-36b4-83c6-bf03d01b2aad> /usr/lib/system/libsystem_pthread.dylib
    0x7ff808a11000 -     0x7ff808a98ff7 libsystem_c.dylib (*) <0773ddbc-707e-3b56-ad3e-97aaa9b2c3ed> /usr/lib/system/libsystem_c.dylib
    0x7ff80880a000 -     0x7ff8088a25a7 dyld (*) <afa3518c-143e-3060-bbe8-624d4ca41063> /usr/lib/dyld
    0x7ff808bc9000 -     0x7ff809065fef com.apple.CoreFoundation (6.9) <315a3f65-0954-3635-96dc-2f65c691d074> /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation 

cc @ayamir @CharlesChiuGit

This is interesting, I never ran into this when I drove 0.9-dev on void, arch and ubuntu. I'll test the non-dev version again.
Is this error happens on macos?

[Jint-lzxy]

Is this error happens on macos?

Yep.

This is interesting, I never ran into this when I drove 0.9-dev on void, arch and ubuntu. I'll test the non-dev version again.

After attaching a debugger to nvim (about to crash), looks like sometimes uv_close is called by an autocmd (a uv_handle_t*) registered by a plugin (whose information can't be obtained as we're directly debugging an executable) that will trigger at VimLeavePre. Current observation indicates that only this callback would lead to a crash when calling external shell commands. The following:

vim.api.nvim_create_autocmd("VimLeavePre", {
	callback = function()
		vim.api.nvim_command([[silent! !echo 'aMessage']])
	end,
})

can reliably reproduce this crash on macOS. idk whether this behavior is the same on other platforms 😄

[Jint-lzxy]

Add suda.vim is accpetable to close #617 but whether it is necessary.

no problem with this, as we just need to load it on demand. A possible implementation:

{scope}["lambdalisue/suda.vim"] = {
	lazy = true,
	cmd = { "SudaRead", "SudaWrite" }
}

[charliie-dev]

nvimdev/lspsaga.nvim@298009c

    keys = {
      exec_action = 'o',
      exec_action = 'o',
      quit = 'q',
      quit = 'q',
      ~~ go_action = 'g', ~~
      expand_or_jump = '<CR>',
      expand_or_jump = '<CR>',
      quit_in_show = { 'q', '<ESC>' },
      quit_in_show = { 'q', '<ESC>' },
    },

go_action is now removed, @Jint-lzxy r u waiting for the new release?

Also I think starting from now, we need to send PRs to 0.9 branch?

[charliie-dev]

can reliably reproduce this crash on macOS. idk whether this behavior is the same on other platforms 😄

This doesn't seem to happen on Win11 with 0.9 (non-dev) version.

[ayamir]

This doesn't seem to happen on Win11 with 0.9 (non-dev) version.

Don't appear on Gentoo too.

[Jint-lzxy]

idk whether this behavior is the same on other platforms

Looks like this is an upstream issue: neovim/neovim#21856.

Also I think starting from now, we need to send PRs to 0.9 branch?

In fact, all previous PRs were sent to the 0.9 branch 😄

go_action is now removed

Oh well spotted! I somehow didn't notice this change

EDIT: Fixed at ac3122d.