tags(5) generators based on tree-sitter

[masatake]

I have spent a long time improving ctags.
However, I am also interested in activating the community developing tags(5) based software.

I found about two tree-sitter based tags generators.

• https://github.com/npezza93/ttags
• https://github.com/jha-naman/treetags

I sent a first pull request to treetags.

[masatake]

[yamato@dev64]~/var/ttags% ./target/release/ttags src/*.rs
[yamato@dev64]~/var/ttags% less tags 
[yamato@dev64]~/var/ttags% ./target/release/ttags src/*.rs
[yamato@dev64]~/var/ttags% mv tags ttags.tags
[yamato@dev64]~/var/ttags% ~/var/treetags/target/release/treetags src/*.rs
[yamato@dev64]~/var/ttags% mv tags treetags.tags
-rw-r--r--. 1 yamato yamato 7684 Oct 17 09:21 treetags.tags
-rw-r--r--. 1 yamato yamato 2165 Oct 17 09:20 ttags.tags

[masatake]

$ find ~/var/linux | xargs cat > /dev/null
$ time ./target/release/ttags ~/var/linux
./target/release/ttags ~/var/linux  388.27s user 7.90s system 93% cpu 7:04.96 total
[yamato@dev64]~/var/ttags% time ~/var/treetags/target/release/treetags -f treetags.tags --workers 16 ~/var/linux           
~/var/treetags/target/release/treetags -f treetags.tags --workers 16   584.86s user 18.07s system 687% cpu 1:27.75 total
~/var/ctags-github/ctags --options=NONE -o ctags.tags -R ~/var/linux  85.27s user 11.32s system 88% cpu 1:48.55 total
[yamato@dev64]~/var/ttags% ls -lh *.tags
-rw-r--r--. 1 yamato yamato 1.5G Oct 17 09:46 ctags.tags
-rw-r--r--. 1 yamato yamato 1.7G Oct 17 09:39 treetags.tags
-rw-r--r--. 1 yamato yamato  78M Oct 17 09:32 ttags.tags
[yamato@dev64]~/var/ttags% time ~/var/treetags/target/release/treetags --sort=no -f treetags-no-sowrt.tags --workers 16 ~/var/linux                              
~/var/treetags/target/release/treetags --sort=no -f treetags-no-sowrt.tags  1  571.17s user 17.87s system 982% cpu 59.968 total

Monitoring the output of the top command, treetags uses multiple CPUs aggressively.
Using external sort looks good idea for treetags.

I also tried ptags that runs multiple ctags processes in parallel. However, I got an error.

[yamato@dev64]~/var/ttags% ~/var/ptags/target/debug/ptags -f ptags.tags -t 1 --bin-ctags /home/yamato/bin/ctags -c --options=NONE  -- ~/var/linux  
failed to call ctags
failed to call ctags
EPERM: Operation not permitted
[yamato@dev64]~/var/ttags% strace -f -e execve ~/var/ptags/target/debug/ptags -f ptags.tags -t 1 --bin-ctags /home/yamato/bin/ctags -c --options=NONE  -- ~/var/linux  
execve("/home/yamato/var/ptags/target/debug/ptags", ["/home/yamato/var/ptags/target/de"..., "-f", "ptags.tags", "-t", "1", "--bin-ctags", "/home/yamato/bin/ctags", "-c", "--options=NONE", "--", "/home/yamato/var/linux"], 0x7ffc304b3868 /* 76 vars */) = 0
strace: Process 2150098 attached
[pid 2150098] execve("/home/yamato/.cargo/bin/git", ["git", "ls-files", "--cached", "--exclude-standard"], 0x7fffd24ebca8 /* 76 vars */) = -1 ENOENT (No such file or directory)
[pid 2150098] execve("/home/yamato/perl5/bin/git", ["git", "ls-files", "--cached", "--exclude-standard"], 0x7fffd24ebca8 /* 76 vars */) = -1 ENOENT (No such file or directory)
[pid 2150098] execve("/usr/share/Modules/bin/git", ["git", "ls-files", "--cached", "--exclude-standard"], 0x7fffd24ebca8 /* 76 vars */) = -1 ENOENT (No such file or directory)
[pid 2150098] execve("/usr/local/sbin/git", ["git", "ls-files", "--cached", "--exclude-standard"], 0x7fffd24ebca8 /* 76 vars */) = -1 ENOENT (No such file or directory)
[pid 2150098] execve("/usr/local/bin/git", ["git", "ls-files", "--cached", "--exclude-standard"], 0x7fffd24ebca8 /* 76 vars */) = -1 ENOENT (No such file or directory)
[pid 2150098] execve("/usr/sbin/git", ["git", "ls-files", "--cached", "--exclude-standard"], 0x7fffd24ebca8 /* 76 vars */) = -1 ENOENT (No such file or directory)
[pid 2150098] execve("/usr/bin/git", ["git", "ls-files", "--cached", "--exclude-standard"], 0x7fffd24ebca8 /* 76 vars */) = 0
[pid 2150098] +++ exited with 0 +++
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=2150098, si_uid=1000, si_status=0, si_utime=2 /* 0.02 s */, si_stime=1 /* 0.01 s */} ---
strace: Process 2150101 attached
strace: Process 2150102 attached
[pid 2150102] execve("/home/yamato/bin/ctags", ["/home/yamato/bin/ctags", "-L -", "-f -", "--options=NONE"], 0x7fffd24ebca8 /* 76 vars */) = 0
failed to call ctags
failed to call ctags
EPERM: Operation not permitted
[pid 2150101] +++ exited with 0 +++
[pid 2150097] +++ exited with 0 +++
+++ exited with 1 +++

This is the first time I run ptags. I will inspect more.

[masatake]

[yamato@dev64]~/var/ttags%  ./target/release/ttags --help                                                                                                          
ttags 0.4.3
Nick Pezza <pezza@hey.com>

USAGE:
    ttags [FLAGS] [OPTIONS] <files>...
    ttags [FLAGS] [OPTIONS] <SUBCOMMAND>

FLAGS:
    -a, --append      Append tags to existing file
    -h, --help        Prints help information
    -r, --relative    Should paths be relative to cwd? By default, it's relative to tag-file
    -V, --version     Prints version information

OPTIONS:
    -f, --tag-file <FILE|->    File to write tags to. Use '-' to output to stdout [default: ./tags]

ARGS:
    <files>...    Specify files to parse for tags

SUBCOMMANDS:
    help    Prints this message or the help of the given subcommand(s)
    lsp     Creates lsp server

The lsp subcommand looks interesting.

I also find lsp using ctags as a backend.

• https://github.com/netmute/ctags-lsp
• https://github.com/yowu/ctags_ls

[masatake]

Other interesting items

• https://github.com/gladiatr72/mcp-ctags
• https://github.com/osinmv/function-lookup-mcp