Folding by custom sexp query

[tlkahn]

I'm considering adding a feature that allows custom folding through a tree-sitter sexp query (validated by the tree-siter-query-builder). With some initial attempts, I couldn't get it working. Would anyone be able to take a look and see if it's possible to implement this feature? (or at least let me know what goes wrong with my approach?)

My steps:

• Appending custom queries by vconcat with range-patterns. Please see my code here
• In Emacs init file, you add the following snippets:

(setq ts-fold-pattern-alist
      '((python-mode . (((argument_list) @name (\#match? @name "\"[^\\n]*\\n+[^\\n]*\""))))))

• Update the tsc--stringify-patterns function of tsc.el to:

(defun tsc--stringify-patterns (patterns)
  "Convert PATTERNS into a query string that can be passed to `tsc--make-query'."
  (cond
   ((stringp patterns) patterns)
   ((sequencep patterns)
    ;; XXX: This is hacky.
    (thread-last (mapconcat (lambda (p) (format "%S" p)) patterns "\n")
      (replace-regexp-in-string (regexp-quote "\\?") "?")
      (replace-regexp-in-string (regexp-quote "\\.") ".")
      (replace-regexp-in-string (regexp-quote "\\n") "\n")
      (replace-regexp-in-string (regexp-quote "\\#") "#")))
   (t (error "Invalid patterns"))))

Outcome:

• Parsing seems to work. If I add (message (format patterns)) before this (line)[https://github.com/emacs-tree-sitter/elisp-tree-sitter/blob/master/core/tsc.el#L242], I will get [(function_definition) @name (class_definition) @name (list) @name (dictionary) @name (comment) @name ((argument_list) @name (#match? @name "[^\n]*\n+[^\n]*")) @name]
• But I got this error: Error: (tsc-query-invalid-node-type "" (11 . 1) 103). It seems tsc is returning invalid nodes, which confused me. The sexp query above works well with tree-siter-query-builder.

Any thoughts?

[jcs090218]

Normally the sexp tokens are hidden in the builder. If I am implementing this feature, I might use these functions to help me get the tokens I need, and avoid regexp as much as possible (depends on what you want to do).

• ts-fold-get-children
• ts-fold-get-children-traverse
• ts-fold-find-children
• ts-fold-find-children-traverse
• ts-fold-find-parent
• tsc-get-next-sibling
• tsc-get-prev-sibling

Once you found the token you need, then just return the fold range using that token. See our documentation https://github.com/emacs-tree-sitter/ts-fold#-writing-new-fold-functions.

Why avoid regex? What is hidden tokens? Read the code review thread #42 (comment).

[tlkahn]

Thanks a lot for the explanation! This is extremely helpful. I will rely on AST walk-through as much as possible.

So here is my use case. I would like to fold function arguments which are multiple lines, but leave the single-line function definition as is. For example, the python functions here, can be marked and folded in some way (similar to VSCode's behavior). Illustrated below:

But without regex support, I don't know how to implement this. I am wondering if there would be a simple and flexible way.

[jcs090218]

It should be pretty simple to do. You need to first declare a starting node, probably argument_list or something similar, then find the end node to make up the folding range. The end node should be the last node in argument_list node. Here is the logic (not the code):

# Once you defined `argument_list` in `ts-fold-parsers.el`, then ts-fold will find it for you!
# 
# The starting node should be `(` token
starting_node = 'argument_list' node

# Get the last children, this should be `)` token
last_child = get_children(starting_node).last  # or find it with sibling, depends on the AST

# Find the pos
start_pos = node_end_pos(starting_node)  # The node is `(`, start fold after this node
end_pos = node_beg_pos(last_child)       # The node is `)`, end fold before this node

# This is the folding range
range = (start_pos, end_pos)

Result should be something similar to:

//       V                  V
void Add(int a, int b, int c)

/* | Will fold into
 * |
 * V
 */
void Add(...)

For range, see function

ts-fold/ts-fold.el

Lines 407 to 413 in 7a46b69

	(defun ts-fold-range-seq (node offset)
	"Return the fold range in sequence starting from NODE.
	Argument OFFSET can be used to tweak the final beginning and end position."
	(let ((beg (1+ (tsc-node-start-position node)))
	(end (1- (tsc-node-end-position node))))
	(ts-fold--cons-add (cons beg end) offset)))

With this method, is likely going to be accurate since it relies on AST and not regex. More importantly, it's fast!

[tlkahn]

Thanks a lot! I ended up with what I want using this line:

(push '(argument_list . ts-fold-range-seq) (alist-get 'python-mode ts-fold-range-alist))

Appreciate the fast help!