How do you actually use branching?

[JonatanSahar]

What I would like to use branching for is a typical scenario where, during a conversation, I need some clarification about an idea or a term, and the process of getting that information sort of derails the conversation and adds a lot of messy context. I would like to be able to branch the conversation at that point, ask my questions, and then get back into the main flow of the conversation.
However, due to the structure of org mode (which is something I've never really understood the reason for), once you open a heading in a certain level, all text after that heading belongs to that heading, so if I cannot continue the conversation directly under That's not very elegant, and also adds some confusing text to the prompt. I can also just give all prompts as headings, like in the default setting - that's maybe the cleanest looking solution?

How do you use it?

[karthink]

How do you use it?

I use it to create conversation branches following a general question or two. In this example, each heading is a parallel conversation branch, and they all follow from the general query about Parquet.

(It's a little difficult to see the prompt text in the HTML export since I didn't use a prompt-prefix in this conversation, but in the Emacs buffer I use gptel-highlight-mode so the distinction between prompts and responses is quite clear.)

I would like to be able to branch the conversation at that point, ask my questions, and then get back into the main flow of the conversation.

Yeah, the hierarchical structure of Org documents cannot be used for this.

Instead, I would run clone-buffer (C-x x n) to create the tangent branch.

I can also just give all prompts as headings, like in the default setting - that's maybe the cleanest looking solution

I know this is gptel's default, but I've never been a fan of this. It doesn't scale well to longer conversations.

[JonatanSahar]

Thanks for the prompt (haha) response!
Are you sure that is the right link? it takes me an org mode doc about parquet.
Can you say more about the use clone-buffer for this purpose?

[karthink]

Yes, the "org mode doc about parquet" is an HTML export of my gptel chat buffer, demonstrating how I use branching responses in gptel. I added each heading to create a branch, asking the LLM about a different aspect of Parquet.

use of clone-buffer for this purpose

clone-buffer will produce an exact copy of the current buffer (presumably your gptel chat buffer), with the same state, major and minor modes, etc. You can ask your "derailing" questions in the cloned buffer without cluttering up the main buffer.

[JonatanSahar]

I see, that's a cool solution, but then I'm losing this added context when later reviewing the conversation (it's a major selling point that I can just save the buffer as an org file into my existing notes directory). I guess you can't have it all.
Currently leanining towards threading a "main" heading.
Such a shame that org doesn't allow heading association by indent or something.

[karthink]

You can also do the following. Starting from an un-branched conversation, where | is the cursor position:

<starting prompt>

<main response>|

You can create your tangent as a heading:

<starting prompt>

<main response>

* Tangent
<prompt for tangential issues>

<response for tangential issues>|

Then you can move back to the main response and continue the conversation there:

<starting prompt>

<main response>

<prompt following up on main response>|

* Tangent
<prompt for tangential issues>

<response for tangential issues>

I do this often. You don't even need branching for this, since anything below the cursor is not sent, but I use it anyway.

[k4r4b3y]

@karthink

In this example, each heading is a parallel conversation branch, and they all follow from the general query about Parquet.

Can you share the raw text file, so that I can take a look at your org mode file structure (headings, their properties drawer, etc.) ?

[kaibagley]

Hey, I'm sure this is obvious to others, but I'm not sure how best to handle the LLM responses when using branching.

In my initial prompt, the LLM will use level 2 org headings (**) on its own to structure its reply. Ill then reply with a branch like this:

ME
What is this thing?

LLM 
** The thing is this?
Foo
** It has this purpose
Bar

ME (tangent)
* Thanks, what about this

LLM
** other thing
** another thing

ME (sub tangent) 
** What do you mean by "other thing"

LLM
** blah (clashing with my sub tangent??)
** blah

My issue is, do i need to tell the LLM to never use org headings? Or are the org headings that the LLM outputs not considered for the branching? Or am I just using this wrong?

Thanks!!!

[karthink]

Yes, branching conversations won't work reliably if the LLM uses org headings.

I use this function to do that:

(add-hook 'gptel-post-response-functions #'my/gptel-remove-headings)

  (defun my/gptel-remove-headings (beg end)
    (when (derived-mode-p 'org-mode)
      (save-excursion
        (goto-char beg)
        (while (re-search-forward org-heading-regexp end t)
          (forward-line 0)
          (delete-char (1+ (length (match-string 1))))
          (insert-and-inherit "*")
          (end-of-line)
          (skip-chars-backward " \t\r")
          (insert-and-inherit "*")))))

Or are the org headings that the LLM outputs not considered for the branching?

No, it's a regular Org document in every way.

[kaibagley]

Amazing, thanks a lot!

[k4r4b3y]

@karthink
I think that hook is good. I was also suffering form LLM responses using the org mode headings, and it messing up with the gptel's "org-mode niceities" (branching, heading-specific settings, etc.) -- and I was scratching my head about how to actually use org-mode niceities.

Removing the org mode headings of LLM in the post answer completion works at a first glance.

[ryanobjc]

I don't actually use branching. I was finding that the AI returning it's own headlines was messing things up. Even if you use a 'higher' headline for the next question, then the previous replies don't become part of the context, and it's kind of confusing!

I instead use gptel-org-set-topic to limit context to top level headlines. I also wrote (or had claude code write for me...) a module to update the headlines returned by AI to not create new top level headlines.

Here is my write up and code: https://github.com/ryanobjc/gptel-org-utils