Multiple sessions with different context files

[cyplas]

Is it possible/easy to have different sets of context files in different sessions? If I understand correctly, gptel keeps track of the context files globally. but i would like to be able to have different context files associated with different gptel buffers, one for my django project, one for my org files, etc. Or is the right way to deal with this currently to create my own commands which use gptel-request accordingly? Any examples out there?

(btw: thank you, @karthink, for this wonderful package!)

[karthink]

I would like this feature too, but when you think about it carefully it's not clear how to implement this in a non-confusing way. See #475, #481 and possibly other threads (search for "context"), and continue the discussion there. If you can help address the many undefined behaviors of this feature in a simple way we can implement it.

[albcorp]

Thanks for the very careful and deliberate work on context in gptel. My use case relies on per-session context that is assembled by a function. I wrap gptel-send in a function, and assemble the context before each request is sent.

For per session context that arises in conversation, media links are perfect. For per session context that does not change per request, a buffer-local gptel-context variable is perfect. For per session context that is computed, something hook-like is required?

What are the best extension points to investigate in gptel for this approach?

My implementation is broken by 0.9.9.4, because it leans hard on the file local variables AND sets gptel--system-message to a long string. With the new gptel--save-state logic, the local variables section overflows the 3000 character limit of Emacs. So, I need to re-factor my implementation to use presets. This seems like an opportunity to try and more closely follow the extension hooks that gptel provides. Possibly gptel-prompt-transform-functions?

[karthink]

Buffer-local or scoped context sets are now possible in gptel, see #475 (comment), the NEWS file, or the documentation of the newly added variable gptel-context for details.

[stefanv]

Buffer-local or scoped context sets are now possible in gptel, see #475 (comment), the NEWS file, or the documentation of the newly added variable gptel-context for details.

I was wondering if it might make sense to add a function that queries for a gptel buffer name, turns that buffer's context local, and then adds the current buffer to it. I can see it working with buffers and files, but I can't figure out how it would work with region context, especially overlapping regions.

This is the type of thing I had in mind for files:

(defun my/gptel-context-get-key (item)
  "Extract the buffer or filename from a gptel context ITEM."
  (if (consp item) (car item) item))

(defun my/gptel-add-context-to ()
  "Make target gptel-context local, merge global context, and add current file."
  (interactive)
  (let* ((source-buf (current-buffer))
         ;; gptel buffers
         (gptel-bufs (cl-remove-if-not (lambda (b) (buffer-local-value 'gptel-mode b))
                                      (buffer-list)))
         (target-name (completing-read "Target gptel buffer: "
                                       (mapcar #'buffer-name gptel-bufs) nil t))
         ;; ask user for one of these
         (target-buf (get-buffer target-name)))

    ;; did we get a gptel buffer?
    (unless (and target-buf (buffer-local-value 'gptel-mode target-buf))
      (user-error "Buffer %s is not a gptel buffer" target-name))

    ;; grab a copy of the global context
    (let ((global-ctx (default-value 'gptel-context)))
      (with-current-buffer target-buf
        ;; Make the gptel window's gptel-context buffer-local
        (make-local-variable 'gptel-context)

        ;; combine the global with the local context, in case local context was already customized
        (let ((combined (copy-sequence gptel-context)))
          (dolist (g-item global-ctx)
            (let ((g-key (my/gptel-context-get-key g-item)))
              (unless (cl-member g-key combined :test
                                 (lambda (k item) (equal k (my/gptel-context-get-key item))))
                (push g-item combined))))
          (setq gptel-context combined))

        ;; add current file to the local context
        (let ((source-key (or (buffer-file-name source-buf) source-buf)))
          (unless (cl-member source-key gptel-context :test
                             (lambda (k item) (equal k (my/gptel-context-get-key item))))
            (push source-key gptel-context)))))

    (message "Added %s to %s context." (buffer-name source-buf) target-name)))

Sorry, this is crude, just trying to get a sense of whether it is worth pursuing.

[karthink]

For per session context that arises in conversation, media links are perfect. For per session context that does not change per request, a buffer-local gptel-context variable is perfect. For per session context that is computed, something hook-like is required?

I'm not sure what "per-session context that is computed" means. Does it mean that the context sources (files, buffers etc) are computed dynamically once, when you start the session, or computed dynamically before every call to gptel-send? I'm assuming it's the latter.

What are the best extension points to investigate in *gptel* for this approach?

gptel-prompt-transform-functions is intended for this use case. It is a pre-request hook that you can use to modify any aspect of the payload, including the context. You can add to gptel-context dynamically using this hook (but keep in mind that it runs in a temporary copy of your current buffer). Buf if you have already wrapped gptel-send in a function, why not assemble the dynamic context there? This is roughly equivalent to using a prompt-transform.

My implementation is broken by 0.9.9.4, because it leans hard on the file local variables AND sets gptel--system-message to a long string.

I don't understand why 0.9.9.4 specifically breaks your implementation -- all versions of gptel should. I also don't understand what the problem is besides long gptel--system-message strings. (Since you used "AND".)

With the new gptel--save-state logic, the local variables section overflows the 3000 character limit of Emacs.

Again, not sure how the new logic is making things worse. It reduces (conditionally) the length of the metadata written to the file. Why wasn't the local variables section overflowing the 3000 chars limit before 0.9.9.4?

So, I need to re-factor my implementation to use presets. This seems like an opportunity to try and more closely follow the extension hooks that *gptel* provides. Possibly gptel-prompt-transform-functions?

Creating a preset with your system message and applying it should remove the system message string from the local variables section, relieving most of the 3000 chars pressure. It can also remove other fields, anything that's handled by the preset. But now your document depends on the preset being defined in your Emacs session to restore the chat state, just as it does the backend. (The correct solution here is to move the metadata from a local variables block to YAML frontmatter. This will bring it in line with gptel's support for Org mode, which has no character limit on metadata. I haven't been able to do this due to YAML parser dependency issues.)

…

Message ID: ***@***.*** com>

[albcorp]

Thanks. I will look at gptel-prompt-transform-functions and moving the metadata from a local variables
block to YAML front matter.

The break between 0.9.9.3 and 0.9.9.4 is reliable and easy to show. I just roll backwards and forwards between versions. The AND is: I store stuff in file local variables AND now it is also storing the system message, which is long.

The cause is the change in gptel--save-state from:

          (unless (equal (default-value 'gptel--system-message)
                           gptel--system-message)
            (add-file-local-variable
             'gptel--system-message     ;TODO: Handle nil case correctly
             (car-safe (gptel--parse-directive gptel--system-message))))

to

          (let ((parsed (car-safe (gptel--parse-directive gptel--system-message))))
            (if (gptel--preset-mismatch-value preset-spec :system parsed)
                (add-file-local-variable 'gptel--system-message parsed)
              (delete-file-local-variable 'gptel--system-message)))

because I don't make gptel--system-message buffer local. Instead, I set it with a function before each request. So, critically, default-value is always the same as the current value, since as per the documentation

This is the value that is seen in buffers that do not have their own values
for this variable. The default value is meaningful for variables with
local bindings in certain buffers.

[karthink]

Thanks. I will look at gptel-prompt-transform-functions and moving the

metadata from a local variables

block to YAML front matter.

To be clear, gptel has no support for metadata as YAML front matter yet.

The break between 0.9.9.3 and 0.9.9.4 is reliable and easy to show. I

just roll backwards and forwards between versions. The AND is: I store stuff in file local variables AND now it is also storing the system message, which is long.

The cause is the change in gptel--save-state from...

Ah, I see. That makes sense. A while ago there was a discussion about whether the system message should _always_ be saved to the file, even when it's the global value and not specific to the chat buffer. I can't find it now because GitHub's issue search is not surfacing it. The outcome of the discussion was that gptel should do this to make the file more self-contained (what if you change the global/default system message and open the chat file?) This was implemented for Org a long time ago but I missed the Markdown path until I refactored gptel--save-state in 0.9.9.4.

because I don't make gptel--system-message buffer local. Instead, I set

it with a function before each request. So, critically, default-value is always the same as the current value, since as per the documentation Your workflow (changing the system message before each request) was a casualty of this change. If you are setting the system message dynamically for each request you have a couple of options. 1. Set gptel--system-message to a function (of no arguments) that returns the right system message when called. This function is called when you Whatever logic you use to assign the next system message can live inside this function. 2. OR, use a prompt transform function that sets the system message from the context at the time of the request. You just have to remember to set it buffer-locally in the buffer where the prompt-transform-functions hook is called. The two are equivalent, but method 1 is more declarative.

…

— Reply to this email directly, view it on GitHub <#548 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACBVOLE7TZP7WBUVI6R62IT4N57VRAVCNFSM6AAAAACHU23E7KVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTKOJUGAZTQNY> . You are receiving this because you were mentioned.Message ID: ***@***.***>