Why aren't all of my files being sent? And are they truncated?

[tedmiddleton]

I've been trying to use copilot chat with several medium-size code bases and I've been a bit frustrated with the results. The big thing that I really want from an LLM in neovim is the ability to analyze and answer questions about the overall structure of the code in a project. To that end, what I've been trying to do is something like

$gpt-4o:openai
#files:`**/*.py`
Can you give me an overview of this project and what it does?

I've used this with several projects, and for whatever reason, either copilot chat doesn't actually send all of the files to the LLM, or the LLM chooses to ignore most of the files. Typically the LLM will answer with a fairly superficial answer to my question, and then list its references, which are never more than 6-8 of the files I put in the context (typically 20-30).

Also, when I look at the context, it says that all of the files are "truncated". Is that how they're being sent to the LLM? Or is that just for listing the context?

[deathbeam]

Yes, it doesnt send all the files. Ignore the truncation, that is just for display. It does filtering because context is limited, it can do truncation if the files are too big. What you see in references after is what was actually sent. The filtering is done based on the query and keywords in query and th most relevant files are selected, so if your question is too broad then the filtering wont rly work. See #1015 for more context

[tedmiddleton]

I see - is there some way of turning this filtering off? I’m using the openai context that someone posted here and my own token credits to analyze a codebase and I really don’t want the filtering at all. I’d like it to send exactly what I ask it to send. This, to me, is really the whole point of being able to use the chat on multiple source files.

[tedmiddleton]

Having educated myself a bit about RAGs and token sizes, I can see now that for large codebases, this is sort of a Hard Problem.

Still, there should be some kind of control or at least predictability over what part of the context really does get used - currently it's completely opaque and usually wrong or unhelpful. If it isn't possible to submit the whole context, I'd rather copilot just tell me so I can pare it down rather than waiting several minutes for Claude to churn out an unhelpful response based on an unexpected subset of the context. A better globbing and multi-selection interface would really help a whole lot more than better secret heuristics about trying to guess what I would want Claude to see most.

[tedmiddleton]

Anyways, just some feedback on an otherwise great tool. I'm going to close this discussion.

[jduthi]

Hi Ted,

Take a look at this project, this might be useful for your usecase 😉

https://github.com/Davidyz/VectorCode

It integrates with CopilotC‑Nvim / CopilotChat

[deathbeam]

6ac77aa

added option to enable/disable resource processing, disabled by default. This behaviour is very useful but can also be very confusing obviously, so i made it optional