Centralized @Codebase and @Docs Indexing for Dev Team

[jm-warren]

Hello, I am currently looking into setting up Continue for a small dev team, using our own locally served open source models to power it. Our codebase is massive and complex with many layers. It also is built using several very large and mature frameworks that have tons of documentation. Based on my research, for best results for codebase and docs indexing, we will want to be careful/specific with how we are creating embeddings. Ideally, I would like to create the embeddings for codebase and various docs sources from a single machine, and then push the results to the devs. This way, the entire team will have consistency for the embeddings across our codebase and docs. The end goal would be to have the centralized embeddings regenerated on some schedule to stay updated, and automatically pushed out to devs local setups.

My question is about whether this is a valid approach -- assuming the dev machines all have the same YAML config for docs sources etc, does Continue allow the embedding DBs to be simply copied and dropped in to another machines Continue installation/data? In addition, can we turn off auto re-indexing so that dev machines do not overwrite the centralized embeddings DB that they receive?

[bdougie]

This sounds like you could use a hosted or cloud version of Lance or Chroma to do this. But also think this fits well within the continue dev data model. I think your approach is valid and entirely up to how you'd like to maintain it.

https://docs.continue.dev/customize/deep-dives/development-data

I also saw we have a guide on custom indexing scripts which is a more practical implementation

https://docs.continue.dev/guides/custom-code-rag#step-4%3A-how-to-put-together-an-indexing-script