-
Notifications
You must be signed in to change notification settings - Fork 64
Add support for ingesting Confluence. #57
Copy link
Copy link
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Describe the issue
A user should be able to ingest documents from confluence by passing the following payload to the /ingest API endpoint:
{
"index_name": "string",
"vector_database": {
"type": "qdrant",
"config": {}
},
"encoder": {
"dimensions": 384,
"model_name": "embed-multilingual-light-v3.0",
"provider": "cohere"
},
"confluence": {
"api_token": <api_token>,
"user_email": < user_email >,
"url": <url>,
},
"webhook_url": "string"
}Ingestion
After load process has finished we need to loop over the downloaded documents and use the embedding service to chunk, split, generate and save embeddings to the embedding db. The Embedding Service and walkthroughs should be sufficient for this.
Relevant files
Route ingestion on request: https://github.com/superagent-ai/super-rag/blob/main/api/ingest.py#L23
Create a new handler: https://github.com/superagent-ai/super-rag/blob/main/service/ingest.py
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request