A RAG is a system that allows us to feed specific information such as documents, text, audio etc to a LLM and query details in natural language.
Such sytems have a wide range of applicatiions which include searching through a large dataset of information.
you can:
- upload a large set of documents and ask it questions such as "summarize these documents" or "give me a bullet point list of the pre-requisites needed to understand these"
- provide audio files and perform natural language search on it such as "what is the tone of the person in the audio" or "generate a transcript of the audio"
- used llama openrouter as the LLM
- used pypdf2 to extract the text
- used langchain text-splitters to chunck them
- used tiktokenizer(by openai) to convert them into tokens