In this repo, I will include all my research work that I will do to upskill my self for Inter-IIT.
-
I have attached the notebook named Lora-Implementation in which I have tried to finetune bloom-1b1 llm model on google colab for qa using Lora, it worked fine but with some overfitting resulting in repeated generation of answers. Also I have pushed the finetuned model to hugging face. (model- alpha-centori/newbloom)
-
I have also tried to implement the Qlora fine tuning from Qlora-Implementation.ipynb in which I have fine-tuned the NousResearch/Llama-2-7b-chat-hf onto a dataset mlabonne/guanaco-llama2-1k and it worked fine, Also I have pushed the fully combined model to hugging face. (model - alpha-centori/Llama-2-7b-chat-finetune)
-
I have attached a colab notebook with the implementation and pipeline of Web Search Agent and Python Interpreter for searching a query on internet and imlement python code if needed in it.