Merge pull request #9 from BrataBuilds/main

Updated the project directory, Refactored the backend codebase, added better instructions on Contributing.md and improved the readme

Author: BrataBuilds

CONTRIBUTING.md

@@ -1,69 +1,91 @@
 # CONTRIBUTING
 This document contains the list of issues, suggestions and improvements that can be added to this project and a proper guide on how to contribute to the project.
+- [CONTRIBUTING](#contributing)
+  - [Registration](#registration)
+  - [Guildlines :](#guildlines-)
+    - [List of major issues :](#list-of-major-issues-)
+  - [List of Improvements and Suggestions](#list-of-improvements-and-suggestions)
+    - [Improvments to the UI](#improvments-to-the-ui)
+      - [Long-term Goals :](#long-term-goals-)
+    - [Improvements to Backend :-](#improvements-to-backend--)
+    - [Long term Goals :](#long-term-goals--1)
+      - [In game control :](#in-game-control-)
+      - [Cross platform compatibility :](#cross-platform-compatibility-)
+  - [Other Known Bugs and Issues : 🪲](#other-known-bugs-and-issues--)
 
 ## Registration 
-To be eligible for MLSA X HACKTOBERFEST: 
-[Star this repo](https://github.com/keploy/keploy)
-[Register here for MLSAKKIIT](https://register.mlsakiit.com/)
-[Register on HacktoberFest](https://hacktoberfest.com/auth/)
+>[!important] Attention
+All contributors must do the following to be eligible for MLSA X HACKTOBERFEST: 
+- [Star this repository](https://github.com/keploy/keploy)
+- [Register here on MLSA KIIT website](https://register.mlsakiit.com/)
+- [Register on HacktoberFest Official Website](https://hacktoberfest.com/auth/)
 
 ## Guildlines : 
 Adhere to [Hacktober Fest Guidlines](https://hacktoberfest.com/) and maintain common ettiquette of contributing to an open source project. 
 
 If you have any questions regarding contributing to this repository please contact the contributor. 
 
 <!-- (link to the whatsapp group) -->
+### List of major issues :
+>[!IMPORTANT]
+It is highly recommended for contributors to first have a look at the list of major issues work on them with higher priority.
 
-## UI Improvements  
-1. Add markdown support for gemini's reponses.
-2. Improve the chat window UI such that it is easier to tell apart the messages of the user and Pixly.
-3. Add the ability to view images results from the web and play recommended youtube videos directly within the overlay.
-4. Make the window resizable and all window elements scalable.
-5. Add window control buttons for minimise and fullscreen (to be added after the previous implementation)
+**Related to UI :-**
+1. Make the window resizable and all window elements scalable.
+2. Add window control buttons for minimise and fullscreen (to be added after the previous implementation)
+3. Whenever you hover over the screenshot button, placehoder text is inserted in the chatbox but it doesn't go away after you stop hovering, and you have to manually press backspace to remove the text.
+4. Refactor the codebase of the overlay modular so that it is easier to work with.
+
+**Frontend+Backend :-**
+1. Chat history is not stored, if you try to follow up gemini with what you asked in the previous chat it will have 0 idea what you are talking about.
 
-### Long-term Goals : 
-6. Add the ability for users to add custom themes of the overlay, different themes belonging to different games, so when the overlay can automatically apply a certain theme when a particular game is detected.
+>[!TIP] Here is a suggested solution: 
+Store the chats of the user in a database, (we are already using sqlite for screenshots, might as well use it), then when we give a new prompt to gemini, old chats are added to the prompt. 
+- Only store the last 30 chats or so.
+- Every game will have its own database table, i.e. chats are stored on a per game basis.
+- Additional meta-data such as timestamp should also be stored so when user asked "What did I do yesterday?" it should be able to retrieve the screenshot from 24 hours ago etc.
 
-## Main Improvements 
-1. Chat history is not stored, add the ability to store the chat history the user across various sessions, preferably in a per game basis, in the vector database.
 
->[!Important] Also add the ability to view the stored chats across various sessions/games from the overlay, add the configuration in the settings menu to delete and edit them. Add a setting to set how many chats per game/session to store.
-
-2. Make a better system for sending existing screenshots. Improve the keyword search and allow user to edit the prompt before sending the screenshot instead of using the default prompt. Allow the chatbot agent to automatically pull a screenshot based on a given time and date duration.
-
-3. Add more entries about wikis, guides, youtube videos, forum posts about more games, especially single  player story based titles like `Elden Ring, Hollow Knight : Silksong, Black Myth: Wukong,Cyberpunk 2077`
-
-4. Add cross platform support, (change the win32 dependency to an alternative)
-
-5. Improve the project structure to better align with [best practices](https://github.com/zhanymkanov/fastapi-best-practices) when using FastAPI. The project already defines pydantic schemas. Also Example project structure. :
-```
-backend/
-│
-├── main.py                      # Entry point (FastAPI app)
-│
-├── routers/                     # Route definitions (API endpoints)
-│   ├── chat.py
-│   ├── screenshots.py
-│   ├── game_detection.py
-│   └── knowledge.py
-│
-├── schemas/                     # Pydantic models
-│   ├── chat.py
-│   ├── screenshot.py
-│   ├── game_detection.py
-│   └── knowledge.py
-│
-├── services/                    # Business logic / helper functions
-│   ├── chatbot.py
-│   ├── screenshot_service.py
-│   ├── knowledge_service.py
-│   └── vector_service.py
-│
-└── core/
-    ├── config.py                # Settings, env vars, constants
-    └── logger.py                # Central logging setup
-
-```
+>[!Important] UI/UX Addition :
+- Add the ability to view the stored chats across various sessions/games from the overlay.
+- Add the configuration in the settings menu to read, delete and edit them. 
+- Add a setting to set how many chats per game/session to store.
+
+**Backend :-**
+1. Chroma db vector collections aren't searched properly, this may have to do with the chroma client not being initialised properly or the collections are not being created properly in `get_or_create_collection()` or the incorrect implementation of `search_knowledge()` in *vector_service.py*. This issue requires a more thorough investigation.
+2. Web Scrapper in *knowledge_manager.py* sometimes gets blocked by certain websites, (*namely* the ones present in *minecraft.csv*)
+>[!tip] Recommended Solutions :
+- Use Proxies to circumvent IP bans.
+- Rotate a list of User Agents and headers.
+- Make it asynchronous using `asyncio + httpx`
+
+
+
+
+## List of Improvements and Suggestions
+>[!tip] Feel free to give us any of your ideas, suggestions and feedback to add to this list.
+
+### Improvments to the UI
+1. Add Markdown support for Gemini's reponses.
+2. Improve the chat Window UI such that it is easier to tell apart the messages of the user and Pixly.
+3. Add the ability to view images results from the web and play recommended youtube videos directly within the overlay.
+
+
+
+#### Long-term Goals : 
+1. Add the ability for users to add custom themes of the overlay, different themes belonging to different games, so when the overlay can automatically apply a certain theme when a particular game is detected.
+
+### Improvements to Backend :-
+
+1. Add more .csv entries about wikis, guides, youtube videos, forum posts about more games, especially single  player story based titles like `Elden Ring, Hollow Knight : Silksong, Black Myth: Wukong,Cyberpunk 2077`
+2. Implement a Better way to store screenshots:
+>[!tip] Suggested Improvemments :
+- Vectorise the screenshots as well.
+- Add tool calling for the agent to call a tool to retrieve the screenshot from a specific time or from a specific game.
+1. Improve the Web Scrapper :
+
+In Addition to fixing the above issues, add the ability to scrape youtube audio transcriptions.
+
 
 ### Long term Goals :
 #### In game control : 
@@ -73,9 +95,10 @@ Add the ability for the agent to control the game and play the game for you and
 Reliance on the win32 api for taking screenshots means we can't transition to a different platform, and are stuck with Windows for now.
 In the future we may wanna add cross platform compatibility with Linux.
 
-## Known Bugs and Issues : 🪲
+## Other Known Bugs and Issues : 🪲
+> [!important] These are a bit obscure and their causes aren't known yet. 
+1. Overlay hangs and then crashes when turning off the *enable screenshots setting.*
+2. `game_detection.py`, it reports the incorrect game being detected, in some cases.
+3. After adding your API Key from the overlay, sometimes it shows that the user has added their key, sometimes it doesn't.
 
-1. Overlay hangs and then crashes when turning off the enable screenshots setting.
-2. Diagnose `knowledge_manager.py`, it is unable to scrape the given webpages. Returns a `403: Forbidden Error`, Possibly has to do with anti-bot anti-web_scraping measures.
-3. Diagnose `game_detection.py`, sometimes it always reports the current game being played as minecraft.
 

README.md

@@ -14,13 +14,15 @@ Pixly - Your AI Gaming Assistant 🎮
 
 Pixly is a desktop overlay that acts as your gaming assistant, combining AI chat with automated, privacy-friendly screenshot capture and a game-specific Retrieval-Augmented Generation (RAG) knowledge base. Pixly detects what game you're playing, retrieves relevant, curated knowledge (wikis, user-supplied YouTube descriptions, and forum posts) via a local vector database, and grounds Gemini responses on those sources.
 
-**🎃 Hacktoberfest 2025 Participant** | Join us in making gaming more accessible with AI! Also make sure to [star this repo](https://github.com/keploy/keploy).
+Make sure to star our repository, your support is much appreciated.
+
+>[!important] 🎃 Hacktoberfest 2025 Participant
+Please make sure to [star this repo](https://github.com/keploy/keploy).
 ## 📋 Table of Contents
 
 - [📋 Table of Contents](#-table-of-contents)
-- [🎮 What Pixly Does](#-what-pixly-does)
 - [🤝 Contributing, Setup and Install](#-contributing-setup-and-install)
-  - [🎃 Hacktoberfest 2025 - How to Contribute](#-hacktoberfest-2025---how-to-contribute)
+- [🎮 What Pixly Does](#-what-pixly-does)
 - [🏗️ Architecture Overview](#️-architecture-overview)
   - [1) UI Overlay (`overlay.py`)](#1-ui-overlay-overlaypy)
   - [2) Backend API (`backend/`)](#2-backend-api-backend)
@@ -35,32 +37,22 @@ Pixly is a desktop overlay that acts as your gaming assistant, combining AI chat
 - [📄 License](#-license)
 - [🙏 Acknowledgments](#-acknowledgments)
 
-## 🎮 What Pixly Does
-
-- 🤖 Intelligent, game-focused chat using Google Gemini with a "Game Expert" system prompt
-- 🎯 Contextual help based on your active game (process detection and/or user message)
-- 📸 Optional screenshot-powered context for visual analysis
-- 🔍 RAG pipeline over per-game CSV knowledge with local vector search (Chroma)
-- 💻 Modern desktop overlay for chatting, settings, and screenshot gallery
-
 ## 🤝 Contributing, Setup and Install
 
 **We welcome Hacktoberfest 2025 contributors!** Whether you're adding new games to the knowledge base, improving the UI, or enhancing AI capabilities, your contributions matter.
 
 - 📖 For Contributing Visit [CONTRIBUTING.md](https://github.com/MLSAKIIT/pixly/blob/main/CONTRIBUTING.md)
 - ⚙️ For Setup and Installation visit [INSTALL.md](https://github.com/MLSAKIIT/pixly/blob/main/INSTALL.md)
 
-### 🎃 Hacktoberfest 2025 - How to Contribute
 
-This project is participating in **Hacktoberfest 2025**! Here are some ways you can contribute:
+## 🎮 What Pixly Does
 
-1. **Add Game Knowledge**: Contribute CSV files with game wikis, YouTube videos, and forum links
-2. **Improve Documentation**: Enhance README, add tutorials, or create guides
-3. **Fix Bugs**: Check our issues and help resolve reported bugs
-4. **Add Features**: Implement new features like support for more games or UI improvements
-5. **Optimize Performance**: Improve RAG retrieval, vector search, or screenshot handling
+- 🤖 Intelligent, game-focused chat using Google Gemini with a "Game Expert" system prompt
+- 🎯 Contextual help based on your active game (process detection and/or user message)
+- 📸 Optional screenshot-powered context for visual analysis
+- 🔍 RAG pipeline over per-game CSV knowledge with local vector search (Chroma)
+- 💻 Modern desktop overlay for chatting, settings, and screenshot gallery
 
-**Note**: Make sure to follow our [Code of Conduct](CODE_OF_CONDUCT.md) and contribution guidelines!
 
 ## 🏗️ Architecture Overview
 
@@ -150,20 +142,29 @@ The detection result is passed into the RAG layer to scope retrieval to the acti
 ## 📁 Project Structure
 
 ```
-Hacktober Fest/
-├── backend/
-│   ├── __init__.py               # FastAPI app initialization
-│   ├── backend.py                # API endpoints and routing
+pixly/
+├── backend/             
+│   ├── backend.py                # FastAPI app initialization
+├── routers/                      # Contains all the API Routers
+|   ├── chat.py                   # Stores chat endpoints
+|   ├── game_detection.py         # Stores game detection and vector search endpoints
+|   ├── screenshot.py             # Stores screenshot endpoints
+|   ├── setting.py                # Stores settings endpoints
+├── services/                     # Contains all the backend services.
 │   ├── chatbot.py                # Gemini integration, RAG-aware chat, runtime reconfigure
 │   ├── screenshot.py             # Encrypted screenshot capture, DB ops, delete support
 │   ├── game_detection.py         # Process/message/screenshot-based game detection
 │   ├── knowledge_manager.py      # CSV ingestion and content extraction (wiki/forum)
 │   └── vector_service.py         # Chroma collections, embeddings, and search
+├── schemas/                      # Contains the schemas for the various requests
+|   ├── chat.py
+|   ├── game_detection.py
+|   ├── knowledge_search.py
+|   ├── settings.py
 ├── overlay.py                    # CustomTkinter overlay (chat, settings, screenshot viewer)
 ├── games_info/                   # Per-game CSVs (e.g., minecraft.csv)
 ├── vector_db/                    # Chroma persistent storage
 ├── PROMPTS.txt                   # System persona + RAG grounding instructions
-├── test_system.py                # Local API test harness
 ├── run.py                        # Backend server launcher
 ├── pyproject.toml                # Dependencies and metadata
 ├── screenshots.db                # Encrypted screenshot database (auto-created)

backend/__init__.py

@@ -1,2 +0,0 @@
-from fastapi import FastAPI
-app = FastAPI()