Skip to content

Latest commit

 

History

History
159 lines (110 loc) · 4.45 KB

readme.MD

File metadata and controls

159 lines (110 loc) · 4.45 KB

🤖 Intelligent Assistance with Face Recognition

Final Year B.Tech Project | Gandhinagar Institute of Technology Created by Dhruv Panchal during internship at Kody Technolab Pvt. Ltd.

📌 Project Overview

This project is a smart, interactive assistant designed to operate within a college or academic environment, combining face recognition, voice-based communication, and a college-specific chatbot.

The system:

  • Detects and identifies people via face recognition (OpenCV)
  • Greets the person by name if recognized, or generically if unknown
  • Converts spoken queries into text
  • Responds using natural language processing trained on a college-specific dataset

🧠 Key Features

Module Description
🎥 Face Detection & Recognition Detects and identifies individuals from camera feed
🗣️ Speech-to-Text Converts voice queries into text for processing
💬 NLP Chatbot Understands user queries and responds using trained model
📚 College Knowledgebase Trained on custom dataset covering 50+ college-related topics
🔊 Text-to-Speech Responds back verbally using TTS engine
💡 Context-Free Doesn’t require prior instruction tuning

💠 Technologies Used

  • Python 3
  • OpenCV – Face detection and recognition
  • NLTK / spaCy – NLP preprocessing
  • Keras + TensorFlow – Custom intent classification model
  • SpeechRecognition / PyAudio – Voice input
  • pyttsx3 / gTTS – Voice output
  • Matplotlib – Training graphs
  • Pickle & JSON – Data serialization
  • Jupyter / VSCode – Development environments

🗂️ Project Structure

IntelligentAssistant/
├── training_data/
│   └── data.json              # Dataset with 52 tags, 160+ patterns
├── model_data/
│   ├── model.keras            # Trained NLP model
│   ├── words.pkl              # Tokenized vocabulary
│   └── classes.pkl            # Output tags (intents)
├── train/
│   └── training.py            # Code to train intent classifier
├── inference/
│   └── prediction.py          # Code to predict tag from user input
├── main.py                    # (To be added) Unified pipeline: face → speech → NLP → TTS
├── README.md                  # Project description and instructions

🦪 Dataset Snapshot

  • Total Tags (Intents): 52

  • Total Patterns: 163

  • Sample Tags:

    • greeting, identity, goodbye
    • courses, fees, admission
    • library, placement, facilities
    • weekend_activities, favorite_class, internship_opportunities, etc.

Example Pattern: “What are the courses offered?” Response: “The college offers IT, CE, ME, CH, and Civil Engineering...”

🚀 How to Run

1️⃣ Train the Chatbot Model

cd train
python training.py

This generates:

  • model.keras
  • words.pkl
  • classes.pkl
  • model_loss_graph.png
  • model_accuracy_graph.png

2️⃣ Predict Responses (Text-Based)

cd inference
python prediction.py

Use chatbot_response("your message") to test.

3️⃣ (Optional) Main Application

Integrate:

  • Webcam feed
  • Voice input/output
  • NLP engine

🧠 Sample Interaction

User: (Camera detects face) → Recognized as "Dhruv"
Bot: Hello Dhruv! How can I help you today?

User: (Speaks) What are the college timings?
Bot: The college is open from 9am to 3pm, Monday to Saturday.

✅ Achievements

  • ✅ Reached ~90% face recognition accuracy
  • ✅ Reached ~97% chatbot classification accuracy
  • ✅ Voice-based system working end-to-end
  • ✅ Custom domain-specific dataset for college environment

📌 Future Work

  • Switch NLP model to Transformers (BERT/Gemma/DistilBERT)
  • Add memory or context handling
  • Build GUI/Web interface using Streamlit or Flask
  • Enhance dataset with real-world phrasing and queries
  • Deploy on a Raspberry Pi + Camera for on-campus assistant

License

This project is developed as part of academic work and is open for educational use.