Skip to content

jagadeshchilla/Genai

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

9 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

Generative AI

๐Ÿš€ Core AI Frameworks

LangChain LCEL

๐Ÿค– AI Models & Providers

OpenAI Ollama Hugging Face Groq Claude

๐Ÿง  Large Language Models

Gemma Llama Llama2

๐Ÿ—„๏ธ Vector Databases

FAISS Chroma Pinecone Astra DB Cassandra

๐Ÿ’ป Development Tools

Python Jupyter Streamlit LangSmith

๐Ÿ—ƒ๏ธ Database Technologies

SQLite MySQL SQLAlchemy Neo4j Cypher

๐ŸŒ Media & Content Processing

YouTube yt-dlp Wikipedia PyPDF

โ˜๏ธ Cloud Platforms & AI Services

AWS Bedrock NVIDIA NIM

๐Ÿค AI Frameworks & Multi-Agent Systems

CrewAI Stable Diffusion

๐Ÿ”ง Advanced AI Techniques

LoRA QLoRA Lamini


1. What is Generative AI?

Generative AI (GenAI) is a branch of artificial intelligence that focuses on creating new, original content. Instead of just analyzing or predicting outcomes, it generates text, images, music, video, or even code.

  • Example:

    • ChatGPT โ†’ generates human-like text.
    • MidJourney/DALLยทE โ†’ generates images.
    • GitHub Copilot โ†’ generates code.

The key strength of GenAI is its ability to learn from vast datasets and then produce creative outputs that mimic human intelligence.


2. AI vs ML vs DL vs Generative AI

Term Meaning Example
AI (Artificial Intelligence) The broad field of machines simulating human intelligence. Self-driving cars, chess-playing bots
ML (Machine Learning) Subset of AI where systems learn from data and improve over time. Spam email filter
DL (Deep Learning) Subset of ML using neural networks with multiple layers to process complex data. Face recognition, speech-to-text
Generative AI Subset of DL/ML where models create new data/content based on what they learned. ChatGPT (text), DALLยทE (images)

๐Ÿ‘‰ Think of it as: AI = Big umbrella
ML = Learning from data
DL = Complex ML with deep neural networks
Generative AI = AI that creates


3. How OpenAI ChatGPT & LLaMA 3 LLM Models are Trained

Large Language Models (LLMs) like ChatGPT (OpenAI) or LLaMA 3 (Meta) follow a multi-step training process:

  1. Pretraining

    • The model is fed with massive amounts of raw text data (books, articles, websites, code).
    • Goal: Learn grammar, facts, and reasoning.
    • Technique: Self-supervised learning (predict the next word in a sentence).
  2. Fine-tuning

    • Model is further trained on high-quality, domain-specific data.
    • Helps adapt the model for real-world applications.
  3. RLHF (Reinforcement Learning with Human Feedback)

    • Human reviewers rank model responses.
    • AI learns what's helpful, safe, and human-like.
  4. Deployment

    • Optimized for speed, scalability, and efficiency (through quantization, distributed GPUs, etc.).

๐Ÿ‘‰ Difference between ChatGPT & LLaMA 3:

  • ChatGPT โ†’ Trained on massive private datasets, optimized for conversation.
  • LLaMA 3 โ†’ Open-source model, lighter but highly efficient, designed for research & customization.

4. Evolution of LLM Models

  • 2017 โ†’ Introduction of Transformers (Google's "Attention is All You Need").
  • 2018 โ†’ BERT (Bidirectional Encoder Representations from Transformers) โ†’ focused on understanding.
  • 2020 โ†’ GPT-3 โ†’ true generative power with 175B parameters.
  • 2022 โ†’ ChatGPT (GPT-3.5) โ†’ made conversational AI mainstream.
  • 2023 โ†’ GPT-4, LLaMA 2, Claude, PaLM โ†’ more efficient, safer, multimodal.
  • 2024โ€“2025 โ†’ LLaMA 3, GPT-4.1/5, Gemini, Mistral โ†’ smaller, faster, open-source models dominating research & industry.

๐Ÿ‘‰ Trend: Models are becoming smarter, safer, multimodal (text+image+audio), and more open-source.


5. All LLM Models โ€“ Analysis

(Reference: artificialanalysis.ai)

Model Developer Strengths Use Case
GPT-4/ChatGPT OpenAI Best reasoning, creative writing, coding Business, research, education
LLaMA 3 Meta Open-source, lightweight, fine-tuning friendly Research, startups, custom apps
Claude Anthropic Strong safety & long context window Enterprises, compliance
Gemini (Google) Google DeepMind Multimodal (text + image + video) Search, content, enterprise
Mistral / Mixtral Mistral AI Efficient, small models with high performance On-device AI, startups
Grok (xAI/Elon Musk) xAI Integrated with X (Twitter) Social + conversational AI

6. Project Structure & Additional Topics

This repository contains comprehensive implementations and documentation for various Generative AI concepts. Each topic has been organized into separate folders with detailed README files:

Comprehensive guide to the LangChain framework, including:

  • Framework overview and ecosystem
  • Integration capabilities with various LLM providers (OpenAI, Ollama, Hugging Face)
  • Core components: chains, agents, memory, prompts
  • Complete RAG implementation pipeline
  • Data processing: ingestion, transformation, embeddings
  • Vector databases: FAISS, Chroma, Pinecone
  • Advanced search techniques and hybrid search
  • Practical implementations and production-ready examples

LangChain Expression Language project featuring:

  • Simple translation application implementation
  • Multi-LLM integration (OpenAI, Groq)
  • Declarative workflow composition
  • Production-ready patterns and examples

Graph database integration with LangChain featuring:

  • Natural language to Cypher query translation
  • Neo4j integration with LangChain's GraphCypherQAChain
  • Movie database with actors, directors, and genres
  • Few-shot prompting for improved query accuracy
  • Integration with Groq's Gemma2-9b-It model
  • Support for complex graph queries and relationships

Advanced machine learning techniques for model customization:

  • Custom dataset preparation for Q&A tasks
  • Model finetuning with configurable hyperparameters
  • LoRA (Low-Rank Adaptation) implementation
  • QLoRA (Quantized LoRA) support
  • Quantization techniques for memory optimization
  • Secure API key management using environment variables
  • Support for various optimization strategies

Enterprise-grade Generative AI applications on AWS Cloud:

  • AWS Bedrock integration with Claude, Llama2, and Stable Diffusion XL
  • RAG-based PDF document analysis and querying
  • Multi-modal AI capabilities (text and image generation)
  • Enterprise security with AWS IAM and VPC integration
  • Scalable cloud-native architecture for production
  • Cost optimization strategies and performance monitoring
  • Comprehensive AWS services integration (S3, Lambda, CloudWatch)

Advanced graph-based AI applications with stateful workflows:

  • Multi-actor systems with coordinated AI agents
  • Graph-based reasoning and multi-hop query capabilities
  • Neo4j integration with LangChain's GraphCypherQAChain
  • Stateful applications with conversation memory
  • Complex relationship analysis and network insights
  • Interactive graph exploration and visualization
  • Production-ready graph database applications

External Resources

Additional Project Categories

Mathematical AI Applications

  • Text to Math Problem Solver: AI-powered mathematical reasoning with step-by-step solutions
  • Multi-modal Intelligence: Combines calculation with Wikipedia research for comprehensive answers
  • Interactive Chat Interface: User-friendly mathematical problem-solving experience

Advanced RAG Systems

  • PDF Query with Astra DB: Enterprise-grade document querying using Cassandra vector store
  • Relevance Scoring: Intelligent document ranking with accuracy measurement
  • Scalable Architecture: Cloud-native database integration for production environments

Code Intelligence

  • Multi-language Code Assistant: Cross-programming language AI assistance
  • Code Generation & Analysis: Real-time programming support and optimization
  • Language-specific Templates: Tailored assistance for different programming paradigms

7. Technologies Used

This project leverages a comprehensive stack of modern AI and machine learning technologies to build robust Generative AI applications:

Core AI Frameworks

LangChain LCEL

Large Language Models & Providers

OpenAI Ollama Hugging Face Groq Gemma Llama Claude Llama2

Vector Databases & Search

FAISS Chroma Pinecone Astra DB Cassandra

Development & Deployment

Python Jupyter Streamlit LangSmith

Database Technologies

SQLite MySQL SQLAlchemy

Media & Content Processing

YouTube yt-dlp Wikipedia PyPDF

Cloud Platforms & AI Services

AWS Bedrock NVIDIA NIM

AI Frameworks & Multi-Agent Systems

CrewAI Stable Diffusion

Additional Technologies

Transformers Embeddings RAG Hybrid Search Model Switching Prompt Engineering Observability Document Processing Text Chunking Conversation Memory Web Search SQL Generation Database Agents Text Summarization Video Processing Multi-Database Zero-Shot Math Solving Wikipedia Integration Code Generation Multi-Language Relevance Scoring AWS Bedrock Image Generation NVIDIA NIM Multi-Agent Systems Agent Orchestration Enterprise AI Graph Databases Cypher Queries GraphCypherQAChain Model Finetuning Parameter Efficient Quantization Memory Optimization


8. Projects

This section showcases practical implementations and projects that demonstrate the technologies and concepts covered in this repository.

Name Description Tech Stack Link
Q&A Chatbot Enhanced Q&A chatbot with switchable model backends (OpenAI/Ollama) using LangChain. Features Streamlit UI, configurable parameters, and LangSmith tracing for observability. Python Streamlit LangChain OpenAI Ollama LangSmith Q-A-CHATBOT
Simple LLM LCEL Language translation application using LangChain Expression Language. Demonstrates declarative workflow composition with multi-LLM integration (OpenAI, Groq) and production-ready patterns. Python LangChain LCEL OpenAI Groq Gemma Simple_LLm_LCEL
RAG Document Q&A Interactive RAG app for querying research papers (PDFs) using FAISS vector search and Groq-hosted Llama 3.1 model. Features document embedding, similarity search, and conversational Q&A. Python Streamlit LangChain FAISS Groq OpenAI Llama RAG-Document-Q-A
RAG Document Q&A2 Advanced RAG implementation with HuggingFace embeddings, Chroma vector store, and conversation history. Features PDF upload, chunking, and multi-turn dialogue with context-aware retrieval. Python Streamlit LangChain Chroma Hugging Face Groq RAG-Document-Q-A2
Chatbot Search Engine Intelligent chatbot with search engine capabilities combining conversational AI with web search functionality for comprehensive information retrieval and response generation. Python LangChain OpenAI Search Engine chatbot_SearchEngine
LangChain SQL DB Chat Agent Intelligent conversational interface for querying SQL databases using natural language. Features AI-powered SQL generation, multi-database support (SQLite/MySQL), and real-time query execution. Python Streamlit LangChain Groq SQLite MySQL SQLAlchemy langchain_with_sqldb
Text Summarizer LangChain Advanced text summarization application using LangChain with chunking strategies, prompt templates, and multiple AI models. Features customizable summarization parameters and batch processing capabilities. Python LangChain Jupyter OpenAI Text Processing TextSummarizer_langchain
YouTube URL Summarizer Intelligent YouTube video summarization tool that extracts and summarizes video content using LangChain. Features URL processing, content extraction, and AI-powered summarization with customizable output formats. Python LangChain YouTube OpenAI Streamlit yt-dlp Youtube-Website-Url-Summarizer
Text to Math Problem Solver AI-powered application that transforms complex mathematical word problems into step-by-step solutions using Google's Gemma 2 model. Features multi-modal intelligence, Wikipedia research integration, and interactive chat interface. Python Streamlit LangChain Groq Gemma Wikipedia Math Solver Text_TO_Math
PDF Query with Astra DB Advanced RAG system for querying PDF documents using Cassandra Astra DB vector store. Features OpenAI embeddings, character text splitting, and intelligent document retrieval with relevance scoring. Python Jupyter LangChain OpenAI Astra DB Cassandra PyPDF PDFQuery_LAngchain_Astradb
Multi-language Code Assistant Intelligent code assistant supporting multiple programming languages with AI-powered code generation, debugging, and optimization. Features language-specific templates and real-time code analysis. Python LangChain OpenAI Code Assistant Multi-Language multi_language_code-assisstant
NVIDIA NIM Document Q&A Advanced document Q&A system using NVIDIA NIM inference services with Llama models. Features optimized RAG pipeline, real-time document processing, and scalable architecture for enterprise document analysis. Python Streamlit NVIDIA NIM LangChain Llama RAG NVIDIA-NIM-Document-Q-A-System
Multi-AI Agent CrewAI Collaborative AI agent system using CrewAI framework for complex task orchestration. Features multi-agent coordination, specialized role assignments, and intelligent workflow management for enterprise automation. Python CrewAI LangChain OpenAI Multi-Agent Workflow Multi_AIAgent_Crewai
AWS Bedrock GenAI Project Comprehensive AWS Bedrock application showcasing RAG-based PDF chat, text generation with Claude/Llama2, and image generation with Stable Diffusion XL. Features multi-model architecture, enterprise-grade security, and scalable cloud-native design. Python Streamlit AWS Bedrock Claude Llama2 Stable Diffusion FAISS Genai_Aws_BedRock
LangGraph Neo4j Integration Advanced graph-based AI application using LangGraph and Neo4j for stateful multi-actor systems. Features graph-based reasoning, multi-hop queries, and complex relationship analysis with movie database integration. Python LangChain Neo4j Cypher Groq Gemma Graph Database langchain_grapghdb

9. Gemini AI Projects

This section showcases projects specifically built with Google's Gemini AI model, demonstrating advanced conversational AI capabilities and modern web application development.

Name Description Tech Stack Live Demo Repository
Gemini AI Chat Application Modern, responsive web application integrating Google's Gemini AI for intelligent conversations with image analysis capabilities. Features beautiful gradient UI, real-time typing animations, and glassmorphism effects. Python Flask Google AI Gemini JavaScript CSS3 ๐Ÿš€ Live Demo Q-A-chatbot-with-gemini
Gemini Pro Q&A with Chat History Interactive Streamlit web application providing real-time Q&A with Gemini AI, featuring persistent conversation memory and clean responsive interface. Demonstrates advanced chat functionality with session management. Python Streamlit Google AI Gemini Chat History No Live Demo Available Q-A_WITH_CHAT_HISTORY_GEMINI
Multi-Language Invoice Extractor AI-powered multilingual invoice data extraction tool using Google's Gemma AI models. Features multiple model options (Gemma 3 27B/12B/4B IT), up to 95% accuracy, and supports various languages for extracting invoice data like amounts, vendor names, and dates. Python Flask Google AI Gemma Image Processing Multilingual No Live Demo Available MultiLanguage_Invoice_Extractor
DocuAI Advanced document analysis and Q&A system using Google Gemini AI with FAISS vector search. Features document upload, intelligent chunking, semantic search, and conversational AI for comprehensive document understanding and information retrieval. Python Flask Google AI Gemini FAISS Document Analysis No Live Demo Available DocuAI
Text to SQL Generator Intelligent text-to-SQL conversion tool powered by Google Gemini AI. Transforms natural language queries into SQL statements with database schema understanding, query optimization, and error handling for efficient database interactions. Python Flask Google AI Gemini SQL Generation Natural Language No Live Demo Available Text_to-SQl
ATS Resume Expert Intelligent resume analysis platform powered by Google's Gemini AI for ATS optimization. Features comprehensive analysis with section-wise scoring, technical skills matching, experience level assessment, and professional enhancement recommendations. Python FastAPI React TypeScript Google AI Gemini Vercel ๐Ÿš€ Live Demo ATS-Resume-Expert
Calories Advisor AI-powered nutrition and calorie tracking application using Google Gemini AI. Features intelligent meal analysis, calorie calculation, nutritional insights, and personalized dietary recommendations with modern React frontend and FastAPI backend. Python FastAPI React TypeScript Google AI Gemini Vercel ๐Ÿš€ Live Demo calories-advisor
YouTube Video Summarizer Enhanced AI-powered YouTube video summarization tool using Google's Gemini AI. Features multiple summary types (bullet points, detailed, key insights, timeline), batch processing, sentiment analysis, keyword extraction, and comprehensive video metadata analysis. Python Streamlit Google AI Gemini YouTube API Video Processing No Live Demo Available YouTube_Video_summarizer
AI News Research Writing Crew Multi-agent AI system using CrewAI framework with Google Gemini AI for automated news research and article writing. Features Senior Researcher and Writer agents that collaborate to uncover trends, analyze data, and generate compelling tech articles with real-time search integration. Python CrewAI Google AI Gemini Multi-Agent News Research No Live Demo Available AI-News-Research-Writing-Crew

10. Resources

Learning Resources

Complete Generative AI Course

Documentation

Core Frameworks

AI Models & Providers

Vector Databases

Cloud Platforms

Development Tools

Specialized Libraries


Happy Learning!

"The journey of a thousand models begins with a single commit."

Made with Love