VerifAI - Multi-Agent Misinformation Detection System

PhD Research Repository
This repository contains the complete implementation, datasets, and experimental scripts for PhD research on misinformation detection and manipulation techniques.
Author: Google Scholar Profile

VerifAI is a comprehensive system for detecting misinformation and manipulation techniques in text content. This repository provides a complete research implementation including:

• Multi-Agent System: LangGraph-based agent system for content analysis
• Backend API: FastAPI REST API for real-time content verification
• Browser Extension: Chrome extension for web-based analysis
• Datasets: Curated datasets for training and evaluation
• Experiments: Reproducible experimental scripts and analysis tools

All components are designed for reproducibility and research purposes.

Components

1. VerifAI Agent System

A multi-agent system built with LangGraph that:

• Manipulation Classifier: Detects psychological manipulation techniques
• Narrative Extractor: Extracts and analyzes narrative structures
• Fact Checker: Performs web searches via Perplexity AI using only trusted sources from debank-sources.json
• Verifier: Combines all analyses into a final assessment

2. FastAPI Backend

A REST API built with FastAPI and uvicorn that:

• Exposes the VerifAI agent system via HTTP endpoints
• Provides real-time content analysis
• Includes comprehensive test coverage (TDD approach)
• Supports CORS for browser-based applications

3. Chrome Extension

A Manifest V3 Chrome extension that:

• Allows text selection analysis on any webpage
• Integrates with browser context menu
• Shows analysis results in an elegant popup
• Communicates with the FastAPI backend

4. Datasets

Curated datasets for research and evaluation:

• Manipulation Dataset: Training and test datasets for manipulation detection (data/manipulation-dataset/)
• Trusted Sources: Verified domain list for fact-checking (data/debank-sources.json)
• All datasets are provided in standard formats (CSV, Parquet) for easy integration

5. Experiments & Reproducibility

Complete experimental framework for reproducible research:

• Experiment Scripts: Configurable experiment runners (experiments/run_experiment.py)
• Model Configurations: JSON-based model configurations (experiments/model_config.json)
• Analysis Tools: Results analysis and visualization scripts (experiments/analyze_results.py)
• Cached Results: Pre-computed results for comparison (experiments/results/)
• All experiments are designed to be fully reproducible with documented configurations

Quick Start

1. Install Dependencies

uv sync

2. Configure Environment

cp .env.example .env
# Edit .env and add your API keys:
# - GEMINI_API_KEY (for LLM operations)
# - PERPLEXITY_API_KEY (for fact-checking with trusted sources)

3. Start the API Server

python scripts/start_api.py

The API will be available at http://localhost:8000

4. Install Chrome Extension

1. Open Chrome and go to chrome://extensions/
2. Enable "Developer mode"
3. Click "Load unpacked" and select the extension folder

Usage

API Usage

# Health check
curl http://localhost:8000/health

# Analyze content
curl -X POST "http://localhost:8000/analyze" \
  -H "Content-Type: application/json" \
  -d '{"content": "Your text to analyze here"}'

Chrome Extension Usage

1. Select any text on a webpage
2. Right-click and choose "Analyze with VerifAI"
3. View the analysis results in the popup

API Response Format

{
  "manipulation": true,
  "techniques": ["emotional_appeal", "false_urgency"],
  "explanation": "Detailed explanation of the analysis...",
  "disinfo": ["List of disinformation flags"]
}

Development

Running Tests

# Run all tests with pytest
python -m pytest tests/ -v

# Or use the test runner script
python scripts/run_all_tests.py

Project Structure

verifai/
  verifai/                    # Core agent system
    nodes/                    # Individual agent nodes
    graph.py                  # LangGraph configuration
  api/                        # FastAPI backend
    main.py                   # API endpoints
  extension/                  # Chrome extension
    manifest.json             # Extension config
    background.js             # Service worker
    content.js                # Content script
    popup.html                # Extension popup
  data/                       # Research datasets
    manipulation-dataset/     # Training and test datasets
    debank-sources.json       # Trusted sources for fact-checking
  experiments/                # Reproducible experiments
    run_experiment.py         # Experiment runner
    analyze_results.py        # Results analysis
    model_config.json         # Model configurations
    results/                  # Experimental results
  tests/                      # Test files
  scripts/                    # Utility scripts
    start_api.py              # Server startup script
    run_all_tests.py          # Test runner script
  docs/                       # Documentation
    PRD_MVP.md                # MVP Product Requirements
    ExtensionPRD.md           # Extension Requirements

Requirements

• Python 3.12+
• Chrome browser (for extension)
• Internet connection (for fact-checking)
• Gemini API key (for LLM operations)
• Perplexity API key (for fact-checking with trusted sources)

Trusted Sources

The fact-checking system uses only verified and trusted domains defined in data/debank-sources.json, including:

• EU institutions (EUvsDisinfo, EDMO, DFRLab)
• EU-based fact-checkers (Bellingcat, EU DisinfoLab)
• Ukrainian organizations (StopFake, VoxCheck, Detector Media)
• International support organizations (RSF, CIR)

This ensures all fact-checking is performed against credible, established sources specialized in countering disinformation.

Citation

If you use this repository in your research, please cite:

@software{verifai2024,
  title = {VerifAI: Multi-Agent Misinformation Detection System},
  author = {Oleh Melnychuk},
  year = {2024},
  url = {https://github.com/olehmell/verifai},
  note = {Oleh Mell Research Repository}
}

For the latest citation information, please refer to the Google Scholar profile.

Contributing

1. Follow TDD practices for new features
2. Ensure all tests pass before submitting
3. Update documentation for new endpoints or features

License

This project is for educational and research purposes.