video-factchecker

video-factchecker is a tool designed to verify the accuracy of claims made in short videos (like social media reels). It uses AI to transcribe the video's audio and then performs a fact-check on the transcribed text, providing a result, a reason, and supporting sources.

✨ Key Features

• Audio Transcription: Utilizes OpenAI's whisper model to transcribe video audio locally, ensuring privacy.
• AI-Powered Fact-Checking: Leverages an advanced language model (e.g., gpt-4.1-nano) via the pydantic-ai library to analyze text and verify its truthfulness.
• Web-Sourced Verification: Integrates duckduckgo to search for online sources to back up its conclusions.
• Structured Output: Provides a clear result (True/False), a detailed explanation, and a list of consulted sources.
• Results Logging: Automatically saves every request and its outcome to a results.csv file for easy tracking and analysis.
• Text-Only Mode: Can also be used to verify simple text-based claims without a video.

⚙️ How It Works

The verification process follows these steps:

1. The check_video.py script loads a video file (e.g., reel.MP4) from the reels/ directory.
2. The Whisper model analyzes the video's audio and transcribes it into text.
3. The transcribed text is passed to an AI Agent.
4. Guided by a system prompt defined in prompt.md, the agent uses search tools to find relevant information.
5. The agent evaluates the claims and formulates a structured response, indicating whether the information is true or false, why, and the sources it found.
6. The result is printed to the console and appended to the results.csv file.

🚀 Installation

Follow these steps to set up the project on your local machine.

1. Clone the repository:

   git clone https://github.com/Heron4gf/video-factchecker.git
   cd video-factchecker

2. Install dependencies: Ensure you have Python 3.9+ installed.

   pip install -r requirements.txt

   Note: whisper requires ffmpeg to be installed on your system. If you don't have it, follow the instructions for your operating system (e.g., sudo apt-get install ffmpeg on Debian/Ubuntu or brew install ffmpeg on macOS).

3. Configure API Credentials: This project requires an OpenAI API key for the fact-checking agent.

   • Create a file named .env in the project's root directory.
   • Add your API key to the file in the following format:

     OPENAI_API_KEY="sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

🎬 Usage

1. Prepare your video:

   • Place the video you want to analyze inside the reels/ folder.
   • Ensure the file is named reel.MP4 (or update the filename in the check_video.py script).

2. Run the analysis: Execute the main script from the project root:

   python check_video.py

3. Check the results:

   • The output will be printed directly to your terminal.
   • For a complete history of all checks, open the results.csv file.

Text-Only Usage

If you want to verify a simple text claim, you can run the factcheck.py script. This script contains examples of true and false statements and will verify each one.

python factcheck.py

📝 Output

The console output for a video analysis will look similar to this:

Result: False
Reason: While bats use echolocation to navigate and hunt in the dark, all bat species have eyes and are capable of sight. No species of bat is completely blind.
Sources: https://www.usgs.gov/faqs/are-bats-blind, https://www.britannica.com/animal/bat-mammal

The results.csv file will contain rows with the following format:

Request	Result	Reason	Sources
The Italian Serie A league is considering playing some matches on Pluto.	False	This is a satirical claim. Pluto is a dwarf planet...	https://en.wikipedia.org/wiki/Pluto; https://www.nasa.gov/pluto
Bats are blind.	False	While they use echolocation, all bats can see.	https://www.nationalgeographic.com/animals/mammals/facts/bats
Human and chimpanzee DNA is approximately 98.8% similar.	True	Scientific studies confirm a high degree of genetic similarity...	https://humanorigins.si.edu/evidence/genetics

📁 Project Structure

video-factchecker/
├── 📁 reels/
│   └── 📄 reel.MP4       # Example video to be analyzed
├── 📄 check_video.py     # Main script to analyze a video
├── 📄 factcheck.py       # Module with fact-checking logic and text-based tests
├── 📄 prompt.md         # System prompt (instructions) for the AI agent
├── 📄 requirements.txt  # List of Python dependencies
└── 📄 results.csv       # CSV file containing the history of all results