Evaluating Deepfake Detectors in the Wild

Viacheslav Pirogov ·  Maksim Artemev

🎮 Play the Game

Try our Deepfake game – can you distinguish between deepfakes and real verifications?

📦 Dataset

Download Swappir – over 500k high-quality deepfake images.

🖼️ ICML Poster

---

This repository contains all the code for the models that were used to generate our datasets, as well as the Deepfake Detector models that were used for evaluation.

Generate datasets

GPEN example

Installation

cd ~/NADDACE/data/GPEN/GPEN
pip install -r requirements.txt

Download dataset

cd ~/NADDACE/data/data/lfw
wget https://huggingface.co/datasets/Sumsub/Swappir/resolve/main/lfw_SimSwap.zip
unzip -q lfw_SimSwap.zip

Pretrained models

wget "https://public-vigen-video.oss-cn-shanghai.aliyuncs.com/robin/models/RetinaFace-R50.pth" -O weights/RetinaFace-R50.pth
wget "https://public-vigen-video.oss-cn-shanghai.aliyuncs.com/robin/models/GPEN-BFR-512.pth" -O weights/GPEN-BFR-512.pth
wget "https://public-vigen-video.oss-cn-shanghai.aliyuncs.com/robin/models/GPEN-BFR-256.pth" -O weights/GPEN-BFR-256.pth
wget "https://public-vigen-video.oss-cn-shanghai.aliyuncs.com/robin/models/realesrnet_x2.pth" -O weights/realesrnet_x2.pth
wget "https://public-vigen-video.oss-cn-shanghai.aliyuncs.com/robin/models/ParseNet-latest.pth" -O weights/ParseNet-latest.pth

Inference

cd ~/NADDACE/data/GPEN
python3 create_data.py \
        --input_dir ~/NADDACE/data/data/lfw/lfw_roop \
        --output_dir ~/NADDACE/data/data/lfw/GPEN_lfw_roop

Test Models

SBI example

Installation

cd ~/NADDACE/models/SBI/SelfBlendedImages
pip install -r requirements.txt

Download Dataset

cd ~/NADDACE/data/data/CelebA_HQ
gdown 1xmSduyzHjywucxcvK9bl3CM7-oM5ksSR
unzip -q CelebA_HQ_roop.zip

Pretrained Models

gdown 1X0-NYT8KPursLZZdxduRQju6E52hauV0 -O ~/NADDACE/models/SBI/SelfBlendedImages/weights/FFc23.tar

Inference

python3 inference.py \
        --input_dir  ~/NADDACE/data/data/CelebA_HQ/CelebA_HQ_roop \
        --output_csv  ~/NADDACE/models/preds/SBI/SBI_CelebA_HQ_roop.csv \
        --max_size_image 1024

Licenses

Our work uses a lot of third party libraries as well pre-trained models. The users should keep in mind that these third party components have their own license and terms, therefore our license is not being applied.

Credits

• GPEN
• Roop
• SimSwap
• Detecting Deepfakes with Self-Blended Images - SBI
• Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalization - CADDM
• End-to-End Reconstruction-Classification Learning for Face Forgery Detection - RECCE
• Multi-attentional Deepfake Detection - MAT
• FaceForensics++ - FF
• Multi-modal Multi-scale Transformers for Deepfake Detection - M2TR