Description

Added VLSU multimodal dataset. This dataset contains multimodal prompts (text+images) which have safety gradings and categories for the text, images and both combined. This functionality creates text prompts, image prompts and multimodal prompts from the same.

Closes #1285.

Files Changed:

pyrit/datasets/seed_datasets/remote/vlsu_multimodal_dataset.py (new)
pyrit/datasets/seed_datasets/remote/__init__.py (updated exports)
pyrit/datasets/__init__.py (updated exports)
tests/unit/datasets/test_vlsu_multimodal_dataset.py (new)
pyrit/models/seed.py (added optional prompt_text option in the image prompt)

Features:

• Loads image-text pairs from the official ML-VLSU repository
• Creates prompts based on safety grades:
  -- Text-only prompts: When text_grade is unsafe/borderline
  -- Image-only prompts: When image_grade is unsafe/borderline
  -- Combined prompts: When combined_grade is unsafe/borderline (captures emergent harm)
• Supports filtering by 15 harm categories (e.g., hate speech, discrimination, violence)
• Configurable unsafe_grades parameter to control which grades trigger prompt creation (options are safe, unsafe, borderline, not_sure)
• Supports random sampling with configurable limit and seed (for quicker testing. I can remove this if undesirable)
• Caches downloaded images locally for faster subsequent loads

Tests and Documentation

• Some essential unit tests added intests/unit/datasets/test_vlsu_multimodal_dataset.py
  -- Test dataset name property
  -- Test initialization with categories
  -- Test invalid categories raise ValueError
  -- Test text-only prompt creation (unsafe text grade)
  -- Test image-only prompt creation (unsafe image grade)
  -- Test combined prompt creation (unsafe combined grade)
  -- Test all prompts created when all grades unsafe
  -- Test borderline grades trigger prompt creation
  -- Test no prompts when all grades safe (expects error)
  -- Test category filtering
  -- Test handling of failed image downloads

• pre-commit run --all-files passes