FEAT: Added VLSU Multimodal Dataset

[riyosha]

Description

Added VLSU multimodal dataset. This dataset contains multimodal prompts (text+images) which have safety gradings and categories for the text, images and both combined. This functionality creates text prompts, image prompts and multimodal prompts from the same.

Closes #1285.

Files Changed:

pyrit/datasets/seed_datasets/remote/vlsu_multimodal_dataset.py (new)
pyrit/datasets/seed_datasets/remote/__init__.py (updated exports)
pyrit/datasets/__init__.py (updated exports)
tests/unit/datasets/test_vlsu_multimodal_dataset.py (new)
pyrit/models/seed.py (added optional prompt_text option in the image prompt)

Features:

• Loads image-text pairs from the official ML-VLSU repository
• Creates prompts based on safety grades:
  -- Text-only prompts: When text_grade is unsafe/borderline
  -- Image-only prompts: When image_grade is unsafe/borderline
  -- Combined prompts: When combined_grade is unsafe/borderline (captures emergent harm)
• Supports filtering by 15 harm categories (e.g., hate speech, discrimination, violence)
• Configurable unsafe_grades parameter to control which grades trigger prompt creation (options are safe, unsafe, borderline, not_sure)
• Supports random sampling with configurable limit and seed (for quicker testing. I can remove this if undesirable)
• Caches downloaded images locally for faster subsequent loads

Tests and Documentation

• Some essential unit tests added intests/unit/datasets/test_vlsu_multimodal_dataset.py
  -- Test dataset name property
  -- Test initialization with categories
  -- Test invalid categories raise ValueError
  -- Test text-only prompt creation (unsafe text grade)
  -- Test image-only prompt creation (unsafe image grade)
  -- Test combined prompt creation (unsafe combined grade)
  -- Test all prompts created when all grades unsafe
  -- Test borderline grades trigger prompt creation
  -- Test no prompts when all grades safe (expects error)
  -- Test category filtering
  -- Test handling of failed image downloads

• pre-commit run --all-files passes

[rlundeen2]

If I understand this dataset correctly, each line in the dataset has both a text part and an image part. So I would just say if the harm is sufficient, make sure both have the same group id and add them

[riyosha]

The text and image parts individually might have a different safety grade than the combined prompt - for instance, text and image individually might be harmful individually, but their combined prompt can be a safe prompt. I thought it could be useful to create individual text and image prompts to use them in cases where the combined prompt would be skipped.

I could create the unsafe text/image prompts specifically when the combined prompt is safe and they'd be skipped otherwise. Or if you think its better to remove them completely, I can just keep the combined prompts too

[rlundeen2]

I think they should always be added (or not) together, because we likely always want to send them together as part of a single Message.

I don't care quite as much about how we decide which ones to add as much as they be added together so we can handle them consistently :)

[rlundeen2]

We should always generate a new uuid. Otherwise there will be weirdness. It's unique to pyrit for when we pull these from the database

[riyosha]

right I overlooked that, makes sense!

[rlundeen2]

This is good! Thank you! I think once comments are addressed and CI passes it should be ready to merge.