Skip to content

Conversation

@ValbuenaVC
Copy link
Contributor

@ValbuenaVC ValbuenaVC commented Nov 18, 2025

Description

Moving dataset loading responsibilities to a dedicated ScenarioDatasetLoader class. Accomplishes a few things:

  • Reorganizes pyrit.scenario to be easier to read and mirror pyrit.executor's layout. scenarios.core is where the base classes are, scenario.dataset is where config and data loading features are kept, scenario.printer is unchanged, and scenario.scenarios is where actual plug-and-play scenarios stay
  • Scenario & ScenarioDatasetLoader supports lazy dataset loading for performance (so actual prompts are loaded into memory only when initialize_async is called, which in turn calls .load() for the loader)
  • AtomicAttack is now parameterized by a single SeedGroup, and other objective and custom prompt parameters have been removed. This is for two reasons:
    • To avoid conflating prompts with attack objectives. AtomicAttacks are now truly atomic in that they represent one goal, but may have several prompts included to achieve it.
    • To allow for dataset loading to be decoupled from Scenario, because all Scenario interfaces now expect a SeedGroup rather than a list of primitives
  • Standardizes interface for Scenarios to use seed-based types (e.g. Seed, SeedPrompt, SeedGroup, etc.)

The new design is meant to decouple as much of the prompt management as possible, so that a Scenario subclass can have a rich custom attack construction if desired, or just use a default SeedGroup if not

Tests and Documentation

TBD

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant