Refactor presets for attacks and manual tests#154
Conversation
RomiconEZ
commented
Jul 6, 2025
- Updated the functionality for creating attack presets
- Added new presets
- Added new tags: model:llm and model:vlm
- Added new meta information for PyPI
- Refactored tests and added a test to verify the presets output
There was a problem hiding this comment.
Pull Request Overview
The PR refactors how attack presets are generated and printed, introduces dynamic tagging for LLM/VLM models, updates metadata for PyPI, and cleans up test utilities.
- Replaced static
preset_configswith a dynamic generator intest_presets.py - Added new
"model:llm"and"model:vlm"tags to attack definitions - Updated example scripts, docs, and metadata (setup.cfg) to reflect new function names and version
Reviewed Changes
Copilot reviewed 42 out of 43 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/llamator/utils/test_presets.py | Dynamic preset builder; replaced static mappings with _build_presets |
| src/llamator/utils/attack_params.py | Removed old generators; unified introspection helpers |
| src/llamator/initial_validation.py | Swapped to new get_class_init_params import |
| src/llamator/attacks/*.py | Added "model:llm" tag (and replaced "multimodal:vlm" with "model:vlm") |
| tests/print_test_preset_test.py | New console utility script to print all presets |
| setup.cfg | Added project URLs and updated author field |
Comments suppressed due to low confidence (2)
tests/print_test_preset_test.py:28
- This script prints presets but has no assertions. Either convert it into a unit test with expected-output checks or move it to an examples folder to prevent pytest from collecting it as a test.
def display_all_presets() -> None:
tests/print_langchain_client_test.py:1
- [nitpick] This script is placed under
tests/but lacks any assertions. Consider relocating it toexamples/or adding assertions to validate the printed output.
if __name__ == "__main__":
nizamovtimur
left a comment
There was a problem hiding this comment.
Отличная работа, нужно еще в code_documentation.rst обновить названия функций. Также посмотреть замечания от Copilot.
Проверил работоспособность в окружении Python 3.9.6 – всё ок.
Примеры в блокнотах надо будет актуализировать, показать варианты пресетов и перепрогнать – в отдельной ветке после мерджа этого PR и бампа версии (nb: в примере с whatsapp упоминается print_preset_tests_params_example).
…sts' for consistency
Обновил. + Заменил bacis_tests_params на basic_tests, чтобы унифицировать название для верхнеуровневых функций. |
Уже все сделал и обновил. Возможные пресеты указаны в документации. Больше информации не нужно. Для ноутбука с vlm вызывается соответсвующий пресет. |