Skip to content

Comments

Refactor presets for attacks and manual tests#154

Merged
RomiconEZ merged 6 commits intomainfrom
refactor-print-test-preset
Jul 15, 2025
Merged

Refactor presets for attacks and manual tests#154
RomiconEZ merged 6 commits intomainfrom
refactor-print-test-preset

Conversation

@RomiconEZ
Copy link
Member

  • Updated the functionality for creating attack presets
  • Added new presets
  • Added new tags: model:llm and model:vlm
  • Added new meta information for PyPI
  • Refactored tests and added a test to verify the presets output

@RomiconEZ RomiconEZ requested review from Copilot and nizamovtimur July 6, 2025 17:58
@RomiconEZ RomiconEZ self-assigned this Jul 6, 2025
@RomiconEZ RomiconEZ added documentation Improvements or additions to documentation improvement Minor improvement in existing functionality enhancement New feature or request labels Jul 6, 2025
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

The PR refactors how attack presets are generated and printed, introduces dynamic tagging for LLM/VLM models, updates metadata for PyPI, and cleans up test utilities.

  • Replaced static preset_configs with a dynamic generator in test_presets.py
  • Added new "model:llm" and "model:vlm" tags to attack definitions
  • Updated example scripts, docs, and metadata (setup.cfg) to reflect new function names and version

Reviewed Changes

Copilot reviewed 42 out of 43 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/llamator/utils/test_presets.py Dynamic preset builder; replaced static mappings with _build_presets
src/llamator/utils/attack_params.py Removed old generators; unified introspection helpers
src/llamator/initial_validation.py Swapped to new get_class_init_params import
src/llamator/attacks/*.py Added "model:llm" tag (and replaced "multimodal:vlm" with "model:vlm")
tests/print_test_preset_test.py New console utility script to print all presets
setup.cfg Added project URLs and updated author field
Comments suppressed due to low confidence (2)

tests/print_test_preset_test.py:28

  • This script prints presets but has no assertions. Either convert it into a unit test with expected-output checks or move it to an examples folder to prevent pytest from collecting it as a test.
def display_all_presets() -> None:

tests/print_langchain_client_test.py:1

  • [nitpick] This script is placed under tests/ but lacks any assertions. Consider relocating it to examples/ or adding assertions to validate the printed output.
if __name__ == "__main__":

Copy link
Member

@nizamovtimur nizamovtimur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Отличная работа, нужно еще в code_documentation.rst обновить названия функций. Также посмотреть замечания от Copilot.

Проверил работоспособность в окружении Python 3.9.6 – всё ок.

Примеры в блокнотах надо будет актуализировать, показать варианты пресетов и перепрогнать – в отдельной ветке после мерджа этого PR и бампа версии (nb: в примере с whatsapp упоминается print_preset_tests_params_example).

nizamovtimur added a commit that referenced this pull request Jul 9, 2025
@RomiconEZ
Copy link
Member Author

Отличная работа, нужно еще в code_documentation.rst обновить названия функций.

Обновил. + Заменил bacis_tests_params на basic_tests, чтобы унифицировать название для верхнеуровневых функций.

@RomiconEZ
Copy link
Member Author

Примеры в блокнотах надо будет актуализировать, показать варианты пресетов и перепрогнать – в отдельной ветке после мерджа этого PR и бампа версии

Уже все сделал и обновил. Возможные пресеты указаны в документации. Больше информации не нужно. Для ноутбука с vlm вызывается соответсвующий пресет.

@RomiconEZ RomiconEZ requested a review from nizamovtimur July 10, 2025 21:00
Copy link
Member

@nizamovtimur nizamovtimur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

N I C E ! ! !

nizamovtimur added a commit that referenced this pull request Jul 13, 2025
* Add Linguistic Sandwich attack (#104)

* refactor attacks descriptions

* fix PAIR description

* add new tag according to #154
@RomiconEZ RomiconEZ merged commit 3ee492e into main Jul 15, 2025
4 of 5 checks passed
@RomiconEZ RomiconEZ deleted the refactor-print-test-preset branch July 15, 2025 09:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation enhancement New feature or request improvement Minor improvement in existing functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants