Merged
Conversation
add fitting datasets to `num_attempts`
Whatsapp example
* Set dependency - httpx == 0.27.2 * Release v1.1.0 * Delete deprecate img and and chroma-data to gitignore
Union history
rewrite all examples notebooks in english
fix attack model system prompt
* Implement class "MultiStageInteractionSession" for multistage attack. Add new functionality for ChatSession class. * Add multistage to the sycophancy and logical tests --------- Co-authored-by: Roman <roman.nieronov@mail.ru>
Refactor sycophancy and logical_inconsistencies and linguistic
…ultiStageInteractionSession.
Add refine_attack_prompt func to MultiStageInteractionSession.
enhance whatsapp example
NickoJo
approved these changes
Jul 15, 2025
nizamovtimur
approved these changes
Jul 15, 2025
Member
There was a problem hiding this comment.
Changelog v3.2.0..v3.3.0
- Redesigned the output of testing parameter presets. Added the following presets:
all,owasp:llm01,owasp:llm07,owasp:llm09,llm,vlm,eng,rus. - Added a new Linguistic Sandwich attack. An adversarial prompt in a low-resource language is sandwiched between benign prompts in other languages.
- In the System Prompt Leakage attack, the heuristiс evaluation has been replaced with LLM-as-a-judge. This checks the similarity between the system's output and the intended prompt based on the system description.
- The static Past Tense attack has become the dynamic Time Machine attack. The attacking model now alters the temporal context of the adversarial prompt.
- Other minor fixes and improvements.
Member
это лучше убрать, смешное слово |
Add NoneType checking for Judge Model responses fix AutoDAN-Turbo
nizamovtimur
requested changes
Jul 21, 2025
Member
nizamovtimur
left a comment
nizamovtimur
left a comment
There was a problem hiding this comment.
требуется добавить в этот релиз фикс из #158
описание релиза можно оставить таким же, как и в моем прошлом ревью
Enhance evaluations
Update CONTRIBUTING.md
There was a problem hiding this comment.
Pull Request Overview
This is a major version release (v3.3.0) that introduces significant improvements to attack preset handling, adds new attack methods, and enhances the overall framework functionality.
- Replaced parameter-based configuration with a dynamic preset system supporting multiple categories and OWASP classifications
- Added new attack modules including Time Machine, Linguistic Sandwich attacks
- Enhanced existing attack modules with improved error handling and model compatibility tags
Reviewed Changes
Copilot reviewed 54 out of 56 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
src/llamator/__version__.py |
Version bump to 3.3.0 |
src/llamator/utils/test_presets.py |
Complete rewrite of preset system with dynamic generation based on attack tags |
src/llamator/utils/attack_params.py |
Refactored to support new preset system and improved parameter handling |
src/llamator/attacks/time_machine.py |
New attack module for temporal framing vulnerabilities |
src/llamator/attacks/linguistic_sandwich.py |
New attack exploiting attention blink in low-resource languages |
tests/print_test_preset_test.py |
New utility script for displaying preset configurations |
| Multiple attack files | Added model compatibility tags and improved descriptions |
nizamovtimur
approved these changes
Jul 27, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changelog v3.3.0
all,owasp:llm01,owasp:llm07,owasp:llm09,llm,vlm,eng,rus.model:llm/vlm