Component
Browser Extension (@keyvaluesystems/agent-opfor-extension)
Description
What happened:
During a red-team assessment, the browser extension appears to generate (and in some cases queue) the next attack prompt immediately after submitting the current one, without waiting for the AI assistant to finish responding.
As a result, follow-up attacks are generated without considering the assistant's previous response.
This causes issues such as:
- Multi-turn attacks not adapting to the assistant's responses.
- Conversation context becoming inconsistent.
- Responses and subsequent prompts overlapping.
- Reduced effectiveness of conversational red-team testing.
This behavior is particularly noticeable when the target model has a longer response generation time.
What I expected:
The extension should treat each interaction as a complete conversation turn.
Specifically, it should:
- Wait until the assistant has fully completed its response.
- Detect that response generation has finished before proceeding.
- Generate the next attack based on the assistant's latest response.
- Continue the assessment only after the previous turn has fully completed.
This ensures that multi-turn attacks accurately reflect how a real user or attacker would interact with the assistant.
Steps to reproduce
- Start a red-team assessment against a conversational AI application.
- Submit an attack prompt.
- Observe that the extension generates or submits the next attack before the assistant has finished responding.
- Notice that the next prompt is not informed by the assistant's completed response.
Expected:
The extension waits for the assistant to finish responding before generating or sending the next attack prompt.
Actual:
The extension proceeds to the next attack without waiting for the current response to complete.
Config file (redacted)
Relevant log output
Opfor version
0.1.0
Operating system
Linux
Node.js version
No response
LLM provider
Groq
Additional context
No response
Component
Browser Extension (
@keyvaluesystems/agent-opfor-extension)Description
What happened:
During a red-team assessment, the browser extension appears to generate (and in some cases queue) the next attack prompt immediately after submitting the current one, without waiting for the AI assistant to finish responding.
As a result, follow-up attacks are generated without considering the assistant's previous response.
This causes issues such as:
This behavior is particularly noticeable when the target model has a longer response generation time.
What I expected:
The extension should treat each interaction as a complete conversation turn.
Specifically, it should:
This ensures that multi-turn attacks accurately reflect how a real user or attacker would interact with the assistant.
Steps to reproduce
Expected:
The extension waits for the assistant to finish responding before generating or sending the next attack prompt.
Actual:
The extension proceeds to the next attack without waiting for the current response to complete.
Config file (redacted)
Relevant log output
Opfor version
0.1.0
Operating system
Linux
Node.js version
No response
LLM provider
Groq
Additional context
No response