Conversation
…enerators/litellm.py
Bug fix - parallelization pickle error Remove broken line Added docstring to probes/base.py and additional exception catch to generators/litellm.py Restore recent change accidentally removed Restore recent change accidentally reverted Remove unnecessary lines
…com/peter216/garak into bugfix/parallelization-pickle-crash
jmartin-tech
left a comment
jmartin-tech
left a comment
There was a problem hiding this comment.
Instead of a attempting to manage this in base.Probe, please update the LiteLLMGenerator to use the _load_unsafe() pattern from #1545
|
Note on change request reasoning, multiprocessing vs threading can be handled differently on various platforms. To ensure maintainability the project targets consistent multiprocessing when working with parallelization a this time. This reduces the number of variables involved in tracking down a reproduction of reported issues. In the future execution patterns may change. |
|
Thank you—I am working on it as time allows. I am a little fuzzy on how to
apply the pattern and also how to debug efficiently but I’ll get there.
…On Wed, Jan 21, 2026 at 10:15 AM Jeffrey Martin ***@***.***> wrote:
*jmartin-tech* left a comment (NVIDIA/garak#1564)
<#1564 (comment)>
Note on change request reasoning, multiprocessing vs threading can be
handled differently on various platforms. To ensure maintainability the
project targets consistent multiprocessing when working with
parallelization a this time. This reduces the number of variables involved
in tracking down a reproduction of reported issues.
In the future execution patterns may change.
—
Reply to this email directly, view it on GitHub
<#1564 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAIJOEU2FVRJE6I6RIJF52D4H6JZ7AVCNFSM6AAAAACSDFQC7KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTONZYG4YTOOJSGA>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
Looks like this generator actually needs to adjust The core issue is that currently Here is a quick stab at what a solution might look like: Tested with: |
|
Hi @jmartin-tech. I appear to have gotten confused. I can't reproduce the bug I saw now on the unpatched branch and I seem to have attached the wrong logs(!) to this PR, making reproduction that much more difficult. If I see a Pickling error again, I'll know better now how to approach it. In the meantime, I am inclined to close this one out (unless you feel otherwise). With apologies, @peter216 |
2 participants
Bug Description
TypeError: cannot pickle 'module' object./home/peter216/git/ossdev/garak.Failure Output
Partial error log
Change Summary
The change adds a pickling-error fallback to a thread pool in
base.pyto avoid a multiprocessing crash when running--parallel_attempts 16with LiteLLM. No known issue number.Verification
garak Command Results
[✔] garak command executed successfully with parallel attempts
Command output log
pytest Results
[✔] Run the tests and ensure they pass
python -m pytest tests/Command output log
Documentation
[✔] Document the fallback behavior and how it works (if needed)