Skip to content

Conversation

@dependabot
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github Oct 28, 2025

Bumps promptfoo from 0.118.17 to 0.119.0.

Release notes

Sourced from promptfoo's releases.

0.119.0

What's Changed

Features

  • feat(webui): filter eval results by metric values with numeric operators (EQ, GT, LTE, etc.) by @​will-holley in #6011
  • feat(providers): 10-100x performance improvement for Python providers with persistent worker pools by @​mldangelo in #5968
  • feat(providers): add OpenAI Agents SDK integration with support for agents, tools, and handoffs by @​mldangelo in #6009
  • feat(providers): add function calling/tool support for Ollama by @​mldangelo in #5977
  • feat(providers): add support for Claude Haiku 4.5 by @​jameshiester in #5937
  • feat(redteam): add jailbreak:meta strategy with intelligent attack taxonomy learning by @​MrFlounder in #6021
  • feat(redteam): add COPPA plugin by @​typpo in #5997
  • feat(redteam): add GDPR preset mappings by @​typpo in #5986
  • feat(redteam): add modifiers support to iterative strategies by @​MrFlounder in #5972
  • feat(redteam): add authoritative markup injection strategy by @​typpo in #5961
  • feat(redteam): add wordplay plugin by @​typpo in #5889
  • feat(redteam): add Simba red team agent strategy by @​sklein12 in #5795
  • feat(redteam): add subcategory filtering to BeaverTails plugin by @​typpo (a70372f)
  • feat(redteam): include pluginId, strategyId, and sessionId in CSV exports by @​sklein12 in #6016
  • feat(webui): persist custom policy names by @​will-holley in #5990
  • feat(webui): show target responses for red team test cases by @​will-holley in #5869
  • feat(cli): log errors to file with console messages by @​sklein12 in #5992
  • feat(cli): show errors in eval progress bar by @​sklein12 in #5942
  • feat(cache): display latency measurements for cached responses by @​mldangelo in #5978

Fixes

  • fix(providers): restore runtime variable substitution in templates by @​mldangelo (5423f80)
  • fix(providers): improve Python provider reliability with automatic python3/python detection and better error handling by @​mldangelo in #6034
  • fix(providers): simulated-user and mischievous-user now respect system prompts in multi-turn conversations by @​mldangelo in #6020
  • fix(providers): improve MCP tool schema compatibility with OpenAI by @​mldangelo in #5965
  • fix(providers): properly store sessionId in metadata by @​sklein12 in #6016
  • fix(redteam): skip session management tests for stateless targets by @​faizanminhas in #5989
  • fix(redteam): improve Crescendo strategy accuracy by @​jameshiester in #5964
  • fix(redteam): reduce duplicate error messages for invalid strategy and plugin ids by @​typpo in #5954
  • fix(fetch): improve retry counter messages and error details by @​LizzHale in #6017, in #6019
  • fix(webui): pass extensions config when running evals by @​theLucasAntunes in #6006
  • fix(webui): fix visibility of reset config button in red team setup by @​will-holley in #5896
  • fix(webui): sync selected plugins to global config by @​will-holley in #5991
  • fix(webui): fix HTTP test agent by @​faizanminhas in #6033
  • fix(webui): reset strategy config dialog when switching strategies by @​sklein12 in #6035

Chores

Documentation

... (truncated)

Changelog

Sourced from promptfoo's changelog.

[0.119.0] - 2025-10-27

Added

  • feat(webui): filtering eval results by metric values w/ numeric operators (e.g. EQ, GT, LTE, etc.) (#6011)
  • feat(providers): add Python provider persistence for 10-100x performance improvement with persistent worker pools (#5968)
  • feat(providers): add OpenAI Agents SDK integration with support for agents, tools, handoffs, and OTLP tracing (#6009)
  • feat(providers): add function calling/tool support for Ollama chat provider (#5977)
  • feat(providers): add support for Claude Haiku 4.5 (#5937)
  • feat(redteam): add jailbreak:meta strategy with intelligent meta-agent that builds dynamic attack taxonomy and learns from full attempt history (#6021)
  • feat(redteam): add COPPA plugin (#5997)
  • feat(redteam): add GDPR preset mappings for red team testing (#5986)
  • feat(redteam): add modifiers support to iterative strategies (#5972)
  • feat(redteam): add authoritative markup injection strategy (#5961)
  • feat(redteam): add wordplay plugin (#5889)
  • feat(redteam): add pluginId, strategyId, sessionId, and sessionIds to metadata columns in CSV export (#6016)
  • feat(redteam): add subcategory filtering to BeaverTails plugin (a70372f)
  • feat(redteam): Add Simba Red Team Agent Strategy (#5795)
  • feat(webui): persist inline-defined custom policy names (#5990)
  • feat(webui): show target response to generated red team plugin test case (#5869)
  • feat(cli): log all errors in a log file and message to the console (#5992)
  • feat(cli): add errors to eval progress bar (#5942)
  • feat(cache): preserve and display latency measurements when provider responses are cached (#5978)

Changed

  • chore(internals): custom policy type def (#6037)
  • chore(changelog): organize and improve Unreleased section with consistent scoping and formatting (#6024)
  • chore(cli): show telemetryDisabled/telemetryDebug in promptfoo debug output (#6015)
  • chore(cli): improve error handling and error logging (#5930)
  • chore(cli): revert "feat: Improved error handling in CLI and error logging" (#5939)
  • chore(webui): add label column to prompts table (#6002)
  • chore(webui): gray out strategies requiring remote generation when disabled (#5985)
  • chore(webui): gray out remote plugins when remote generation is disabled (#5970)
  • chore(webui): improve test transform modal editor (#5962)
  • chore(webui): add readOnly prop to EvalOutputPromptDialog (#5952)
  • refactor(webui): organize red team plugins page into tabs with separate components (#5865)
  • chore(redteam): remove "LLM Risk Assessment" prefix (#6004)
  • chore(redteam): add top-level redteam telemetry events (#5951)
  • refactor(webui): reduce unnecessary API health requests (#5979)
  • chore(api): export GUARDRAIL_BLOCKED_REASON constant for external use (#5956)
  • chore(providers): add rendered request headers to http provider debug output (#5950)
  • refactor(transforms): refactor transform code to avoid 'require' (#5943)
  • refactor(transforms): refactor createRequest/ResponseTransform functions into separate module (#5925)
  • chore(examples): consolidate Ollama examples into unified directory (#5977)
  • chore(deps): move dependencies to optional instead of peer (#5948)
  • chore(deps): move natural to optional dependency (#5946)
  • chore(redteam): improve GOAT and Crescendo error logs with additional error details for easier debugging (#6036)

Fixed

... (truncated)

Commits
  • 804c511 chore: bump version 0.119.0 (#6039)
  • d1731e5 chore: Better error logging for goat and crescendo strategies (#6036)
  • 23381e0 chore(internals): Updates custom policy type (#6037)
  • 6da6a8f fix(providers): improve Python provider reliability and logging (#6034)
  • 619718b fix(webui): reset red team strategy config dialog state when switching strate...
  • e6b2565 fix: HTTP test agent (#6033)
  • 2679389 fix(app): test target (#6031)
  • ea4aadc feat: Simba, the most advanced Red Team Agent (#5795)
  • 2a2d9be revert(providers): eager template rendering that broke runtime variable subst...
  • a26ec96 feat(app): Eval results metric value filtering (#6011)
  • Additional commits viewable in compare view

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Bumps [promptfoo](https://github.com/promptfoo/promptfoo) from 0.118.17 to 0.119.0.
- [Release notes](https://github.com/promptfoo/promptfoo/releases)
- [Changelog](https://github.com/promptfoo/promptfoo/blob/main/CHANGELOG.md)
- [Commits](promptfoo/promptfoo@0.118.17...0.119.0)

---
updated-dependencies:
- dependency-name: promptfoo
  dependency-version: 0.119.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot dependabot bot added dependencies Pull requests that update a dependency file javascript Pull requests that update Javascript code labels Oct 28, 2025
@dependabot @github
Copy link
Contributor Author

dependabot bot commented on behalf of github Oct 30, 2025

A newer version of promptfoo exists, but since this PR has been edited by someone other than Dependabot I haven't updated it. You'll get a PR for the updated version as normal once this PR is merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file javascript Pull requests that update Javascript code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant