read temp response json file failed with latest version #6072
Description
Describe the bug
I upgrade to the latest version from 0.118.17 and found my existing eval cannot run successfully. I have checked the log, it seems the new version will try to read some temp json file when trying to use similarity or llm-rubric assert.
below is the error log:
Error: ENOENT: no such file or directory, open 'C:\Users\fma\AppData\Local\Temp\promptfoo-worker-resp-1761880046684-948141998bb528.json'
Error: ENOENT: no such file or directory, open 'C:\Users\fma\AppData\Local\Temp\promptfoo-worker-resp-1761880046684-948141998bb528.json'
at Object.readFileSync (node:fs:440:20)
at PythonWorker.executeCall (C:\Users\fma\AppData\Roaming\nvm\v24.11.0\node_modules\promptfoo\dist\src\python\worker.js:119:49)
at async PythonWorker.call (C:\Users\fma\AppData\Roaming\nvm\v24.11.0\node_modules\promptfoo\dist\src\python\worker.js:87:20)
at async PythonProvider.executePythonScript (C:\Users\fma\AppData\Roaming\nvm\v24.11.0\node_modules\promptfoo\dist\src\providers\pythonCompletion.js:169:22)
at async matchesSimilarity (C:\Users\fma\AppData\Roaming\nvm\v24.11.0\node_modules\promptfoo\dist\src\matchers.js:229:35)
at async handleSimilar (C:\Users\fma\AppData\Roaming\nvm\v24.11.0\node_modules\promptfoo\dist\src\assertions\similar.js:36:17)
at async runAssertion (C:\Users\fma\AppData\Roaming\nvm\v24.11.0\node_modules\promptfoo\dist\src\assertions\index.js:362:24)
at async C:\Users\fma\AppData\Roaming\nvm\v24.11.0\node_modules\promptfoo\dist\src\assertions\index.js:409:24
To Reproduce
use below eval with latest version promptfoo
tools_call.zip
Please note that I am using local LM studio model to do the eval
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
System information:
If possible, please output the results of promptfoo debug and paste the output here.
- Promptfoo version:
Additional context
Add any other context about the problem here.
Metadata
Metadata
Assignees
Labels
No labels