-
Notifications
You must be signed in to change notification settings - Fork 12
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Classifications
- Running
python ./jellybench_py/hwi.pymanually (within the Poetry shell) works and produces correct results. - This happens on a Virtual Hardware (VM, WSL, ...)
- The issue occurs during the benchmark section of the script (after
Starting Benchmark...). - A server issue or faulty server response is causing the problem.
- jellybench was run in
--debugmode (recommended for issue reports). - I am using the latest version (branch
develop) of jellybench. I understand older versions are not supported.
Description
Hi,
I'm testing Jellybench on various systems. If the system can't handle a run with a speed of >= 1 Jellybench crashes with the following error:
Error Code or Output
> > > > Failed: ['performance']
| > > Current Test: 2160p - 720p
| > > > Current Device: cpu
| > > > ffmpeg command: ./jellybench_data/ffmpeg/ffmpeg_files/ffmpeg -c:v h264 -i /home/redacted/jellybench_py/jellybench_data/videos/Test%20Jellyfin%204K%20AVC%20150M.mp4 -noautoscale -an -sn -vf "scale=trunc(min(max(iw\,ih*a)\,528)/2)*2:trunc(ow/a/2)*2,format=yuv420p" -c:v libx265 -preset veryfast -b:v 3616000 -maxrate 3616000 -f null - -benchmark
| > > > > Starting run with 1 Workers... completed with speed 0.99
| > > > > Failed: ['limited']
Traceback (most recent call last):
File "/home/redacted/jellybench_py/bin/jellybench", line 6, in <module>
sys.exit(main())
^^^^^^
File "/home/redacted/jellybench_py/jellybench_py/core.py", line 1045, in main
return cli()
^^^^^
File "/home/redacted/jellybench_py/jellybench_py/core.py", line 998, in cli
valid, runs, result = benchmark(
^^^^^^^^^^
File "/home/redacted/jellybench_py/jellybench_py/core.py", line 360, in benchmark
"single_worker_speed": max_pass_run_data["speed"],
~~~~~~~~~~~~~~~~~^^^^^^^^^
KeyError: 'speed'Steps to Reproduce
Run Jellybench on a system that can't handle cpu intensive transcoding tasks
Operating System
Linux
System Architecture
x86_64 (64-bit)
Python Version
3.12.3
CPU[s]
Intel(R) Core(TM) i5-7500T CPU @ 2.70GHz
GPU[s]
Intel HD Graphics 630
Additional Context
No response
Possible Solution
Hotfix is to set "speed" and "rss_kb"to 0 if not set. Don't know how the API will handle this, though. Change line 360 and 361 in core.py from
"single_worker_speed": max_pass_run_data["speed"],
"single_worker_rss_kb": max_pass_run_data["rss_kb"],
to
"single_worker_speed": max_pass_run_data.get("speed", 0),
"single_worker_rss_kb": max_pass_run_data.get("rss_kb", 0),
josephyooo
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working