/usr/bin/jq: Argument list too long

[mahdirezaie336]

I ran the runSonarQube.sh file successfully. But at the end of the build process, it waits for a few minutes and then gives me this error:

...
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:47 min
[INFO] Finished at: 2022-12-02T19:41:51+03:30
[INFO] ------------------------------------------------------------------------



./scripts/runSonarQube.sh: line 60: /usr/bin/jq: Argument list too long

My OS is ubuntu 2022 and I use SonarQube verison 9.7.1.

I guess it reaches the maximum size of ARG_MAX in linux which is related to size of args buffer. It is mentioned here and here.
I couldn't fix it by myself.

[darkspirit510]

Hi, thank you for your issue!

Are you talking about SonarQube CE or any paid version?

Sascha

[mahdirezaie336]

Thanks
I use SonarQube CE. Is there any differences?

[mahdirezaie336]

Any updates?

[darkspirit510]

I use SonarQube CE. Is there any differences?
Don't think so. I just wanted to be sure when trying to reproduce.

Any updates?
No, had a lot to do in the last weeks. Will hopefully check it today, but more likely after christmas.

[darkspirit510]

Could you checkout #189 and see, if it works? I reduced the "chunk" size. At least it works on a clean ubuntu installation. If it does not work, I have to rewrite the script completely.

[zoobinn]

@darkspirit510 tested with your changes. Less arguments too long but still the behavior is not stable. And also for Score calculation literally is not correct. Using SonarQube Enterprise 9.9 LTS, Can clearly see the run populates close to ~18K Vulnerability Issues and ~1200 Hotspot. However, the benchmark script is only processing ~200 Vulnerability. Guess the runSonarqube.sh scripts needs a fundamental re-write.

[darkspirit510]

@darkspirit510 did you enter enterprise credentials to the script? (the community edition reports way less results). i don't have access to an enterprise instance. But I guess, jq is not made for generating such a big json file (but I am not sure about this). Feel free to fix it 😉

[zoobinn]

yes, most probably JQ can not handle the size of SonarQube JSON, the more rules the bigger the size of JSON array . Please check this PR. To avoid JQ errors reduced page size to 20 while writing results incrementally in to 2 files.
https://github.com/OWASP-Benchmark/BenchmarkJava/pull/196/files

Tracking this with SonarSource as well (Re Broken Benchmark and PR): https://community.sonarsource.com/t/tech-story-takeaways-from-building-a-sast-product-and-why-owasp-benchmark-is-not-enough/15126/4

[darkspirit510]

Hi @mahdirezaie336 sorry for the late reply. Please retry if #286 solved this issue. I removed the custom jq part completely.