Unconfined Mode for all or VM-based runtimes

[MaxGraey]

Hi, how about use unconfined mode for docker for all benchmarks or at least for VM-based like Ruby, Node.js, Lua?

[rbergen]

That's not something that's controlled by the PrimeView front-end, but by the benchmark tool in Primes. That being the benchmark tool to which the option to use unconfined mode was added with #570. PrimeView only shows, filters and sorts the reports generated by that tool.

[MaxGraey]

Okay, I just wasn't sure exactly where to open this discussion. We can continue this discussion here, right? Or should we move it to PlummersSoftwareLLC/Primes?

[rbergen]

We need to have this discussion here (i.e. in the Primes repository), because it concerns this repository's contents/behaviour.

[marghidanu]

Right now, we're using our own machines to run the benchmarks; the unconfined mode is optional (for security reasons). We might enable it at some point.

[MaxGraey]

In fact, I think it's the right thing to do. But it would at least be interesting to do it for the snapshots published here:
https://plummerssoftwarellc.github.io/PrimeView
wdyt?

[rbergen]

Those are the snapshots generated on the machines we (the maintainers) are running, and all of us happen to be security-minded. So, no, for the snapshots that are currently generated and submitted to PrimeView on a daily basis, unconfined mode will most likely not be enabled. Speaking for myself: I certainly won't.

As @marghidanu has indicated, if and when we reach the point where other people submit results from their machines, each of them can individually decide otherwise on this point. That said, enabling unconfined mode by default for any solution is not something I expect to happen.

[MaxGraey]

Well, it turns out it's this rally not very fair. At least for VM-based langs. Without this mode, VM-based languages are about twice as slow in the docker as native languages. At the same time you also do not want to mark somehow such runtimes in PrimeView as I suggested here: PlummersSoftwareLLC/PrimeView#16

[rbergen]

We're not being unfair on purpose. The unconfined execution of docker images requires the disabling of seccomp, which is a recognized security measure. As maintainers of this repo, we're not just responsible for the quality of the code in the repo, but we also have to mind the interests of the many users of it. Amongst others, this means we cannot do things like accept end-user licenses on their behalf (which is the reason for the exclusion of a number of solutions), include binary files, or decrease the security level of the project on their machines. If people want to run unconfined benchmarks then they can already do so, but it will have to be a conscious decision and require deliberate action from an individual user, to do so.

If there's anybody who would like to dedicate hardware for the purpose of executing a daily, unconfined benchmark run on their machine, they can reach out to us. We can then see if we can incorporate those results in the daily reports.

With regards to the other discussion in PrimeView, I stand by my conclusion that that suggestion is not implementable in its current form. I have indicated my reasons in the discussion. I did miss your last comment to it, which I have just responded to, but the key issue is that I still think that there is a lot of ambiguity hiding under the surface. By pure coincidence, I came across a Quora answer yesterday that discusses this, albeit in a somewhat indirect manner.

[MaxGraey]

Okay, can we at least make a note in PrimeViewer that some rantings run significantly slower in docker? And mention Lua, Ruby, and node.js? Without mentioning specific solutions