Proposal: Reduce memory requirements for generating results by offering a reduced intermediate results file

[dionysius]

Description of Problem:

Today with containerisation of systems and/or applications there's very low memory available for running oscap. Hello cgroups, docker, lxc, lxd, incus, kubernetes, etc.

Most OVAL definitions in available repositories are huge. The fact that the results file is in XML format I believe it is impossible to stream results directly to the file due to the format constraints - so the oscap process always needs at minimum so much system memory available as the resulting results file size before it can write such file to disk.

So at the core the proposal is splitting the probing and generating results as best as possible. The execution of probing the system could also optimise its memory consumption for that and generate an efficient intermediate format for the pure results only - and a separate system with enough resources could combine the input definitions and this minified pure results file to generate the final OVAL results XML file.

Additional Information:

With oscap-chroot one could at least offload the resource requirements to the host but it comes with the following drawbacks:

• it will probably never offer all types of probes working
• the reduction of a job to only probing is also beneficial for this system
• you need readable filesystem access of that container in some way

There exists oscap oval collect with some interesting arguments - I've just found this now while writing up this issue - but there are zero mentions of it in the oscap user manual. What is it's use case, what is it good for? Maybe this --syschar output file could be such an intermediate format, is it complete enough for my proposal? Because if it is, one could generate the OVAL report XML file using this file and the input file (those could be options in a new results command, e.g. oval eval generate results <collect-file> <input-file> > <results-file>)

I don't know whether it is also beneficial to offer such "minified"/"pure" probe file in the OVAL definition for v6 onwards and thus relay this proposal there as well?