Disable logging

[paper1235]

Hello,

I was wondering, is it possible to disable the logging mechanism? That is, just let the benchmark load images to the model and have the model run inference on them without logging values?

Right now I tried commenting out GlobalLogger().StartIOThread() and a few other things in the StartTest function in loadgen.cc but the code seems to get stuck here: GlobalLogger().GetLatenciesBlocking(expected_latencies)); in the IssueQueries function in RunPerformanceMode for SingleStream

My end goal is to parallelize/multithread the inference engine, and I want to start by profiling the benchmarks, but the IO is too dominant so I want to remove it.

I'm not sure if this something that's easy to fix, but wanted to ask.

Any help would be appreciated,
Thanks!

P.S Hope this was the right place to put this question, if not please let me know.

[guschmue]

Yes, you can tell loadgen to turn of the trace. The python code for this looks like:

    log_output_settings = lg.LogOutputSettings()
    log_output_settings.outdir = output_dir
    log_output_settings.copy_summary_to_stdout = False
    log_settings = lg.LogSettings()
    log_settings.enable_trace = args.debug
    log_settings.log_output = log_output_settings

The loadgen trace is async and does not introduce much overhead.

I have not maintained this app - hope it still works. Always wanted to make a better version of it but have not found the time.

[paper1235]

What about for the cpp version? Would I also put that in the run.py, or would I put this somewhere in mlperf_bench.cc or the like? Presumably I just put this line to false: log_settings.log_output.copy_summary_to_stdout = true; ?

EDIT: that doesn't work (for obvious reasons I see now) - still working on disabling the logger
Thanks for getting back!

[guschmue]

For C++ api there are pretty much the same things you can do.
I think only this line is important:

log_settings.enable_trace = true

all other logging should not be hurting.

If you only care about profiling, might be easier to use this app:
https://github.com/microsoft/onnxruntime/tree/master/onnxruntime/test/perftest

[EtoDemerzel0427]

Hi @guschmue, I tried to disable the trace too (in MLCommons/inference), and was surprised to find that it does not work. The mlperf_log_trace.json file is still generated. Do you have any idea why?

[guschmue]

I think it will always create the file but it should be empty.

[EtoDemerzel0427]

I think it will always create the file but it should be empty.

Ahh, yes, but I would like to add this to this discussion in case people meet the same problem, besides what you have mentioned above, we should also use:

lg.StartTestWtihLogSettings(sut, qsl, settings, log_settings)

to make the change in log_settings take effect.

But as you said, though the trace file is large, which made me think it will hurt the performance badly, it turns out the QPS was n't affected much.