How to report results?

[vsoch]

Hi! My results look like a sequence of different times:

CPU  time:    4.7255269000000003E-002
 CPU  time:    4.7147374999999998E-002
 CPU  time:    4.7264544999999998E-002
 CPU  time:    4.7244825999999997E-002
 CPU  time:    4.7250764000000001E-002
 CPU  time:    4.7116656999999999E-002
 CPU  time:    4.7215857000000000E-002
 CPU  time:    4.7147675000000000E-002
 CPU  time:    4.7149863000000000E-002
 CPU  time:    4.7154943999999997E-002
 CPU  time:    4.7168058999999998E-002
 CPU  time:    4.7134361000000000E-002
 CPU  time:    4.7133198000000001E-002
 CPU  time:    4.7110191000000003E-002
 CPU  time:    4.7188423000000000E-002
 CPU  time:    4.7054049000000001E-002
 CPU  time:    4.7088406999999999E-002
 CPU  time:    4.7562233000000002E-002

Is there a suggested way to report / plot results for different scales /setups? Thanks!

[philipwjones]

@vsoch we haven't kept these up much (last commit was 3 years ago....) and they are probably somewhat outdated. We were mostly using these to explore different loop orders and optimization techniques as we transitioned some of our codes. We haven't been using them as a standard benchmark and so we don't really have a standard way to report the results. Whatever is useful for you. @sarats may have stronger/different opinions since he was responsible for putting this together and interacting with vendors and the performance community.

[sarats]

Hi @vsoch, we have used these kernels during the Exascale Computing project to share with vendors and test out early releases of compilers and simulators. We were also exercising different loop orders etc. to understand optimization tradeoffs.

I'm revisiting putting together larger E3SM test cases (EAMxx and Omega) for incorporating into LCF test suites and eventually external benchmark suites.

If I have additional context on what you are looking for, we can colloborate on something of mutual interest. My email is sarat (at) ornl.gov

[vsoch]

Hi @sarats - apologies for slow reply, it fell deep into my email!

I don't have a specific (scientific) interest - I ran the atm executable as part of a large suite of apps for a scaling study and I'm wanting to know what to plot / report / talk about.

[vsoch]

These are the mins and maxes for each cluster size:

64
0.074450594
0.044978788
16
0.058204117
0.045042457
32
0.075480355
0.044818925
4
0.063608585
0.045062068
8
0.055005241
0.045073904

Run like this:

flux run -N64 -n 5632 -o cpu-affinity=per-task atm

I'm not sure the CPU times output is that interesting, but it definitely took longer to run as we increased the scale. I would presume just because it's running on more CPU but not sure. If you can help me to understand the output it would be greatly appreciated.

[vsoch]

Here is a non squished plot :)