Add Payu JSON profiling parser

[edoyango]

Is waiting for infrastructure in #10 to be merged

This adds parsers intended for extracting total job walltime from payu. payu emits a json file located at archive/output000/payu_jobs/0/run/<jobid>.gadi-pbs.json which contains job information from payu and PBS - delineated by payu tasks. This would be useful to understand payu overhead, and extracting total runtime for scaling.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.27%. Comparing base (d178fb9) to head (b605e49).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #15      +/-   ##
==========================================
+ Coverage   99.20%   99.27%   +0.07%     
==========================================
  Files           8        9       +1     
  Lines         250      274      +24     
==========================================
+ Hits          248      272      +24     
  Misses          2        2              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[edoyango]

hi @micaeljtoliveira @manodeep, I've added a parser for the payu json log that payu spits out for each job. If multiple experiment-generator is told to do multiple runs per branch, then a separate json is generated for each run (as they're submitted as separate jobs). Could you have a look?

For context, this is for getting the total walltime of the payu run command, to get a more realistic/reliable number for scaling. The dump also includes the other steps in payu which is also included in the parsing. It discards everything else unrelated to timing.

[manodeep]

Looking good! There are a couple of copy-paste errors and some missing docstrings, but otherwise this should work.

I assume the tests passed locally?

[edoyango]

@manodeep Thanks for reviewing. I've fixed the copy-paste errors and the fh handling and memory we can add later if needed. If there aren't any issues, can you approve this pull request?

[manodeep]

LGTM!

Agreed - the filename/filehandle stuff we can add in later, but wanted to make sure that the docstrings clarify what is accepted as input.

[manodeep]

@edoyango Merging is blocked because of commits without verified signatures - can you please fix that?