[DRAFT] add Nanojournalist

[cpfiffer]

This PR adds Nanojournalist, which tracks the time series evolution of all the tests in the benchmark folder.

This is very very unpolished and I mostly wanted to put this as a draft in case anyone wants to take a look, but there is just a litany of hideous code in here.

Some RFCs:

• What do we do with this info?
• Could we have a Genie dashboard?
• What things do we want to know about, i.e. what kinds of analyses are people interested in?
• Can we add some kind of alert system on top of this that isn't already covered elsewhere (I am not super knowledgeable about how these reports are actually used)

[Zentrik]

If you could take a look at tealquaternion.camdvr.org and let me know if there's anything missing or if you have any suggestions that would be appreciated.

The error bars look useful. Also, this looks a lot easier to setup.

[tecosaur]

If a "latest data" compressed arrow file is made available (compressed it should be ~40 MB), I could easily have that be automatically used and periodically re-downloaded with the GLMakie demo I shared in #internals over on Slack. @Zentrik I think your site is great, I also think sometimes it's nice to have something local 🙂.

[Zentrik]

I agree, e.g. comparing locally collected benchmarks with the site would be a pain.

[tecosaur]

https://github.com/tecosaur/julia-nanosoldier-aggregation is now public, and using the arrow preview of the data @cpfiffer kindly provided.

[cpfiffer]

This is super cool! Thanks for running that.

[tecosaur]

@cpfiffer have you got an near-term plans to take this further?

I think it would be really neat it if my nanosoldier aggregation widget/dashboard could be hooked into a live/updating data source :)

[cpfiffer]

I didn't have near term plans but we could cook it up. Finding hosting is probably the hardest bit. We could also wrap this in a Genie app as well, perhaps?

[tecosaur]

Finding hosting is probably the hardest bit.

If we were to use say compressed arrow files, that's small enough for GitHub releases isn't it?

[tecosaur]

@cpfiffer do you anticipate getting back to this any time soon?

[cpfiffer]

Probably not, I'm fairly swamped with work/new job stuff/etc. Might be a bit.