[WIP] perf: common neural network workloads benchmarks

[avik-pal]

currently add CPU versions. should be easy to extend to GPU once SciMLBenchmarks has GPU runners available.

TODOs

• Restore Manifest to use the released versions of packages
• Boilerplate
  • Inference
  • Inference Plots
  • Training
  • Training Plots
• Models
  • Simple MLP
    • Inference
    • Training
  • Simple MLP with BN -- wait on perf: fusing activation functions and other misc perf improvements LuxDL/LuxLib.jl#126
    • Inference
    • Training
  • LeNet
    • Inference
    • Training
  • ResNet
    • Inference
    • Training
• Add Reactant Inference Code
  • Training code will have to wait till the compile PR lands in Lux

[ChrisRackauckas]

@thazhemadam @staticfloat what's the easiest way to set this up so that benchmarks can choose a separate runner for GPUs?

[avik-pal]

Allowing different runners based on the jmd metadata would be a nice way. Something like

---
title: Simple Neural Networks
author: Avik Pal
backend: CUDA <--- If nothing present then use CPU. In future we can allow AMDGPU/Metal/etc with the same syntax
---

[avik-pal]

Lux now matches SimpleChains in inference timings 😅. The ones where we fall behind are because Octavian is somewhat slow on EPYC machines so it is turned off

[ChrisRackauckas-Claude]

Superseded by #1530 which completes this work: rebased onto master, updated to latest package versions, added Reactant.jl (via @compile and TrainState API), JAX and PyTorch benchmarks (via PythonCall with Python-side timing), completed all benchmark sections (MLP relu/gelu, MLP+BN, LeNet, ResNet) with both inference and training, and configured GPU runner.