try-NLP-optima-from-2025-03-30-study-with-adamw

[david-thrower]

• Set parameters to match the optima from the hyperparam optimizationn study done on GCP on 2025-03-30.
• Use the AdamW optimizer.

See if we observe evidence of improvement.