LARS and LAMB issues related to str2optimizer32bit

[RasmusHoier]

EDIT: This seems more appropriate for an issue, so I have opened an issue instead: #1810

I am using bitsandbytes (version 0.48.2) and I am trying to run use LARS in my torch code. Strangely I can get other bitsandbytes optimizers including LAMB running, but LARS wont run.

Error message:

.venv/lib/python3.12/site-packages/bitsandbytes/backends/cuda/ops.py", line 635, in optimizer_update_32bit_impl
raise ValueError(
ValueError: Unsupported optimizer name: lars. Supported optimizers: ['adam', 'momentum', 'rmsprop', 'lion', 'adagrad', 'ademamix']

Looking at bitsandbytes/backends/cuda/ops.py it seems to be because
optim_fns = str2optimizer32bit.get(optimizer_name, None) has value None, because str2optimizer32bit does not have a case for lars.

Looking at str2optimizer32bit I see that it does indeed lack a case for "lars", but also the case for "lamb" seems to be identical to the case for ADAM. Here is a link to the relevant lines?

TLDR:

• Why is "lars" not handled by str2optimizer32bit in bitsandbytes/backends/cuda/ops.py?
• Why does "lamb" map to the same c functions as "adam"in in bitsandbytes/backends/cuda/ops.py?

Possibly related issues:

• LAMB missing in str2optimizer32bit? #1118
• [Bug]: Cannot use LARS or LAMB Nerogar/OneTrainer#390