[WandbLogger] resume="never" without id still tries to resume existing run

[oooooio]

Bug description

When using Lightning CLI with WandbLogger, I did not set an id and configured resume="never". However, the run fails with the following error:

wandb.errors.errors.UsageError: You provided an invalid value for the `resume` argument. The value 'never' is not a valid option for resuming a run (AUWave) that already exists.

According to W&B semantics, resume="never" should not attempt to resume any existing run. If no id is set, it should not conflict with an existing run either.

•When resume="never" and no id is explicitly provided, the logger should not try to resume an existing run.
•Lightning should not accidentally propagate a historical run id from the working directory into W&B, leading to a conflict with the "never" setting.

What version are you seeing the problem on?

v2.5

Reproduced in studio

No response

How to reproduce the bug

lightning run model fit \
  --trainer.logger lightning.pytorch.loggers.WandbLogger \
  --trainer.logger.init_args.project test
  --trainer.logger.init_args.save_dir ./wandb_logs
  --trainer.logger.init_args.log_model true
  --trainer.logger.init_args.resume never
  # Note: no id provided

Error messages and logs

# Error messages and logs here please

Environment

Current environment

#- PyTorch Lightning Version (e.g., 2.5.0):
#- PyTorch Version (e.g., 2.5):
#- Python version (e.g., 3.12):
#- OS (e.g., Linux):
#- CUDA/cuDNN version:
#- GPU models and configuration:
#- How you installed Lightning(`conda`, `pip`, source):

More info

No response

cc @lantiga @morganmcg1 @borisdayma @scottire @parambharat @mauvilsa

[SkafteNicki]

Hi @oooooio, thanks for raising this issue. The reason for this seems unrelated to Lightning CLI, but instead is due to the resume argument being hardcoded to "allow":

pytorch-lightning/src/lightning/pytorch/loggers/wandb.py

Lines 339 to 353 in 3726e54

	self._wandb_init: dict[str, Any] = {
	"name": name,
	"project": project,
	"dir": save_dir or dir,
	"id": version or id,
	"resume": "allow",
	"anonymous": ("allow" if anonymous else None),
	}
	self._wandb_init.update(**kwargs)
	# extract parameters
	self._project = self._wandb_init.get("project")
	self._save_dir = self._wandb_init.get("dir")
	self._name = self._wandb_init.get("name")
	self._id = self._wandb_init.get("id")
	self._checkpoint_name = checkpoint_name

[SkafteNicki]

Alright let me correct myself, the self._wandb_init.update(**kwargs) lines does correctly update the resume` argument based on the user provided arguments.

That said I am kind of unable to reproduce the issue as described. This is my testing script:

import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.data import DataLoader, TensorDataset

import lightning as L
from lightning.pytorch.cli import LightningCLI


class SimpleModel(L.LightningModule):
    """Simple model for CLI reproduction of issue #21248."""
    
    def __init__(self, learning_rate: float = 0.01):
        super().__init__()
        self.save_hyperparameters()
        self.layer = nn.Linear(10, 1)
    
    def forward(self, x):
        return self.layer(x)
    
    def training_step(self, batch, batch_idx):
        x, y = batch
        y_hat = self(x)
        loss = F.mse_loss(y_hat, y)
        self.log("train_loss", loss)
        return loss
    
    def configure_optimizers(self):
        return torch.optim.Adam(self.parameters(), lr=self.hparams.learning_rate)

class SimpleDataModule(L.LightningDataModule):
    """Simple data module for CLI reproduction."""
    
    def __init__(self, batch_size: int = 10):
        super().__init__()
        self.batch_size = batch_size
    
    def setup(self, stage: str = None):
        # Create dummy data
        x = torch.randn(100, 10)
        y = torch.randn(100, 1)
        self.dataset = TensorDataset(x, y)
    
    def train_dataloader(self):
        return DataLoader(self.dataset, batch_size=self.batch_size)

if __name__ == "__main__":
    LightningCLI(SimpleModel, SimpleDataModule)

running with

python script.py fit \
  --trainer.logger lightning.pytorch.loggers.WandbLogger \
  --trainer.logger.init_args.project test
  --trainer.logger.init_args.save_dir ./wandb_logs
  --trainer.logger.init_args.log_model true
  --trainer.logger.init_args.resume never

does not trigger the error described in the issue. However, adding something like

--trainer.logger.init_args.version experiment

will trigger the error because now there are conflicting arguments: resume argument indicates not to resume and the version indicates that a experiment should be resumed. @oooooio are you sure you are not implicitly setting the experiment id in some way? can you provide a bit more information to debug the issue?

[oooooio]

@SkafteNicki Yes, I set logger.init_args["version"] = "v1.0.0", but wandb.init doesn’t accept this parameter. Why does this cause the issue?
If I use the logger without specifying a model version, how can I manage my experiment versions? Should I use tags or groups instead?