TensorBoard logger step is inconsistent with trainer.gloabel_step under manual-optimization

[jim4399266]

I recently want to combine adversarial learning(FGM and AWP) with Lightning frame, so I found a tutorial and changed my module to manual-optimization. When I checked logs in TensorBoard I found the steps are 4 times as many as that not under manual-optimization( my accumulate_grad_batches = 4 ).
As the pic shows, , oringe line is common method and green line is manual-optimization, I checked the trainer.globel_step and they were all the same in both method. How to fix this problem? Thanks a lot!

Following is my TensorBoardLooger settings:

logger = TensorBoardLogger(save_dir=config['output_dir'],
                               name=f"from_{config['model'].split('/')[-1]}/")

Following is my Trainer settings:

trainer = pl.Trainer(
        gpus=config['gpus'],
        strategy=None,
        replace_sampler_ddp=False,
        logger=logger,
        log_every_n_steps=10,
        amp_backend="native",
        amp_level=None,
        precision=32,
        max_epochs=config['max_epochs'],
        callbacks=callbacks,
        gradient_clip_val=None,
        accumulate_grad_batches=None,
        val_check_interval=config['val_check_interval'],
    )

Following is my training_step function:

def training_step(self, batch, batch_idx):
        inputs, labels = batch  
        y_preds = self.forward(inputs).float()  
        loss = self.criterion(y_preds, labels)  
        self.log('train/loss', loss )  
        if self.hparams.config['gradient_accumulation_steps'] > 1:
            loss = loss / self.hparams.config['gradient_accumulation_steps']  # gradient_accumulation_steps = 4
        self.manual_backward(loss)
        # AWP
        #self.awp.attack_backward(inputs, labels, self.trainer.current_epoch)
        torch.nn.utils.clip_grad_norm(self.parameters(), self.hparams.config['max_grad_norm'])
        if (batch_idx + 1) % self.hparams.config['gradient_accumulation_steps'] == 0:
            opt = self.optimizers()
            opt.step()
            opt.zero_grad()
            sch = self.lr_schedulers()
            sch.step()
        return loss