Train on cpu but gives error "You have asked for native AMP on CPU, but AMP is only available on GPU"

[patricwan]

hi dear friends,
I wanted to train on cpu for debug purpose which can help me understand more about codes.
so in my trainer class, i didn't pass gpus parameter, the parameter looks like below:

args of trainer Namespace(accelerator=None, accumulate_grad_batches=1, adam_epsilon=1e-08, amp_backend='native', amp_level='O2', auto_lr_find=False, auto_scale_batch_size=False, auto_select_gpus=False, batch_size=10, benchmark=False, bert_config_dir='/Users/i052090/Downloads/segmentation/data/bertmany/bert-base-uncased', bert_dropout=0.2, bert_max_length=128, best_dev_f1=0.0, check_val_every_n_epoch=1, checkpoint_callback=True, data_dir='data/conll03', dataname='conll03', default_root_dir='./conll03/spanner_bert-base-uncased_spMLen_usePruneTrue_useSpLenTrue_useSpMorphTrue_SpWtTrue_value0.5_38274488', deterministic=False, distributed_backend=None, fast_dev_run=False, final_div_factor=10000.0, flush_logs_every_n_steps=100, fp_epoch_result='./conll03/spanner_bert-base-uncased_spMLen_usePruneTrue_useSpLenTrue_useSpMorphTrue_SpWtTrue_value0.5_38274488/epoch_results.txt', gpus=None, gradient_clip_algorithm='norm', gradient_clip_val=1.0, label2idx_list=[('O', 0), ('ORG', 1), ('PER', 2), ('LOC', 3), ('MISC', 4)], limit_predict_batches=1.0, limit_test_batches=1.0, limit_train_batches=1.0, limit_val_batches=1.0, log_every_n_steps=50, log_gpu_memory=None, logger=True, lr=1e-05, max_epochs=1, max_spanLen=4, max_steps=None, max_time=None, min_epochs=None, min_steps=None, modelName='spanner_bert-base-uncased_spMLen_usePruneTrue_useSpLenTrue_useSpMorphTrue_SpWtTrue_value0.5', model_dropout=0.2, morph2idx_list=[('isupper', 1), ('islower', 2), ('istitle', 3), ('isdigit', 4), ('other', 5)], morph_emb_dim=100, move_metrics_to_cpu=False, multiple_trainloader_mode='max_size_cycle', n_class=5, neg_span_weight=0.5, num_nodes=1, num_processes=1, num_sanity_val_steps=2, optimizer='adamw', overfit_batches=0.0, param_name='epoch1_batchsize10_lr1e-5_maxlen128', plugins=None, precision=16, prepare_data_per_node=True, pretrained_checkpoint='', process_position=0, profiler=None, progress_bar_refresh_rate=1, random_int='38274488', reload_dataloaders_every_epoch=False, replace_sampler_ddp=True, resume_from_checkpoint=None, spanLen_emb_dim=100, span_combination_mode='x,y', stochastic_weight_avg=False, sync_batchnorm=False, terminate_on_nan=False, tokenLen_emb_dim=50, tpu_cores=None, track_grad_norm=-1, truncated_bptt_steps=None, use_morph=True, use_prune=True, use_spanLen=True, use_span_weight=True, use_tokenLen=False, val_check_interval=0.25, warmup_steps=0, weight_decay=0.01, weights_save_path=None, weights_summary='top', workers=0)

But it would give errors:
File "trainer.py", line 572, in main
trainer = Trainer.from_argparse_args(
File "/Applications/anaconda3/lib/python3.8/site-packages/pytorch_lightning/trainer/properties.py", line 207, in from_argparse_args
return from_argparse_args(cls, args, **kwargs)
File "/Applications/anaconda3/lib/python3.8/site-packages/pytorch_lightning/utilities/argparse.py", line 52, in from_argparse_args
return cls(**trainer_kwargs)
File "/Applications/anaconda3/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/env_vars_connector.py", line 40, in insert_env_defaults
return fn(self, **kwargs)
File "/Applications/anaconda3/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 319, in init
self.accelerator_connector = AcceleratorConnector(
File "/Applications/anaconda3/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/accelerator_connector.py", line 136, in init
self.accelerator = self.select_accelerator()
File "/Applications/anaconda3/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/accelerator_connector.py", line 483, in select_accelerator
precision_plugin=self.precision_plugin,
File "/Applications/anaconda3/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/accelerator_connector.py", line 220, in precision_plugin
self._precision_plugin = self.select_precision_plugin()
File "/Applications/anaconda3/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/accelerator_connector.py", line 350, in select_precision_plugin
raise MisconfigurationException(
pytorch_lightning.utilities.exceptions.MisconfigurationException: You have asked for native AMP on CPU, but AMP is only available on GPU.

thanks,

[carmocca]

Setting Trainer(precision=16) is only supported on GPU!

If you have them available, you can do: Trainer(gpus=N, precision=16)