[MPSFloatType Type Error] Running PyTorch code on Mac MPS, runs fine on TinyVGG model but type error when running efficientnet_b0 model

[gulnuravci]

Hello!

I tried to summarize in the title, but basically I'm running my code on Vscode on Mac and am using Apple's MPS support for PyTorch for faster computing: https://developer.apple.com/metal/pytorch/#:~:text=PyTorch%20uses%20the%20new%20Metal,and%20run%20operations%20on%20Mac..

When I run the TinyVGG model we created in 04. PyTorch Custom Datasets with device = "mps" it works fine (since we correctly send things to the device), but I run into this type error: RuntimeError: Input type (MPSFloatType) and weight type (torch.FloatTensor) should be the same when running the transfer learning code we discussed in 07. PyTorch Transfer Learning. I'm aware something isn't on the correct device, but I couldn't figure out what it is based on google search and looking at PyTorch documentation. Would appreciate any advice!

[mrdbourke]

Hey @gulnuravci ,

Did you setup your device to be "mps"? (it looks like you have already done this)

You may have to set this up at the start to make sure your objects all use the target device.

For example, see here:

• PyTorch documentation for MPS device - https://pytorch.org/docs/stable/notes/mps.html
• My guide on setting up PyTorch for MPS - https://github.com/mrdbourke/pytorch-apple-silicon

When does the error happen?

When you run model.to(device) or something else?

--

Note: If you're using PyTorch 2.0+, you can globally set devices, see: https://www.learnpytorch.io/pytorch_2_intro/#11-globally-set-devices

For example (for "cuda"):

import torch

# Set the device
device = "cuda" if torch.cuda.is_available() else "cpu"

# Set the device globally
torch.set_default_device(device)

# All tensors created will be on the global device by default
layer = torch.nn.Linear(20, 30)
print(f"Layer weights are on device: {layer.weight.device}")
print(f"Layer creating data on device: {layer(torch.randn(128, 20)).device}")

Layer weights are on device: cuda:0
Layer creating data on device: cuda:0

Also see the PyTorch documentation for more: https://pytorch.org/tutorials/recipes/recipes/changing_default_device.html

You could try this with "mps" and see if it helps, otherwise it may be an issue with the efficientnet_b0 architecture and having incompatible operations with the MPS backend (not all operations are compatible with the MPS backend yet).

[gulnuravci]

Hi @mrdbourke,

I did some more debugging with your suggestions and here's what I found:

1. I had already tried with my device set to "mps" as they describe in the documentation and sending the appropriate items to device, but I hadn't tried to set the device globally. When I did do that, I run into a "Expected a 'mps:0' generator device but found 'cpu'" error, which didn't go away when I added an extra parameter generator=torch.Generator(device=device) (with device being "mps") when creating the data loaders, but did go away when I turned shuffle off shuffle=False when creating the training dataloader.

2. (Regardless of the generator error) When I step through the code in debugging mode, "Input type (MPSFloatType) and weight type (torch.FloatTensor) should be the same" error specifically occurs when I try a forward pass with the model y_pred = model(X). I tried to keep an eye out on datatypes in the debugging window and what device X and y were on, and they all seem to be on "mps", which is why I'm also thinking it might be something with the efficientnet_b0 architecture (though I'm still not 100% sure).