torch.tensor and numpy.array do not share memory, do they?

[sh-am-si]

Following your lecture I did something like this

tn = torch.tensor([1,2,3])
ar = tn.numpy()
tn, ar

Output
(tensor([1, 2, 3]), array([1, 2, 3]))

tn = tn + 1
tn, ar

Output
(tensor([2, 3, 4]), array([1, 2, 3]))
It is clear as we created a new object tn, but with the old name.
However, if slightly modify the statement

tn += 1
tn, ar

Output
(tensor([2, 3, 4]), array([2, 3, 4]))

Now, let's try modify one particular item

tn[0] = -10
tn, ar

Output
(tensor([-10, 2, 3]), array([-10, 2, 3]))

Briefly, methinks, they share memory.

[Perian-Yan]

Hi,
tn and ar actually share the memory (if force=False by default), and changes on one will be reflected on the other. See https://pytorch.org/docs/stable/generated/torch.Tensor.numpy.html

I think what makes you confuse is the difference between tn += 1 and tn = tn+1. The former one will do an in-place increment on tn, while the latter one, as you said, is reassigning to a new variable with the old name. See this thread for details https://stackoverflow.com/questions/41446833/what-is-the-difference-between-i-i-1-and-i-1-in-a-for-loop

Hope this helps.

[sh-am-si]

It is pretty interesting

with torch.no_grad():
    tn = torch.tensor([1., 2., 3.])
ar1 = tn.detach().cpu().resolve_conj().resolve_neg().numpy()
ar2 = tn.numpy(force=True)
ar3 = tn.clone().numpy(force=False)
print(tn, ar1, ar2, ar3)
tn += 1
print(tn, ar1, ar2, ar3)

Output:
tensor([1., 2., 3.]) [1. 2. 3.] [1. 2. 3.] [1. 2. 3.]
tensor([2., 3., 4.]) [2. 3. 4.] [2. 3. 4.] [1. 2. 3.]

I couldn't find a way to detach the array from tensor, unless explicitly calling the clone function. Than I noticed in the documentation this:

if True, the ndarray may be a copy of the tensor instead of always sharing memory, defaults to False.

To be honest, I don't remember using words may be in docs

[Perian-Yan]

That's really interesting :D
If you move tn to 'cuda', then force=True will forcibly convert a tensor on gpu to a numpy array. As written in the doc: If the tensor isn’t on the CPU or the conjugate or negative bit is set, the tensor won’t share its storage with the returned array.

import torch
import numpy as np

device = 'cuda' if torch.cuda.is_available() else 'cpu'

with torch.no_grad():
    tn = torch.tensor([1., 2., 3.],device=device)
ar1 = tn.detach().cpu().resolve_conj().resolve_neg().numpy()
ar2 = tn.numpy(force=True)

# ar3 = tn.detach().cpu().clone().numpy(force=False)
print(tn, ar1, ar2)
tn += 1
print(tn, ar1, ar2)

Output:
tensor([1., 2., 3.], device='cuda:0') [1. 2. 3.] [1. 2. 3.]
tensor([2., 3., 4.], device='cuda:0') [1. 2. 3.] [1. 2. 3.]