[ITensors] [BUG] Elementwise operations/Broadcasting bug

[kmp5VT]

Description of bug
Broadcast does not work for elementwise operations on ITensor. This operation should also be covered by hadamard_product so maybe we should forward the operation to that function?

Minimal runnable code

julia> m = random_itensor(Index.((5,6)))
julia> m .* m

ERROR: The noncommon indices of ((dim=5|id=769), (dim=6|id=415)) and ((dim=5|id=769), (dim=6|id=415)) must be the same as the indices ((dim=5|id=769), (dim=6|id=415)).
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compute_contraction_labels(Cis::Tuple{…}, Ais::Tuple{…}, Bis::Tuple{…})
   @ ITensors ~/.julia/packages/ITensors/VGXV2/src/indexset.jl:715
 [3] _contract!!(C::NDTensors.DenseTensor{…}, A::NDTensors.DenseTensor{…}, B::NDTensors.DenseTensor{…})
   @ ITensors ~/.julia/packages/ITensors/VGXV2/src/tensor_operations/tensor_algebra.jl:172
 [4] contract!(C::ITensor, A::ITensor, B::ITensor)
   @ ITensors ~/.julia/packages/ITensors/VGXV2/src/tensor_operations/tensor_algebra.jl:181
 [5] mul!
   @ ~/.julia/packages/ITensors/VGXV2/src/itensor.jl:1857 [inlined]
 [6] copyto!
   @ ~/.julia/packages/ITensors/VGXV2/src/broadcast.jl:470 [inlined]
 [7] copy
   @ ./broadcast.jl:897 [inlined]
 [8] materialize(bc::Base.Broadcast.Broadcasted{ITensors.ITensorStyle, Nothing, typeof(*), Tuple{ITensor, ITensor}})
   @ Base.Broadcast ./broadcast.jl:872
 [9] top-level scope
   @ ~/.julia/dev/ITensorCPD/test/runtests.jl:18
Some type information was truncated. Use `show(err)` to see complete types.

I think this might also be a bug

m .+ m
ERROR: When adding two ITensors in-place, one must be the same as the output ITensor
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] copyto!
   @ ~/.julia/packages/ITensors/VGXV2/src/broadcast.jl:308 [inlined]
 [3] copy
   @ ./broadcast.jl:897 [inlined]
 [4] materialize(bc::Base.Broadcast.Broadcasted{ITensors.ITensorStyle, Nothing, typeof(+), Tuple{ITensor, ITensor}})
   @ Base.Broadcast ./broadcast.jl:872
 [5] top-level scope
   @ ~/.julia/dev/ITensorCPD/test/runtests.jl:18

[mtfishman]

The first one is deliberate, but in retrospect was a mistake. I defined a .* b for ITensors as contraction so we could use c .= a .* b as a shorthand for in-place tensor contraction, i.e. it is a shorthand for contract!(c, a, b). As you point out, that goes against the conventional usage of broadcasting syntax to mean elementwise operations, so I think it wasn't a good idea to do that. In NamedDimsArrays.jl a .* b means a Hadamard product, so that issue will be fixed once that is released and used in the next major release of ITensors.jl. For now you can use hadamard_product[!] or ⊙.

Probably in future versions we will provide a syntax like @! c = a * b for in-place contraction, similar to https://github.com/simonbyrne/InplaceOps.jl.

The second one is an unfortunate limitation which originates from the design of how ITensor broadcast expressions are lowered to expressions involving permutedims in NDTensors, I've tried fixing that at various points but it is a lot of work given the current design (the assumption described in the error message is really hardcoded in many places in NDTensors). NamedDimsArrays handles broadcasting in a much saner way so that limitation is already fixed there, but for now you can always just call the non-broadcast version.