Seems that Doc.from_docs does not preserve Doc.tensor

[omarbenhamid]

Hello,

Thanks for the quality open source library for NLP.

It seems that Doc.from_docs does not preserve Doc.tensor (currently for any span) and documentation tells nothing about it.

I was about to create an issue but I am wondering if it is a bug, a feature request, or more a documentation update ?

Tensor can be the concatenation of tensors of all sub document ? On the other hand the tensor shape is maybe something specific to the embedding used ...

[adrianeboyd]

Thanks for the note, it is the case that .tensor is not included. I'd be a little worried about adding it now by default because it will really bloat the final documents in ways that current users might not expect, but we'll discuss it internally and decided what to do, probably either more documentation or an option to exclude as for doc.to_bytes.

[omarbenhamid]

Ok perfect. Waiting for your decision then.

For information, this causes of issue #9 of spacypdfreader so it will probably be important to know how to fix it

@SamEdwardes

[adrianeboyd]

Okay, let's support exclude: #10689