"debug data" comparison of train/dev labels seems too harsh

[arc12]

How to reproduce the behaviour

python -m spacy debug data ... used when the "dev" data does not contain all the labels present in the "train" data causes an error report: "The train and dev labels are not the same."

Your Environment

Spacy v3

Expected behaviour

Spacy tolerates mismatch of some labels, especially missing labels in "dev", and only issues a warning debug message.
If the labels are disjoint, report an error.

[svlandeg]

Hi! As a maintenance note: this is a good question for the discussions forum, so I'll move it there. The original issue will be closed and will link through to the open discussion thread.

[adrianeboyd]

We updated some of the textcat analysis in debug data and this error will be changed to a warning in the next patch release, v3.0.7.