docs_to_json compatibility with spacy 3.0

[EY4L]

Will this function get updated to include doc.spans['sc'] and generally be compatible with spacy 3.0?

[ljvmiranda921]

Hi @EY4L, since docs_to_json generates the v2 training format, we don't have any plans of updating this anymore. If you're creating training data from the text themselves, you might be better off interacting with the span attributes rather than using this function. I'm curious as to why you want to use JSON? Is there anything specific in this format that you need in your problem?

[EY4L]

Hi @ljvmiranda921, JSON is useful as it's a very readable storage format to look at training data before converting to spacy binary training format.

I have added the span categories by accessing the doc.spans['sc'] group and I'm training an NER component and SpanCat component on different entities respectively.

If I wanted to change docs_to_json to work with SpanCat, what attributes of the SpanGroup should I add to this function?

[ljvmiranda921]

Hi @EY4L , if it's for readability, you should be able to use doc.to_json() (note that this is a method under the Doc object, not some standalone utility). This should give you a dictionary / JSON-like representation of your Doc. However, this format cannot be used right away for training (it's not the same as the JSONL format back in v2), so we'll still use the spaCy binary for training in v3, or read it via doc.from_json() then store as DocBin than using spacy convert