Spacy training format

[JonathanBoyd96]

I have a working program using spacy to analyze a pdf file and pull data from it using the base en_core_web_sm model. Now I want to train my own model to be more accurate and add specifics. I have got a model working and use python to create a training model. However, the json data im attempting to use seems to have issues properly being interpreted.

Here is example json training data:

[
    {
        "text": "Lisa Wu, 435 Maple Lane Apt 203, Salt Lake City, UT 84123",
        "entities": [
            [0, 8, "NAME"],
            [9, 31, "ADDRESS"],
            [32, 50, "CITY"],
            [51, 53, "STATE"],
            [54, 60, "ZIP"]
        ]
    },
    {
        "text": "Erik Adams, 555 Birch Road Suite 404, Columbus, OH 43215",
        "entities": [
            [0, 11, "NAME"],
            [12, 38, "ADDRESS"],
            [39, 58, "CITY"],
            [59, 61, "STATE"],
            [62, 68, "ZIP"]
        ]
    }
]

When I attempt to train using this json data I get the following errors:

/opt/anaconda3/lib/python3.12/site-packages/spacy/training/iob_utils.py:149: UserWarning: [W030] Some entities could not be aligned in the text "Lisa Wu, 435 Maple Lane Apt 203, Salt Lake City, U..." with entities "[[0, 8, 'NAME'], [9, 31, 'ADDRESS'], [32, 50, 'CIT...". Use `spacy.training.offsets_to_biluo_tags(nlp.make_doc(text), entities)` to check the alignment. Misaligned entities ('-') will be ignored during training.
  warnings.warn(
Text: Lisa Wu, 435 Maple Lane Apt 203, Salt Lake City, UT 84123
Entities: [[0, 8, 'NAME'], [9, 31, 'ADDRESS'], [32, 50, 'CITY'], [51, 53, 'STATE'], [54, 60, 'ZIP']]
BILUO Tags: ['B-NAME', 'I-NAME', 'L-NAME', 'B-ADDRESS', 'I-ADDRESS', 'I-ADDRESS', 'I-ADDRESS', 'L-ADDRESS', 'O', '-', '-', '-', '-', '-', '-']
----------
/opt/anaconda3/lib/python3.12/site-packages/spacy/training/iob_utils.py:149: UserWarning: [W030] Some entities could not be aligned in the text "Erik Adams, 555 Birch Road Suite 404, Columbus, OH..." with entities "[[0, 11, 'NAME'], [12, 38, 'ADDRESS'], [39, 58, 'C...". Use `spacy.training.offsets_to_biluo_tags(nlp.make_doc(text), entities)` to check the alignment. Misaligned entities ('-') will be ignored during training.
  warnings.warn(
Text: Erik Adams, 555 Birch Road Suite 404, Columbus, OH 43215
Entities: [[0, 11, 'NAME'], [12, 38, 'ADDRESS'], [39, 58, 'CITY'], [59, 61, 'STATE'], [62, 68, 'ZIP']]
BILUO Tags: ['B-NAME', 'I-NAME', 'L-NAME', '-', '-', '-', '-', '-', '-', '-', '-', '-', '-']

I have attempted to use the spacy DocBin process to convert the json data into a .spacy file but I had no luck there either because for some reason DocBin doesnt seem to be in the spacy.trainer library for 3.8.2?? I could use help there as well if that is an easier solution.

[honnibal]

The DocBin class is at from spacy.tokens import DocBin

[JonathanBoyd96]

Thanks!