Can't update en_core_web_trf model

[DmitriiTrofimov]

spacy == 3.2.1
en_core_web_trf == 3.2.0

I try to update en_core_web_trf model to adjust dependency parser.

I try to do it in this way

    nlp = spacy.load("en_core_web_trf")
    redoc = json_to_annotations({
  "id": 0,
  "paragraphs": [
    {
      "raw": "Was a shirt being washed by her.",
      "sentences": [
        {
          "tokens": [
            {
              "id": 0,
              "orth": "Was",
              "space": " ",
              "tag": "VBD",
              "pos": "AUX",
              "morph": "Mood=Ind|Number=Sing|Person=3|Tense=Past|VerbForm=Fin",
              "lemma": "be",
              "head": 4,
              "dep": "aux",
              "ner": "O"
            },
            {
              "id": 1,
              "orth": "a",
              "space": " ",
              "tag": "DT",
              "pos": "DET",
              "morph": "Definite=Ind|PronType=Art",
              "lemma": "a",
              "head": 1,
              "dep": "det",
              "ner": "O"
            },
            {
              "id": 2,
              "orth": "shirt",
              "space": " ",
              "tag": "NN",
              "pos": "NOUN",
              "morph": "Number=Sing",
              "lemma": "shirt",
              "head": 2,
              "dep": "nsubjpass",
              "ner": "O"
            },
            {
              "id": 3,
              "orth": "being",
              "space": " ",
              "tag": "VBG",
              "pos": "AUX",
              "morph": "VerbForm=Ger",
              "lemma": "be",
              "head": 1,
              "dep": "auxpass",
              "ner": "O"
            },
            {
              "id": 4,
              "orth": "washed",
              "space": " ",
              "tag": "VBN",
              "pos": "VERB",
              "morph": "Aspect=Perf|Tense=Past|VerbForm=Part",
              "lemma": "wash",
              "head": 0,
              "dep": "ROOT",
              "ner": "O"
            },
            {
              "id": 5,
              "orth": "by",
              "space": " ",
              "tag": "IN",
              "pos": "ADP",
              "morph": "",
              "lemma": "by",
              "head": -1,
              "dep": "agent",
              "ner": "O"
            },
            {
              "id": 6,
              "orth": "her",
              "space": "",
              "tag": "PRP",
              "pos": "PRON",
              "morph": "Case=Acc|Gender=Fem|Number=Sing|Person=3|PronType=Prs",
              "lemma": "she",
              "head": -1,
              "dep": "pobj",
              "ner": "O"
            },
            {
              "id": 7,
              "orth": ".",
              "space": "",
              "tag": ".",
              "pos": "PUNCT",
              "morph": "PunctType=Peri",
              "lemma": ".",
              "head": -3,
              "dep": "punct",
              "ner": "O"
            }
          ],
          "brackets": []
        }
      ],
      "cats": [],
      "entities": [],
      "links": []
    },
    {
      "raw": "Is a mango being eaten by her.",
      "sentences": [
        {
          "tokens": [
            {
              "id": 0,
              "orth": "Is",
              "space": " ",
              "tag": "VBZ",
              "pos": "AUX",
              "morph": "Mood=Ind|Number=Sing|Person=3|Tense=Pres|VerbForm=Fin",
              "lemma": "be",
              "head": 4,
              "dep": "aux",
              "ner": "O"
            },
            {
              "id": 1,
              "orth": "a",
              "space": " ",
              "tag": "DT",
              "pos": "DET",
              "morph": "Definite=Ind|PronType=Art",
              "lemma": "a",
              "head": 1,
              "dep": "det",
              "ner": "O"
            },
            {
              "id": 2,
              "orth": "mango",
              "space": " ",
              "tag": "NN",
              "pos": "NOUN",
              "morph": "Number=Sing",
              "lemma": "mango",
              "head": 2,
              "dep": "nsubjpass",
              "ner": "O"
            },
            {
              "id": 3,
              "orth": "being",
              "space": " ",
              "tag": "VBG",
              "pos": "AUX",
              "morph": "VerbForm=Ger",
              "lemma": "be",
              "head": 1,
              "dep": "auxpass",
              "ner": "O"
            },
            {
              "id": 4,
              "orth": "eaten",
              "space": " ",
              "tag": "VBN",
              "pos": "VERB",
              "morph": "Aspect=Perf|Tense=Past|VerbForm=Part",
              "lemma": "eat",
              "head": 0,
              "dep": "ROOT",
              "ner": "O"
            },
            {
              "id": 5,
              "orth": "by",
              "space": " ",
              "tag": "IN",
              "pos": "ADP",
              "morph": "",
              "lemma": "by",
              "head": -1,
              "dep": "agent",
              "ner": "O"
            },
            {
              "id": 6,
              "orth": "her",
              "space": "",
              "tag": "PRP",
              "pos": "PRON",
              "morph": "Case=Acc|Gender=Fem|Number=Sing|Person=3|PronType=Prs",
              "lemma": "she",
              "head": -1,
              "dep": "pobj",
              "ner": "O"
            },
            {
              "id": 7,
              "orth": ".",
              "space": "",
              "tag": ".",
              "pos": "PUNCT",
              "morph": "PunctType=Peri",
              "lemma": ".",
              "head": -3,
              "dep": "punct",
              "ner": "O"
            }
          ],
          "brackets": []
        }
      ],
      "cats": [],
      "entities": [],
      "links": []
    }
  ]
})
    examples = []
    for item in redoc:
        doc = nlp.make_doc(item["text"])
        examples.append(Example.from_dict(doc, item))

    optimizer = nlp.resume_training()
    losses = {}
    for itn in range(100):
        random.shuffle(examples)
        nlp.update(examples, sgd=optimizer, exclude=('tagger', 'ner'), losses=losses, drop=0.2)

Then I get absolutely the same dependencies like without any adjusts. But it works with en_core_web_lg model. What do I do wrong?

[DmitriiTrofimov]

I think I have too little data here to retrain parser. In any case, attempts to update the parser in this case give too inaccurate results.

[polm]

It's true you don't have enough data. You also need to adjust the learning rate - Transformers need a different setting. See here for someone who tried something similar.

In general tuning the parser like this is going to be difficult though.

[DmitriiTrofimov]

Thanks a lot. It's very helpful.