I have three questions to consult, thank you~

[qingyun1988]

1. A token includes "pos" and "tag" attributes, can they both be trained or only "pos" can be trained? if so, why is there only "pos" but no "tag" demo in 'prody.gy/demo‘?
2. There is only "pos" in 'prody.gy/demo‘, and there is only "tag" in "json trainng format" section of page "https://spacy.io/api/data-formats#json-input", why?
3. Can an existing transform model be trained or only the sm/lg model can be trained?

Thank you very much!

[svlandeg]

Hi!

I think a lot of the confusion is coming from the fact that in spaCy v2.x, the tagger would assign both token.tag and token.pos, while it only assigns token.tag since v3.0. The migration guide specifies how to use the new attribute_ruler to obtain token.pos annotations.

The current release and documentation of Prodigy focuses on Prodigy v.10, which is compatible with spaCy 2. We're working on Prodigy v.11, with updated tagging recipes, to be compatible with spaCy v3, but this is not officially released yet.

Can an existing transform model be trained or only the sm/lg model can be trained?

Yep - you should be able to fine-tune an existing transformer-based pipeline as you would with a "normal" pretrained pipeline.

[adrianeboyd]

In spacy v2 using the built-in components you can only train a model for tag, not pos, but the prodigy v10 recipes are called pos, which is a bit confusing. See some notes about the pos recipes if you're extending an existing tagger: #6283 (comment)

[qingyun1988]

I have just seen your reply because I haven't logged on to this website recently. Thanks for your help and reply.

[qingyun1988]

Make a detailed inquiry.

it only assigns token.tag since v3.0

I would like to know if the token.tag in v3.0 is equal to token.tag or token.pos in v2.0? In other words, are the tags of token.tag in v3.0 'VBD,VBG,VBN...etc' or 'ADJ,NOUN,VERB...etc'? thank you!

[svlandeg]

The token.tag_ semantics have remained largely the same.

FYI - for any given pretrained pipeline, you can find the list of tags in the docs in the expandable section "Label Scheme", cf for instance https://spacy.io/models/en and you'll see the tag labels under tagger.

[qingyun1988]

Thank you very much! I just saw this reply from you after you reminded me in that post.