Change CNN window size in entity recognizer

[raghuapce]

@honnibal

The below links are not working. I would like to increase the CNN window size from 4 to 6 in entity recognizer in spacy v2 (2.2.4). Also, It would be great if you could document the working of Spacy entity recognizer in detail.

There's two places you could change this. One is the window size in the entity recognizer's CNN: https://github.com/explosion/spaCy/blob/master/spacy/_ml.py#L242 . The default setting is depth 4, which means the token's vector is sensitive to information from 4 words on either side.

The other place to change this is in the extraction of state tokens. I'm preparing something that explains this part of the procedure better. Implementation: https://github.com/explosion/spaCy/blob/master/spacy/syntax/_state.pxd#L130

This is not a part of the implementation I'm proud of... The entity recognizer uses the "6' case. You'll need to change the nr_features constant in EntityRecognizer within pipeline.pyx if you add more features (yeah, I know...). The features themselves aren't that satisfying for the entity recognizer. It feels like we should be able to make better use of the entity history.

Originally posted by @honnibal in #1532 (comment)

[polm]

That answer is from Nov 2017, four years ago, just after the release of v2.0. The links go to the master branch without any particular commit, so they don't work because those files are gone. You can use them as a starting point to find the relevant code by going back in the history if you want.

In v3 the window size is exposed as a config parameter and easy to change if you want.

Also, It would be great if you could document the working of Spacy entity recognizer in detail.

We are working on documenting internals more, and I don't think the EntityRecognizer has changed that much since v2.2, but I don't think we're interested in going back and providing rich documentation for previous versions now that v3 has been out for a while.

[raghuapce]

In v3, I am searching for the parameter to increase the window size in entity recognizer. Could you help me to find out the same? Also, I assume the parameter should be included in the config file.

[polm]

Window size is a parameter of the tok2vec, not the NER component directly. It is in the config as window_size.

[raghuapce]

Thanks polm. But I have a doubt on calculating number of tokens on either side to concatenate during convolutions.

In the spacy documentation, The description for window_szie is "The number of tokens on either side to concatenate during the convolutions. The receptive field of the CNN will be depth * (window_size * 2 + 1), so a 4-layer network with a window size of 2 will be sensitive to 17 words at a time."

In the above example, after applying the values, the formula looks like 4 * (2 * 2 + 1) and it returns 20. But If I put up like 4 * (2 * 2) + 1, then it returns 17. Please confirm which one of this is correct way of calculation?

[polm]

I'm not really sure what's going on with that formula in the docs, and I'll look into it, but you can ignore it for now - it's probably just a mistake and should be 20. Regardless of depth window size has a pretty simple definition and works as you'd expect.