Spacy 3.0 - nlp.add_pipe(ruler) failes #6265

AlexSchmidke · 2020-10-17T13:55:10Z

AlexSchmidke
Oct 17, 2020

Hi,

we are trying to use the EntityRuler with the new spacy nightly version. The Code word using the old spacy version but fails at the nlp.add_pipe(ruler) step using spacy nightly 3.0.

The Code (just the example from the spacy website):

import spacy
from spacy.pipeline import EntityRuler

nlp = spacy.load("en_core_web_trf")
ruler = EntityRuler(nlp)
ruler.add_patterns([{"label": "ORG", "pattern": "Apple"}])
nlp.add_pipe(ruler)

The same Code runs with the Old Spacy Version (Using the Old Language Set) but fails with following error using spacy nightly

ValueError: [E966] nlp.add_pipe now takes the string name of the registered component factory, not a callable component. Expected string, but got <spacy.pipeline.entityruler.EntityRuler object at 0x7f6237e88640> (name: 'None').

If you created your component with nlp.create_pipe('name'): remove nlp.create_pipe and call nlp.add_pipe('name') instead.
If you passed in a component like TextCategorizer(): call nlp.add_pipe with the string name instead, e.g. nlp.add_pipe('textcat').
If you're using a custom component: Add the decorator @Language.component (for function components) or @Language.factory (for class components / factories) to your custom component and assign it a name, e.g. @Language.component('your_name'). You can then run nlp.add_pipe('your_name') to add it to the pipeline.

My Environment

copy-paste the result here.-->

Operating System: Ubuntu 18.04
Python Version Used: 3.7
spaCy Version Used: 3.0
Environment Information:

Answered by honnibal

Oct 17, 2020

Yes this is one of the workflows that's changed in v3. We've tried to make the error message as helpful as possible. If you have a look at the error you pasted, is it clear to you what needs to be done? I thought the wording there was quite good, but I understand it's always hard to guess how things come across to users. You can also find this covered in the migration guide: https://nightly.spacy.io/usage/v3#migrating-add-pipe

View full answer

honnibal · 2020-10-17T14:39:56Z

honnibal
Oct 17, 2020
Maintainer

Yes this is one of the workflows that's changed in v3. We've tried to make the error message as helpful as possible. If you have a look at the error you pasted, is it clear to you what needs to be done? I thought the wording there was quite good, but I understand it's always hard to guess how things come across to users. You can also find this covered in the migration guide: https://nightly.spacy.io/usage/v3#migrating-add-pipe

0 replies

adrianeboyd · 2020-10-19T06:55:53Z

adrianeboyd
Oct 19, 2020

Maybe we could provide the string name in the error message for the built-in components in case two? I think a lot of the names aren't obvious from the class names...

0 replies

Giyn · 2021-02-07T06:38:47Z

Giyn
Feb 7, 2021

I also encountered the same problem when using spacy 3.0. Have you found an effective solution now?

2 replies

adrianeboyd Feb 8, 2021

The solution is in the migration guide, however the "string name" part still could be made clearer for users. It would look like this instead:

ruler = nlp.add_pipe("entity_ruler")
ruler.add_patterns(...)

See the examples in the sidebar here: https://spacy.io/api/entityruler

danilo-css Dec 7, 2023

@adrianeboyd you're a life saver. Thanks. Adding to your contribution, I just wanna say this worked for me:

patterns = [
    {"label": "MONEY", "pattern": [{"LOWER": "us$"}, {"LIKE_NUM": True}]},
]

# Add patterns to ruler
ruler.add_patterns(patterns)

# Add the entity ruler to the pipeline
nlp.add_pipe('entity_ruler')
nlp.get_pipe('entity_ruler').add_patterns(patterns)

rohan2734 · 2021-02-10T03:10:44Z

rohan2734
Feb 10, 2021

It still doesnt work for me, It shows the same error
and Entity ruler also doesnt help me

#add a sengmentation rule

# @Language.component('set_custom_boundaries')
def set_custom_boundaries(doc):
    doc.is_parsed = True
#     for token in doc:
    for token in doc[:-1]:
#         print(token)
#         print(token.i)
        if token.text == ';':
            doc[token.i+1].is_sent_start = True
    return doc
        
# set_custom_boundaires(doc)

# #
# @Language.component("set_custom_boundaries")

nlp.add_pipe(set_custom_boundaries,before='parser')

nlp.pipe_names

6 replies

rohan2734 Feb 10, 2021

I have even tried that too sir,
this is the code

@Language.component("set_custom_boundaries")

nlp.add_pipe("set_custom_boundaries",before='parser')

nlp.pipe_names

this is the error

  F File "<ipython-input-2-d455df790d88>", line 4
    nlp.add_pipe("set_custom_boundaries",before='parser')
    ^
SyntaxError: invalid syntax

I even added the doc.parser=True, and i tried it too,but it is not working.

adrianeboyd Feb 10, 2021

Please take a look at the docs and examples here for examples of how to use the component decorator: https://spacy.io/usage/processing-pipelines#custom-components

rohan2734 Feb 10, 2021

Hey thankyou it is working now

leighton155 Feb 11, 2021

What about LanguageDetector? https://spacy.io/universe/project/spacy-langdetect

The online documentation still says to use nlp.add_pipe(LanguageDetector(), name='language_detector', last=True). Then the error shows nlp.add_pipe now takes the string name of the registered component factory, not a callable component.

If I try nlp.add_pipe('langdetect') the error says

ValueError: [E002] Can't find factory for 'langdetect' for language English (en). This usually happens when spaCy calls nlp.create_pipe with a custom component name that's not registered on the current language class. If you're using a Transformer, make sure to install 'spacy-transformers'. If you're using a custom component, make sure you've added the decorator @Language.component (for function components) or @Language.factory (for class components).

Available factories: attribute_ruler, tok2vec, merge_noun_chunks, merge_entities, merge_subtokens, token_splitter, parser, beam_parser, entity_linker, ner, beam_ner, entity_ruler, lemmatizer, tagger, morphologizer, senter, sentencizer, textcat, textcat_multilabel, es.lemmatizer, en.lemmatizer

It doesn't appear that Language Detector is an available factory.

adidonatocda May 17, 2021

Not sure if you are still having this issue in Spacy 3.x , I ran into a similar issue and fixed it by adding the following imports:

from spacy.language import Language

@Language.component("my_component")
def my_component(doc):
   # Do something to the doc here
   return doc

This is outlined in the online docs: https://spacy.io/usage/processing-pipelines#custom-components

tushitdave · 2021-06-23T10:54:08Z

tushitdave
Jun 23, 2021

This is also one of the errors when you bring language factory my component. Still thinking how to resolve this but no valid sources so far
TypeError: module, class, method, function, traceback, frame, or code object was expected, got cython_function_or_method

8 replies

polm Jun 23, 2021

OK, thanks for the details. Please read the Markdown guide and use the Preview function to make sure your post is readable.

I'm not sure how you're defining your component but I guess it's a Cython cdef class or possibly function? You need to make it a Python object/function, like the error says. Can you show the code for MyComponent?

Also please upgrade spaCy to the latest instead of 3.0.0 - we've added several bugfixes since then.

polm Jun 23, 2021

Wait. I'm confused now. Based on the stack trace, the error is happening in code you commented out in your code sample. What code are you actually running?

#set up the pipeline
@Language.component("ner")
def ner(train_data):
    return train_data

You probably shouldn't name a component "ner" as that might conflict with the existing pipelines. You also have a separate, unrelated ner variable later in your code, so that function name should be changed (if you're actually using that code). Also, since this is a stateless component, the argument for it is a Doc, not training data. See the docs.

tushitdave Jun 30, 2021

Hi , This is in reference to the below code, I am unable to create custom pipeline(input and output both are presented in the code).
Input:-
def add_new_ruler(skill_pattern_path):
new_ent_ruler = EntityRuler(nlp).from_disk(skill_pattern_path)
new_ent_ruler = nlp.create_pipe('new_ent_ruler',new_ent_ruler)
nlp.add_pipe(new_ent_ruler, after='parser' , before ='ner')

print(nlp.pipe_names)

output:-
spacy pipeline issue
['tok2vec', 'tagger', 'parser', 'ner', 'attribute_ruler', 'lemmatizer']

python version 3.8.5
spacy version: 3.0.0
windows:10

tushitdave Jun 30, 2021

@polm - I am able to successfully run the code however I can't see my customized pipeline after the parser and before the NER. Is there anything I am missing here

svlandeg Jun 30, 2021

I only see this now, but we can continue the discussion in this specific thread you opened, to avoid duplicating stuff: #8550

Uh oh!

Spacy 3.0 - nlp.add_pipe(ruler) failes #6265

Uh oh!

Uh oh!

My Environment

Name Version Build Channel

Replies: 5 comments · 16 replies

Uh oh!

honnibal Oct 17, 2020 Maintainer

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Replies: 5 comments 16 replies

honnibal
Oct 17, 2020
Maintainer