AttributeError: [E046] Can't retrieve unregistered extension attribute 'phrases'. Did you forget to call the set_extension method?

[fabmeyer]

Traceback (most recent call last):
  File "analysis/LanguageModel/LanguageModel.py", line 78, in <module>
    language_model('artificia intelligenc', model)
  File "analysis/LanguageModel/LanguageModel.py", line 71, in language_model
    sentences = extract_sentences(' '.join(definitions), model, number=3)
  File "/home/fabmeyer/Dev/Python/analytics/analysis/LanguageModel/SentenceExtractor.py", line 12, in extract_sentences
    for phrase in doc._.phrases:
  File "/home/fabmeyer/.local/share/virtualenvs/analytics-nko8goOs/lib/python3.8/site-packages/spacy/tokens/underscore.py", line 47, in __getattr__
    raise AttributeError(Errors.E046.format(name=name))
AttributeError: [E046] Can't retrieve unregistered extension attribute 'phrases'. Did you forget to call the `set_extension` method?

Info about spaCy

• spaCy version: 3.2.4
• Platform: Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.29
• Python version: 3.8.0
• Pipelines: en_use_lg (0.4.3), en_core_web_lg (3.2.0), en_core_web_sm (3.2.0), en_core_web_trf (3.2.0)

Some weeks ago I wrote (copy pasted) a script that does extract the most important sentences. Now I am getting this error. I haven't found anything useful on Google.

[polm]

Can you share your code? It looks like, as the error indicates, you never called set_extension, but without any of your code it's hard to tell.

If I had to guess, it looks like maybe you imported your custom code when building a model, but now you've loaded the model and didn't import your code. If that's the problem, you should be able to fix the issue by importing the code with your custom function before loading the model.

[darrkj]

I often get that error and it comes up in a couple of different instances. The most common issue for me is the extension attribute is is created in one component and a latter in the pipeline component needs that attribute for training purposes. The solution in that instance is to add something like this to the config file.

[training]
annotating_components = ["tok2vec", "phrases"]

The other common issue I encounter is when there are two components that depend on each other and are very tightly coupled and even if the attribute is not needed for training a later component in the pipeline the logic consumes the attribute. You can add some logic to the lang factory for any attributes that are required as seen below.

@Language.factory(
    "xyz. SentenceExtractor.v1",
    requires=["doc._.phrases"],

The cause is often that for some edge case document the attribute is not properly set. The link below shows an example of catching this which basically checks after the logic that would create the attribute asserts something exists there and if not creates it. In this example it basically just checks to make sure it is not over writing anything but there can be edge cases and good to assert something is there and have an empty container to populate so that it exists, even if empty.

https://github.com/explosion/healthsea/blob/main/project/scripts/clausecat/clause_segmentation.py#L56

[fabmeyer]

@polm @darrkj Many thanks for your answers. @darrkj has provided a codesnippet that solves the problem (somehow, I don't fully understand it). In case anyone is interested here is the code that I am running to extract the x most important sentences from a string:

from math import sqrt

def extract_sentences(company_description, model, number):
    if company_description:
        # get important sentences from description with the help of spaCy
        doc = model(company_description)
        # Doc.set_extension('phrases', default=True, force=True)
        sent_bounds = [[sent.start, sent.end, set([])] for sent in doc.sents]
        limit_phrases = 4
        phrase_id = 0
        unit_vector = []
        # add phrases
        phrases = []
        # Check if ._.phrases exists and only overwrite when len() == 0
        if not doc.has_extension('phrases') or (
            doc.has_extension('phrases') and len(doc._.phrases) == 0
        ):
            doc.set_extension('phrases', default={}, force=True)
            doc._.phrases = phrases
        # examine top-ranked phrases
        for phrase in doc._.phrases:
            unit_vector.append(phrase.rank)
            for chunk in phrase.chunks:
                # construct a list of the sentence boundaries
                for sent_start, sent_end, sent_vector in sent_bounds:
                    # iterate through the top-ranked phrases, add them to the vector for each sentence
                    if chunk.start >= sent_start and chunk.end <= sent_end:
                        sent_vector.add(phrase_id)
                        break
            phrase_id += 1
            if phrase_id >= limit_phrases:
                break
        sum_ranks = sum(unit_vector)
        # normalize the vector
        unit_vector = [rank / sum_ranks for rank in unit_vector]
        sent_rank = []
        sent_id = 0
        # iterate through the sentences, add their euclidean distance from the unit vector
        for sent_start, sent_end, sent_vector in sent_bounds:
            sum_sq = 0.0
            for phrase_id in range(len(unit_vector)):
                if phrase_id not in sent_vector:
                    sum_sq += unit_vector[phrase_id] ** 2
            sent_rank.append({'sent_id': sent_id, 'rank': sqrt(sum_sq), 'sentence': doc[sent_start:sent_end].text})
            sent_id += 1
        # sort the sentences by their rank
        sent_rank = sorted(sent_rank, key=lambda x: x['rank'])
        # extract the n most important sentences (with the lowest euclidean distance)
        shortened_description = ''
        for i in range(min(number, len(sent_rank))):
            if shortened_description == '':
                shortened_description = sent_rank[i]['sentence']
            else:
                shortened_description += ' ' + sent_rank[i]['sentence']
        # shortened_description = sent_rank[0]['sentence'] + ' ' + sent_rank[1]['sentence']
        return shortened_description

with model = an arbitrary spacy model

[polm]

When you call set_extension it makes global changes. While the code you have works, it would be better to just make sure that set_extension is called at the top of your code wherever you're using it.