Problem with prediction "ValueError: Cannot get dimension 'nO' for model 'sparse_linear': value unset" after load text to nlp

[arioslock]

I'm encountering an issue with a spaCy model trained from a config file. I've successfully trained, evaluated, and saved the model. However, when attempting to load text into the nlp component for prediction, I encounter the following error:

Copy code ValueError: Cannot get dimension 'nO' for model 'sparse_linear': value unset

`

 nlp = spacy.blank("pl")
 df_spacy = pd.DataFrame(df_to_model[['desc_transformed', 'final_label_grouped']])
 df_spacy.reset_index(inplace=True)

 class_names = ['anulacja', 'dostawa', 'other', 'reklamacja', 'zwrot']
 categories = df_spacy['final_label_grouped'].values.reshape(-1, 1)
 encoder = OneHotEncoder()
 one_hot_encoded = encoder.fit_transform(categories)
 one_hot_array = one_hot_encoded.toarray()
 df_encoded = pd.DataFrame(one_hot_array, columns=class_names).astype(int)
 label = list(df_encoded.columns)
 df_encoded = df_encoded.to_dict('index')
 dataset = list(zip(df_to_model['desc_transformed'], [{'categories':cat} for cat in df_encoded.values()]))

 train_data, test_data = train_test_split(dataset,
                                      test_size=0.25,
                                      random_state=42)
 dev_data, test_data = train_test_split(test_data, 
                                     test_size=0.4,
                                     random_state=42)
 print(f"Total: {len(dataset)} - Train:  {len(train_data)} - Dev: {len(dev_data)} - Test: {len(test_data)}")

 def convert(data, outfile):
      db = spacy.tokens.DocBin()
      for text, labels in data:
           doc = spacy.tokens.Doc(nlp.vocab, words=text.split())
           doc.cats.update(labels['categories'])
           db.add(doc)
      db.to_disk(outfile)
 convert(train_data, "./train.spacy")
 convert(dev_data, "./dev.spacy")
 convert(test_data, "./test.spacy")

 textcat = nlp.add_pipe("textcat_multilabel")

 for i in label:
     textcat.add_label(i)

 ! python -m spacy init fill-config base_config.cfg config.cfg
 ! python -m spacy train config.cfg --output ./output --paths.train ./train.spacy --paths.dev ./dev.spacy
 ! python -m spacy evaluate ./model/model-best/ ./test.spacy

 output_dir = 'F:\\Classification_models\\spaCy_model'
 nlp.to_disk(output_dir)
 saved_model_dir = 'F:\Classification_models\\spaCy_model'
 nlp = spacy.load(saved_model_dir)

 def predict_spaCy(text):
     doc = nlp(text) I get an error when I try to use this function and attempt to load text into the nlp component.
     label = max(doc.cats, key=doc.cats.get)
return label

 text_to_classify = "klient pytanie produkt ASIN   połączenie dachówka wentylacyjny   link zapytanie web_site pozdrawiać persName"
 predicted_label = predict_spaCy(text_to_classify) # This is line that crash my code
 print("Predicted Label:", predicted_label)

`

I've searched extensively for a solution but haven't found one. Could anyone provide insight into what might be causing this error and how it can be resolved?

After running ! python -m spacy evaluate ./model/model-best/ ./test.spacy, I receive results along with the following UserWarning:

UserWarning: [W095] Model 'en_pipeline' (0.0.0) was trained with spaCy v3.6.1 and may not be 100% compatible with the current version (3.7.2). If you see errors or degraded performance, download a newer compatible model or retrain your custom model with the current spaCy version. For more details and available updates, run: python -m spacy validate

However, this is a freshly trained model with a fresh configuration file. When I execute ! python -m spacy validate, I get:
`

 ================= Installed pipeline packages (spaCy v3.7.2) =================
 ℹ spaCy installation:
 [c:\Users\48575\AppData\Local\Programs\Python\Python311\Lib\site-packages\spacy]               
 (file:///C:/Users/48575/AppData/Local/Programs/Python/Python311/Lib/site-packages/spacy)
 NAME              SPACY            VERSION                            
 en_core_web_sm    >=3.7.2,<3.8.0   3.7.1   ✔
 pl_core_news_lg   >=3.7.0,<3.8.0   3.7.0   ✔
 pl_core_news_md   >=3.7.0,<3.8.0   3.7.0   ✔
 pl_core_news_sm   >=3.7.0,<3.8.0   3.7.0   ✔

`

I believe this could be the root of the problem, but I consistently encounter the same user warning after training, indicating that I'm unable to obtain a model trained with spaCy 3.7.2.