Error using some pretrained pipelines in Spark/PySpark 3.x

[muhammetsnts]

Description

When I try to use match_chunks and match_datetime pretrained pipelines in sparknlp_version 3.0.1, get errors while downloading these pipelines.

Current Behavior

Steps to Reproduce

1. pipeline = PretrainedPipeline('match_chunks', lang='en')
2. pipeline = PretrainedPipeline('match_datetime', lang='en')

Context

Your Environment

• Spark NLP version sparknlp.version(): 3.0.1
• Apache NLP version spark.version: 3.1.1
• Java version java -version: openjdk version "1.8.0_282"

[maziyarpanahi]

@Digaari Only report anything related to public and open-source. We are not responsible for anything else.

[Digaari]

Faced similar issue with check_spelling_dl in the same environment.

[maziyarpanahi]

Any model/pipeline with that specific error means they are not trained/saved in Spark 3.x/Scala 2.12, they need to be trained/saved using Spark 3.x.

Please make a list and pass it to the Models Hub team to fix them

[muhammetsnts]

@Digaari here is a list of pipelines that not working on spark 3.x.

• clean_slang
• check_spelling_dl
• match_chunks
• match_datetime

[muhammetsnts]

@maziyarpanahi this issue is still open and we've tested these models, they are still broken. As you said, @josejuanmartinez can assign someone from modelshub team.

[maziyarpanahi]

Thanks @muhammetsnts

These pipelines need to be re-do/re-uploaded by using Apache Spark 3.x. Some models/pipelines need two copies one for spark 2.x and one for soark 3.x.

So these work in Spark 2.x but the 3.x are missing. Please assign a member to save and upload these pipelines with the same version/metadata but on spark 3.x this time.

[cholojuanito]

I'm still running into this issue with the following environment

• Spark NLP version sparknlp.version(): 3.4.2
• Apache NLP version spark.version: 3.1.2
• Java version java -version: openjdk version "11.0.14"