Skip to content

[BUG] Holt Winters Anomaly Strategy fails due to NoClassDefFoundError #226

@komashk

Description

@komashk

Holt Winters Anomaly Strategy fails due to NoClassDefFoundError

To Reproduce
Steps to reproduce the behavior:

  1. Replicating Glue tutorial for the anomaly detection using Holt Winters Anomaly Strategy.
  2. Attempted with the following combinations of versions:
  • AWS Glue version 4, Spark 3.3, pydeequ 1.4.0, deequ jar com.amazon.deequ:deequ:2.0.7-spark-3.3.
  • AWS Glue version 3, Spark 3.1, pydeeqy 1.4.0, deequ jar com.amazon.deequ:deequ:2.0.7-spark-3.1
  1. I ran the code below after generating the metrics for years 2013 and 2014:
for month in range(1,9):
    date = "\'2015" +'/'+str(month)+"\'"
    df = df_2015.filter("review_date =" + date)
    key_tags = {'tag':  date}
    result_key_2015 = ResultKey(session, ResultKey.current_milli_time(), key_tags)

    jewelry_result = VerificationSuite(session).onData(df)\
        .useRepository(metricsRepository) \
        .saveOrAppendResult(result_key_2015) \
        .addAnomalyCheck(HoltWinters(session, MetricInterval.Monthly, SeriesSeasonality.Yearly), Sum('total_votes'))\
        .run()
  1. I get the following error:
Py4JJavaError: An error occurred while calling o560.run.
: java.lang.NoClassDefFoundError: breeze/stats/package$
	at com.amazon.deequ.anomalydetection.seasonal.HoltWinters.detect(HoltWinters.scala:243)
	at com.amazon.deequ.anomalydetection.AnomalyDetector.detectAnomaliesInHistory(AnomalyDetector.scala:98)
.....

Expected behavior
The expected behavior is to obtain a message that anomaly has been detected.

What can be done to fix this error? Thank you.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions