Skip to content

Comments

Fix missing evaluation dependencies (exec_eval.py, parse.py) from test-suite-sql-eval#2

Open
ewerthonk wants to merge 2 commits intoemorynlp:mainfrom
ewerthonk:main
Open

Fix missing evaluation dependencies (exec_eval.py, parse.py) from test-suite-sql-eval#2
ewerthonk wants to merge 2 commits intoemorynlp:mainfrom
ewerthonk:main

Conversation

@ewerthonk
Copy link

Description

Hello team! I am currently reading through your paper and am looking to use ETM as the evaluation metric for my master's degree research on Text-to-SQL tasks.

While testing the repository locally to evaluate some sets, I encountered a missing dependencies issue with the evaluation script, specifically failing to locate [exec_eval.py] and parse.py logic which seemed to be missing from ETM_utils.

After some research, I found that these files originated from the taoyds/test-suite-sql-eval repository. This pull request adds the missing evaluation scripts directly into ETM_utils/ and updates the internal imports in my fork to ensure the [treeMatch.py] evaluator works successfully out of the box when ran locally.

I've tested the changes locally and confirmed that it executes and returns the scores successfully.

Changes Included

  • Added [ETM_utils/exec_eval.py] and ETM_utils/parse.py from the test-suite-sql-eval suite.
  • Re-organized any related local imports in [ETM.py] and [evaluation.py] to point to the correct evaluation module path.

Thank you for providing the repo to the community, hoping this helps anyone else trying to run the evaluations locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant