Skip to content

Commit ffdae72

Browse files
authored
Custom evaluator for Open-NMT2 model (#2847)
* Rebase * Added decode strategy * Intermediate changes * Rebase * DecodeStrategy implementation * DLSDK workable implementation of processing pipeline * Models called with dictionary instead of tuple * Move adapter settings to launcher section to allow path preprocessing. Disable GNMTGlobalScorer Add support for launch tests with ONNX models Encapsulate decoder functionality in standalone class to share it in DLSDK and ONNX models Remove extra comments * Code refactoring: - BeamSearch parameters moved to configuration file - BeamSearch code refactoring, implement all functionality in one class - Encapsulate all state data into OpenNMTDecoder class, including memory and mem_len, refactor state initialization, tiling and reordering - Fix PyLint warnings - Remove extra comments * Update README, change module name. * OpenNMTEvaluator inherits BaseCustomEvaluator * Fix linting issues, remove extra comments * Refactor code to inherit from base_models classes * Fix linting issues * Remove overloaded method * Pass raw outputs from model to encoder callback * Fix linting issues * Fix linting warnings, remove obsolete parameters, remove support of storing intermediate results * Fix linting issues, remove unused code
1 parent 7dcc90e commit ffdae72

File tree

3 files changed

+450
-1
lines changed

3 files changed

+450
-1
lines changed

tools/accuracy_checker/openvino/tools/accuracy_checker/evaluators/custom_evaluators/README.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,3 +56,6 @@ Optionally you can provide `module_config` section which contains config for cus
5656

5757
* **Tacotron2 Evaluator** demonstrates how to evaluate custom Tacotron2 model for text to speech task.
5858
Evaluator code: `<omz_dir>/tools/accuracy_checker/openvino/tools/accuracy_checker/evaluators/custom_evaluators/tacotron2_evaluator.py`
59+
60+
* **Open-NMT Evaluator** demonstrates how to evaluate Open-NMT model for text translation task.
61+
Evaluator code: `<omz_dir>/tools/accuracy_checker/openvino/tools/accuracy_checker/evaluators/custom_evaluators/opennmt_encoder_decoder_generator_evaluator.py`

0 commit comments

Comments
 (0)