Skip to content

Commit 898ca10

Browse files
xaduprexadupre
authored
Add ExtendedReferenceEvaluator (#8)
* Add ExtendedReferenceEvaluator * save * doc * ci * fix test * add data * remove file * fix issues * codecov * cod * doc
1 parent ee1c214 commit 898ca10

File tree

73 files changed

+2720
-5
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

73 files changed

+2720
-5
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,5 +56,12 @@ jobs:
5656
run: |
5757
pip install pytest
5858
export PYTHONPATH=.
59-
UNITTEST_GOING=1 pytest --durations=10 _unittests
59+
UNITTEST_GOING=1 pytest --durations=10 _unittests --ignore _unittests/ut_reference/test_backend_extended_reference_evaluator.py
60+
export PYTHONPATH=
61+
62+
- name: run backend tests
63+
run: |
64+
pip install pytest
65+
export PYTHONPATH=.
66+
UNITTEST_GOING=1 pytest --durations=10 _unittests/ut_reference/test_backend_extended_reference_evaluator.py
6067
export PYTHONPATH=

‎.github/workflows/documentation.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ jobs:
5959
pip install pytest
6060
pip install pytest-cov
6161
export PYTHONPATH=.
62-
UNITTEST_GOING=1 pytest --cov=./onnx_diagnostic/ --cov-report=xml --durations=10 _unittests
62+
UNITTEST_GOING=1 pytest --cov=./onnx_diagnostic/ --cov-report=xml --durations=10 _unittests --ignore _unittests/ut_reference/test_backend_extended_reference_evaluator.py
6363
export PYTHONPATH=
6464
6565
- name: Upload coverage reports to Codecov

‎CHANGELOGS.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ Change Logs
44
0.2.0
55
+++++
66

7+
* :pr:`8`: adds ``ExtendedReferenceEvaluator``
78
* :pr:`7`: improves function ``investigate_onnxruntime_issue``
89

910
0.1.0

‎README.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,8 @@ or
5050
<https://sdpython.github.io/doc/onnx-diagnostic/dev/auto_examples/plot_export_with_dynamic_shapes_auto.html>`_
5151
* `Steel method forward to guess the dynamic shapes
5252
<https://sdpython.github.io/doc/onnx-diagnostic/dev/auto_examples/plot_export_tiny_llm.html>`_
53+
* `Running ReferenceEvaluator on a failing model
54+
<https://sdpython.github.io/doc/onnx-diagnostic/dev/auto_examples/plot_failing_reference_evaluator.html>`_
5355
* `Find where a model is failing by running submodels
5456
<https://sdpython.github.io/doc/onnx-diagnostic/dev/auto_examples/plot_failing_model_extract.html>`_
5557

‎_doc/api/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ API of onnx_diagnostic
77
:maxdepth: 1
88
:caption: submodules
99

10+
reference/index
1011
torch_export_patches/index
1112
torch_models/index
1213

‎_doc/api/reference/evaluator.rst‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
2+
onnx_diagnostic.reference.evaluator
3+
===================================
4+
5+
.. automodule:: onnx_diagnostic.reference.evaluator
6+
:members:
7+
:no-undoc-members:
8+
:exclude-members: ExtendedReferenceEvaluator

‎_doc/api/reference/index.rst‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
onnx_diagnostic.reference
2+
=========================
3+
4+
.. toctree::
5+
:maxdepth: 1
6+
:caption: submodules
7+
8+
ops/index
9+
10+
.. toctree::
11+
:maxdepth: 1
12+
:caption: modules
13+
14+
evaluator
15+
quantized_tensor
16+
17+
ExtendedReferenceEvaluator
18+
++++++++++++++++++++++++++
19+
20+
.. autoclass:: onnx_diagnostic.reference.ExtendedReferenceEvaluator
21+
:members:
22+
23+
Other functions
24+
+++++++++++++++
25+
26+
.. automodule:: onnx_diagnostic.reference
27+
:members:
28+
:no-undoc-members:

‎_doc/api/reference/ops/index.rst‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
2+
onnx_diagnostic.reference.ops
3+
=============================
4+
5+
6+
7+
.. toctree::
8+
:maxdepth: 1
9+
:caption: modules
10+
11+
12+
op_add_add_mul_mul
13+
op_average_pool_grad
14+
op_cast_like
15+
op_complex
16+
op_concat
17+
op_constant_of_shape
18+
op_fused_matmul
19+
op_gather_grad
20+
op_memcpy_host
21+
op_mul_sigmoid
22+
op_negxplus1
23+
op_quick_gelu
24+
op_replace_zero
25+
op_rotary
26+
op_qlinear_average_pool
27+
op_qlinear_conv
28+
op_scatter_elements
29+
op_scatternd_of_shape
30+
op_simplified_layer_normalization
31+
op_skip_layer_normalization
32+
op_slice
33+
op_transpose_cast
34+
op_tri_matrix
35+
36+
37+
.. automodule:: onnx_diagnostic.reference.ops
38+
:members:
39+
:no-undoc-members:

‎_doc/api/reference/ops/op_add_add_mul_mul.rst‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
2+
onnx_diagnostic.reference.ops.op_add_add_mul_mul
3+
================================================
4+
5+
.. automodule:: onnx_diagnostic.reference.ops.op_add_add_mul_mul
6+
:members:

‎_doc/api/reference/ops/op_average_pool_grad.rst‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
2+
onnx_diagnostic.reference.ops.op_average_pool_grad
3+
==================================================
4+
5+
.. automodule:: onnx_diagnostic.reference.ops.op_average_pool_grad
6+
:members:

0 commit comments

Comments
 (0)