Skip to content

Conversation

ricardoV94
Copy link
Member

@ricardoV94 ricardoV94 commented Mar 30, 2025

PyTensor exceptions are very verbose by default. Users often struggle to even find the actual error message.

This PR makes exceptions more minimal with a hint to set the relevant flag exception_verbosity to medium (the old low or high) for more details.

The type error in the following snippet:

import pytensor
import pytensor.tensor as pt

x = pt.matrix("x")
y = pt.specify_shape(x, (2, 3))
fn = pytensor.function([x], y, mode="FAST_RUN")

fn([1])

Now looks like:

Traceback (most recent call last):
  File "/home/ricardo/miniforge3/envs/pytensor-dev/lib/python3.12/site-packages/IPython/core/interactiveshell.py", line 3577, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-6c9f1f34f2ad>", line 8, in <module>
    fn([1])
  File "/home/ricardo/Documents/pytensor/pytensor/compile/function/types.py", line 960, in __call__
    arg_container.storage[0] = arg_container.type.filter(
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ricardo/Documents/pytensor/pytensor/tensor/type.py", line 254, in filter
    raise TypeError(
TypeError: Wrong number of dimensions: expected 2, got 1 with shape (1,).
Invalid argument to PyTensor function at index 0.

Whereas with the old default looked like

Traceback (most recent call last):
  File "/home/ricardo/miniforge3/envs/pytensor/lib/python3.12/site-packages/IPython/core/interactiveshell.py", line 3577, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-3-6c9f1f34f2ad>", line 8, in <module>
    fn([1])
  File "/home/ricardo/Documents/pytensor/pytensor/compile/function/types.py", line 944, in __call__
    arg_container.storage[0] = arg_container.type.filter(
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ricardo/Documents/pytensor/pytensor/tensor/type.py", line 242, in filter
    raise TypeError(
TypeError: Bad input argument to pytensor function with name "<ipython-input-3-6c9f1f34f2ad>:6" at index 0 (0-based).  

Backtrace when that variable is created:
  File "/app/extra/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_ipython_console_011.py", line 438, in add_exec
    res = self.ipython.run_cell(line, store_history=True)
  File "/home/ricardo/miniforge3/envs/pytensor/lib/python3.12/site-packages/IPython/core/interactiveshell.py", line 3075, in run_cell
    result = self._run_cell(
  File "/home/ricardo/miniforge3/envs/pytensor/lib/python3.12/site-packages/IPython/core/interactiveshell.py", line 3130, in _run_cell
    result = runner(coro)
  File "/home/ricardo/miniforge3/envs/pytensor/lib/python3.12/site-packages/IPython/core/async_helpers.py", line 128, in _pseudo_sync_runner
    coro.send(None)
  File "/home/ricardo/miniforge3/envs/pytensor/lib/python3.12/site-packages/IPython/core/interactiveshell.py", line 3334, in run_cell_async
    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
  File "/home/ricardo/miniforge3/envs/pytensor/lib/python3.12/site-packages/IPython/core/interactiveshell.py", line 3517, in run_ast_nodes
    if await self.run_code(code, result, async_=asy):
  File "/home/ricardo/miniforge3/envs/pytensor/lib/python3.12/site-packages/IPython/core/interactiveshell.py", line 3577, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-3-6c9f1f34f2ad>", line 4, in <module>
    x = pt.matrix("x")
Wrong number of dimensions: expected 2, got 1 with shape (1,).

A runtime error:

fn([[1]])

Now looks like this

Traceback (most recent call last):
  File "/home/ricardo/miniforge3/envs/pytensor/lib/python3.12/site-packages/IPython/core/interactiveshell.py", line 3577, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-8-df63f13088e4>", line 8, in <module>
    fn([[1]])
  File "/home/ricardo/Documents/pytensor/pytensor/compile/function/types.py", line 1040, in __call__
    raise_with_op(
  File "/home/ricardo/Documents/pytensor/pytensor/link/utils.py", line 320, in raise_with_op
    raise exc_value.with_traceback(exc_trace)
  File "/home/ricardo/Documents/pytensor/pytensor/compile/function/types.py", line 1030, in __call__
    outputs = vm() if output_subset is None else vm(output_subset=output_subset)
              ^^^^
AssertionError: SpecifyShape: dim 0 of input has shape 1, expected 2.

HINT: Set PyTensor `config.exception_verbosity` to `medium` or `high` for more information about the source of the error.

Whereas before it looked like

Traceback (most recent call last):
  File "/home/ricardo/Documents/pytensor/pytensor/compile/function/types.py", line 1037, in __call__
    outputs = vm() if output_subset is None else vm(output_subset=output_subset)
              ^^^^
AssertionError: SpecifyShape: dim 0 of input has shape 1, expected 2.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ricardo/miniforge3/envs/pytensor/lib/python3.12/site-packages/IPython/core/interactiveshell.py", line 3577, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-df63f13088e4>", line 8, in <module>
    fn([[1]])
  File "/home/ricardo/Documents/pytensor/pytensor/compile/function/types.py", line 1047, in __call__
    raise_with_op(
  File "/home/ricardo/Documents/pytensor/pytensor/link/utils.py", line 526, in raise_with_op
    raise exc_value.with_traceback(exc_trace)
  File "/home/ricardo/Documents/pytensor/pytensor/compile/function/types.py", line 1037, in __call__
    outputs = vm() if output_subset is None else vm(output_subset=output_subset)
              ^^^^
AssertionError: SpecifyShape: dim 0 of input has shape 1, expected 2.
Apply node that caused the error: SpecifyShape(x, 2, 3)
Toposort index: 0
Inputs types: [TensorType(float64, shape=(None, None)), TensorType(int8, shape=()), TensorType(int8, shape=())]
Inputs shapes: [(1, 1), (), ()]
Inputs strides: [(8, 8), (), ()]
Inputs values: [array([[1.]]), array(2, dtype=int8), array(3, dtype=int8)]
Outputs clients: [[DeepCopyOp(SpecifyShape.0)]]

Backtrace when the node is created (use PyTensor flag traceback__limit=N to make it longer):
  File "/app/extra/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_ipython_console_011.py", line 438, in add_exec
    res = self.ipython.run_cell(line, store_history=True)
  File "/home/ricardo/miniforge3/envs/pytensor/lib/python3.12/site-packages/IPython/core/interactiveshell.py", line 3075, in run_cell
    result = self._run_cell(
  File "/home/ricardo/miniforge3/envs/pytensor/lib/python3.12/site-packages/IPython/core/interactiveshell.py", line 3130, in _run_cell
    result = runner(coro)
  File "/home/ricardo/miniforge3/envs/pytensor/lib/python3.12/site-packages/IPython/core/async_helpers.py", line 128, in _pseudo_sync_runner
    coro.send(None)
  File "/home/ricardo/miniforge3/envs/pytensor/lib/python3.12/site-packages/IPython/core/interactiveshell.py", line 3334, in run_cell_async
    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
  File "/home/ricardo/miniforge3/envs/pytensor/lib/python3.12/site-packages/IPython/core/interactiveshell.py", line 3517, in run_ast_nodes
    if await self.run_code(code, result, async_=asy):
  File "/home/ricardo/miniforge3/envs/pytensor/lib/python3.12/site-packages/IPython/core/interactiveshell.py", line 3577, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-df63f13088e4>", line 5, in <module>
    y = pt.specify_shape(x, (2, 3))

HINT: Use the PyTensor flag `exception_verbosity=high` for a debug print-out and storage map footprint of this Apply node.

📚 Documentation preview 📚: https://pytensor--1330.org.readthedocs.build/en/1330/

@ricardoV94 ricardoV94 force-pushed the reduce_exception_verbosity branch from 21f4232 to 9e376ef Compare March 30, 2025 20:46
@ricardoV94
Copy link
Member Author

The add_note feature was only added in 3.11, so I would wait

@ricardoV94 ricardoV94 marked this pull request as draft April 2, 2025 17:14
@ricardoV94 ricardoV94 force-pushed the reduce_exception_verbosity branch from 9e376ef to ff479ac Compare October 9, 2025 10:16
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR simplifies PyTensor exception messages by reducing their verbosity by default, making it easier for users to find the actual error message. The changes introduce a new "low" verbosity level that shows minimal information with a hint to increase verbosity for more details.

  • Adds a new "low" exception verbosity level as the default
  • Simplifies error message formatting in function calls
  • Provides hints to users about how to get more detailed error information

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
pytensor/link/utils.py Adds hint message for low verbosity exceptions
pytensor/configdefaults.py Updates exception verbosity config to include new "medium" level and simplified description
pytensor/compile/function/types.py Refactors exception handling to use cleaner messages and respect verbosity settings

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@ricardoV94 ricardoV94 force-pushed the reduce_exception_verbosity branch from ff479ac to 8002918 Compare October 9, 2025 10:35
@ricardoV94 ricardoV94 requested a review from Copilot October 9, 2025 10:35
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@ricardoV94 ricardoV94 force-pushed the reduce_exception_verbosity branch from 8002918 to f575025 Compare October 9, 2025 10:42
@ricardoV94 ricardoV94 force-pushed the reduce_exception_verbosity branch from f575025 to 11376ac Compare October 9, 2025 12:37
@ricardoV94
Copy link
Member Author

Failing jax test addressed in #1646

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant