Skip to content

[ET] Fix serde handling of inf #6832

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Nov 13, 2024
Merged

[ET] Fix serde handling of inf #6832

merged 1 commit into from
Nov 13, 2024
+7 −1

Conversation

@pytorchbot
Copy link
Collaborator

This PR was created by the merge bot to help merge the original PR into the main branch.
ghstack PR number: #6799
^ Please use this as the source of truth for the PR details, comments, and reviews
ghstack PR base: https://github.com/pytorch/executorch/tree/gh/SS-JIA/144/base
ghstack PR head: https://github.com/pytorch/executorch/tree/gh/SS-JIA/144/head
Merge bot PR base: https://github.com/pytorch/executorch/tree/main
Merge bot PR head: https://github.com/pytorch/executorch/tree/gh/SS-JIA/144/orig
@diff-train-skip-merge

@SS-JIA
[ET] Fix serde handling of inf
0cd7288 
Pull Request resolved: #6799

## Context

I was trying to inspect the ETDump of a Vulkan LLaMa inference run, and experienced an `Exception` stemming from a symbol range containing an infinity.

The full error log is

```
Traceback (most recent call last):
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/sympy/core/expr.py", line 4035, in _mag
    mag_first_dig = int(ceil(log10(xpos)))
OverflowError: cannot convert float infinity to integer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/__run_lpar_main__.py", line 31, in <module>
    __invoke_main()
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/__run_lpar_main__.py", line 28, in __invoke_main
    run_as_main(module, main_function)
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/__par__/meta_only/bootstrap.py", line 98, in run_as_main
    oss_run_as_main(
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/__par__/bootstrap.py", line 94, in run_as_main
    main()
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/executorch/backends/vulkan/fb/tools/inspector/inspector_cli.py", line 41, in main
    inspector = Inspector(
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/executorch/devtools/inspector/_inspector.py", line 1025, in __init__
    self._etrecord = parse_etrecord(etrecord_path=etrecord)
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/executorch/devtools/etrecord/_etrecord.py", line 318, in parse_etrecord
    edge_dialect_program = deserialize(serialized_artifact)
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/executorch/exir/serde/serialize.py", line 683, in deserialize
    return ExportedProgramDeserializer(expected_opset_version).deserialize(
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/executorch/exir/serde/serialize.py", line 598, in deserialize
    res = GraphModuleDeserializer().deserialize(
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/executorch/exir/serde/export_serialize.py", line 1855, in deserialize
    lower = int(vr.lower)
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/sympy/core/expr.py", line 308, in __int__
    r = self.round(2)
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/sympy/core/expr.py", line 3856, in round
    digits_to_decimal = _mag(x)  # _mag(12) = 2, _mag(.012) = -1
  File "/data/users/ssjia/fbsource/buck-out/v2/gen/fbcode/17b27aeeb079adbf/executorch/backends/vulkan/fb/tools/inspector/__inspector_cli__/inspector_cli#link-tree/sympy/core/expr.py", line 4037, in _mag
    mag_first_dig = int(ceil(Float(mpf_log(xpos._mpf_, 53))/log(10)))
AttributeError: 'Infinity' object has no attribute '_mpf_'
```

This arises because the range of a symbol is `[-inf to inf]`. Since infinity cannot be converted to `int` in Python, an exception is raised.

## Changes

Check if a range bound is negative or positive infinity and set it explicitly instead of converted the symbolic integer to `int`.
ghstack-source-id: 253217725
@exported-using-ghexport

Differential Revision: [D65848116](https://our.internmc.facebook.com/intern/diff/D65848116/)
@pytorch-bot
Copy link

pytorch-bot bot commented Nov 13, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6832

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures

As of commit 0cd7288 with merge base dc41596 (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 13, 2024
@SS-JIA SS-JIA merged commit 93b202c into main Nov 13, 2024
34 of 35 checks passed
@SS-JIA SS-JIA deleted the gh/SS-JIA/144/orig branch November 13, 2024 20:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@SS-JIA SS-JIA SS-JIA approved these changes

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@pytorchbot @SS-JIA @facebook-github-bot