Skip to content

Use integer division in encode_phys to prevent rounding errors with int64 #611

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

yves-chevallier
Copy link

While testing int64 values with SDOs, I observed an unexpected behavior:

>>> value = 0x55554444AAAABBBB
>>> d.node.sdo[16384][4].phys = value
>>> hex(d.node.sdo[16384][4].phys)
'0x55554444aaaabc00'

Upon investigation, it appears that the issue originates from the encode_phys function:

>>> value = 0x55554444AAAABBBB
>>> sdo.od.factor
1
>>> hex(sdo.od.encode_phys(value))
'0x55554444aaaabc00'

The problem lies in the fact that the division is performed using floating-point arithmetic rather than integer arithmetic, leading to rounding that causes a loss of up to 10 bits of precision.

def encode_phys(self, value: Union[int, bool, float, str, bytes]) -> int:
    if self.data_type in INTEGER_TYPES:
        value /= self.factor
        value = int(round(value))
    return value

To address this, we should detect whether the input value is an integer and, if so, perform integer division (//) instead of floating-point division. Additionally, it may be prudent to emit a warning when factor is not an integer, as this could lead to precision loss.

Copy link
Member

@acolomb acolomb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not convinved this is the right approach. Truncating may be worse than rounding in some existing use cases, so we should avoid breaking them / changing to different values unexpectedly. OTOH, you can always do the calculation yourself and assign to .raw instead. Which might be better for just the few cases where large integers cause inaccuracy.

elif isinstance(value, int) and isinstance(self.factor, int):
value = value // self.factor
else:
value = int(round(value / self.factor))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

round() already returns an integer if ndigits is omitted.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round was the original code. My contribution is only the // division, which is my use case is the right thing to do. What do you suggest?

@sveinse
Copy link
Collaborator

sveinse commented Aug 12, 2025

The problem lies in the fact that the division is performed using floating-point arithmetic rather than integer arithmetic, leading to rounding that causes a loss of up to 10 bits of precision.

I think TS has a point here. It should be possible to reliably pass int64 through the encode function.

To address this, we should detect whether the input value is an integer and, if so, perform integer division (//) instead of floating-point division.

This could be a good solution since 64-bit double can only hold up to 52 bits (+ sign bit) in the mantissa, so any values higher than that will get truncated in the float division.

Additionally, it may be prudent to emit a warning when factor is not an integer, as this could lead to precision loss.

Is it a use case to have non-integer factors? When do we get into this use case? Does the standard permit float scale factors?

Another alternative would be to skip the division altogether if the factor is 1. In most cases factor is not used at all, and I'd like to suggest that int64 with scale factor other than 1 is somewhat uncommon use case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants