-
Notifications
You must be signed in to change notification settings - Fork 25
Open
Labels
Numpy:LargePrecisionIssue when numpy has to deal with large precisionIssue when numpy has to deal with large precisionOS:WindowsIssue when running in WindowsIssue when running in Windows
Milestone
Description
Getting strange results when using larger bit sizes in numpy calls
# This works
w = Fxp([1, 1, 1, 1], dtype='fxp-s29/0')
y=np.cumsum(w)
y.info()
# This doesn't
w = Fxp([1, 1, 1, 1], dtype='fxp-s32/0')
y=np.cumsum(w)
y.info()
The first call returns [1 2 3 4], while the 2nd returns [-1 -1 -1 -1]
Version info
OS - Windows 10
Python - 3.8.5
Numpy - 1.21.2
fxpmath - 0.4.8
Metadata
Metadata
Assignees
Labels
Numpy:LargePrecisionIssue when numpy has to deal with large precisionIssue when numpy has to deal with large precisionOS:WindowsIssue when running in WindowsIssue when running in Windows