Skip to content

Fix NumPy Alignment: int //, int %, and signed right-shift semantics; bool ** dtype #5259

@ajpotts

Description

@ajpotts

Fix NumPy Alignment: int //, int %, and signed right-shift semantics; bool ** dtype

Summary

The NumPy alignment suite currently fails for several binary operations where Arkouda semantics diverge from NumPy:

  1. Signed integer floor division (//) differs for negative values.
  2. Signed integer modulo (%) / remainder differs for negative values (tied to floor-division semantics).
  3. Signed integer right shift (>>) differs for negative values (arithmetic vs logical / truncation behavior).
  4. Boolean power (**) dtype promotion differs (NumPy returns signed int; Arkouda returns unsigned).

These should be fixed in Arkouda to match NumPy semantics.

Evidence (from alignment tests)

  • floordiv mismatches with max absolute difference 1 on many elements → consistent with floor vs trunc behavior.
  • mod mismatches with large differences → consistent with NumPy remainder sign rules.
  • rshift mismatches on negatives (NumPy often yields -1 where Arkouda yields 0) → consistent with arithmetic vs logical shift.
  • pow on bool: expected dtype kind 'i' but got 'u'.

Expected NumPy Semantics

1) Floor division (a // b)

For signed integers, NumPy uses mathematical floor:

  • -3 // 2 == -2
  • 3 // -2 == -2
  • -3 // -2 == 1

2) Remainder / modulo (a % b)

NumPy defines remainder consistent with floor division:

  • a == (a // b) * b + (a % b)
  • a % b has the same sign as b (the divisor)
    Examples:
  • -3 % 2 == 1
  • 3 % -2 == -1

3) Right shift (a >> k) for signed ints

NumPy behaves like an arithmetic right shift on signed integers (sign-propagating) for typical two's-complement behavior:

  • -1 >> 1 == -1
  • -3 >> 1 == -2

4) Boolean power dtype

NumPy’s bool ** bool results are promoted to a signed integer dtype (alignment suite expects kind 'i').

Current Arkouda Likely Semantics (inferred)

  • // and % appear to use truncation toward zero and a remainder consistent with truncation (C-like), not floor.
  • >> appears to behave like a logical shift (zero-fill) for signed ints, not arithmetic shift.
  • bool ** bool appears to promote to uint rather than signed int.

Proposed Fixes

A) Signed integer floor division and remainder

Implement NumPy-compatible floor-division and remainder for signed integer types.

For a and b integers (b != 0), let:

  • q0 = trunc_div(a, b) (current behavior)
  • r0 = a - q0*b
    Then adjust to floor-division:
  • If r0 != 0 and ((r0 > 0) != (b > 0)) (i.e., remainder and divisor have opposite signs),
    • q = q0 - 1
    • r = r0 + b
      Else:
    • q = q0
    • r = r0

This ensures:

  • q == floor(a/b)
  • r == a % b per NumPy

B) Signed right shift

Ensure signed right shift is arithmetic:

  • Preserve sign bit when shifting right for signed integer dtypes.
  • For unsigned dtypes, keep logical shift.

C) bool ** bool dtype promotion

Align dtype promotion with NumPy for power on bool operands:

  • Ensure result is signed int (likely int64) rather than uint64.

Tests to Add / Update

  1. Targeted unit tests with small explicit vectors:
    • a = [-3, -2, -1, 0, 1, 2, 3], b = [2, -2, 3, -3, 5, -5, 7] (broadcast as needed)
    • Assert Arkouda matches NumPy for // and %
  2. Right shift explicit tests:
    • a = [-1, -2, -3, -4, 1, 2, 3], shift counts [1, 2, 3]
  3. bool power dtype/value:
    • a = [False, True], b = [False, True]
    • Assert values match NumPy and dtype kind is 'i'

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions