Issue with ctypes in AIX

[Ayappan]

BPO	38628
Nosy	@vsajip, @ronaldoussoren, @amauryfa, @abalkin, @meadori, @eryksun, @aixtools, @isidentical, @Sanket, @trex58

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2019-10-29.09:17:23.348>
labels = ['ctypes', 'type-bug', '3.8', '3.9', '3.10']
title = 'Issue with ctypes in AIX'
updated_at = <Date 2020-08-16.16:30:56.589>
user = 'https://bugs.python.org/Ayappan'

bugs.python.org fields:

activity = <Date 2020-08-16.16:30:56.589>
actor = 'vinay.sajip'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['ctypes']
creation = <Date 2019-10-29.09:17:23.348>
creator = 'Ayappan'
dependencies = []
files = []
hgrepos = []
issue_num = 38628
keywords = []
message_count = 33.0
messages = ['355632', '356588', '361324', '361374', '361433', '374173', '374174', '374175', '374178', '374180', '374183', '374184', '374192', '374369', '374375', '374389', '374392', '374395', '374685', '374686', '374726', '374730', '374747', '374834', '374837', '374861', '374863', '375146', '375493', '375507', '375509', '375511', '375517']
nosy_count = 12.0
nosy_names = ['vinay.sajip', 'ronaldoussoren', 'amaury.forgeotdarc', 'belopolsky', 'meador.inge', 'eryksun', 'David.Edelsohn', 'Michael.Felt', 'Ayappan', 'BTaskaya', 'sanket', 'T.Rex']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue38628'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

[Ayappan]

There seems to be a behavioral issue with ctypes in AIX.
The specific symptom is that passing structures containing arrays to a C function by value appears to be broken.
Consider the below program.,

#!/usr/bin/env python3

from ctypes import *

libc = CDLL('libc.a(shr_64.o)')


class MemchrArgsHack(Structure):
    _fields_ = [("s", c_char_p), ("c", c_ulong), ("n", c_ulong)]


memchr_args_hack = MemchrArgsHack()
memchr_args_hack.s = b"abcdef"
memchr_args_hack.c = ord('d')
memchr_args_hack.n = 7


class MemchrArgsHack2(Structure):
    _fields_ = [("s", c_char_p), ("c_n", c_ulong * 2)]


memchr_args_hack2 = MemchrArgsHack2()
memchr_args_hack2.s = b"abcdef"
memchr_args_hack2.c_n[0] = ord('d')
memchr_args_hack2.c_n[1] = 7

print(
CFUNCTYPE(c_char_p, c_char_p, c_uint, c_ulong,
c_void_p)(('memchr', libc))(b"abcdef", c_uint(ord('d')),
c_ulong(7), None))
print(
CFUNCTYPE(c_char_p, MemchrArgsHack,
c_void_p)(('memchr', libc))(memchr_args_hack, None))
print(
CFUNCTYPE(c_char_p, MemchrArgsHack2,
c_void_p)(('memchr', libc))(memchr_args_hack2, None))

This one uses memchr from the C library and passing it structures that would map to the registers that correspond to the arguments of memchr. This works for the first structure type in the reproducer; however, using the second structure type (which should be treated the same way) does not work.

In the failing case, the last register that should be used for the structure is not populated from the structure. The reproducer passes an extra argument so that the register is instead populated from that argument for the failing case (instead of working because the register still contains the correct value from a previous call).

The output should be the same for all three calls, but we get:
b'def'
b'def'
None

The last line is None, because memchr got 0 (the None passed on the call) for its length argument.

[Ayappan]

Any update on this ?

[DavidEdelsohn]

How was Python compiled? With GCC? Which version of GCC?
I assume that Python was built as a 64 bit application based on libc loading the 64 bit member shr_64.o.

Does the testcase work in 32 bit mode?
Does the testcase work if Python is compiled by XLC?

This likely is an incompatibility in libffi with libffi loading the registers incorrectly for the call into libc.a(shr_64.o).

It seems rather fragile to pass a struct that is supposed to have the same parameter layout as the function signature.

[DavidEdelsohn]

Is this a legal use of Python ctypes? I don't see anything in the Python documentation that one can call a ctypes function with an argument list that does not match the function signature and expect it to work. Maybe this works on x86 by accident, but that doesn't mean that it is guaranteed to work everywhere and is permitted.

[DavidEdelsohn]

The bug report implies a different bug than what is being reported. The bug is not related to calling a LIBC function with an argument list that does not match the function signature.

The true issue is that a Python ctypes structure definition on AIX that contains an array as in the example does not create an argument list that matches the AIX ABI for argument passing. An example that directly uses libffi seems to work, but invoking libffi via Python ctypes does not.

In other words, Python ctypes structures created with _fields_ equivalent to

struct {
  const char *s;
  unsigned long d;
  size_t n;
}

should produce the same argument list as

struct {
  const char *s;
  unsigned long c_n[2];
}

but the version with the array does not.

libffi passes arrays as pointers, so Python ctypes converts arrays passed by value as libffi structs. This occurs in cpython/Modules/_ctypes/stgdict.c . It is likely that ctypes is not generating the correct libffi descriptor.

The memchr example visually demonstrates the incorrect argument list, but is not intended to be correct, safe use of ctypes.

[trex58]

On Fedora32/PPC64LE (5.7.9-200.fc32.ppc64le), with little change:
libc = CDLL('/usr/lib64/libc.so.6')
I get the correct answer:
b'def'
b'def'
b'def'
# python3 --version
Python 3.8.3
libffi : 3.1-24

On Fedora32/x86_64 (5.7.9-200.fc32.x86_64), with a little change:
libc = CDLL('/usr/lib64/libc-2.31.so')
that crashes:
b'def'
Segmentation fault (core dumped)
# python3 --version
Python 3.8.3
libffi : 3.1-24

AIX : libffi-3.2.1
On AIX 7.2, with Python 3.8.5 compiled with XLC v13, in 64bit:
b'def'
b'def'
None
On AIX 7.2, with Python 3.8.5 compiled with GCC 8.4, in 64bit:
b'def'
b'def'
None

On AIX 7.2, with Python 3.8.5 compiled with XLC v13, in 32bit:
( libc = CDLL('libc.a(shr.o)') )
b'def'
b'def'
b'def'
On AIX 7.2, with Python 3.8.5 compiled with GCC 8.4, in 32bit:
b'def'
b'def'
b'def'

Preliminary conclusions:

• this is a 64bit issue on AIX and it is independent of the compiler
• it is worse on Fedora/x86_64
• it works perfectly on Fedora/PPC64LE
  what a mess.

[trex58]

Fedora32/x86_64

[root@destiny10 tmp]# gdb /usr/bin/python3.8 core
...
Core was generated by `python3 ./Pb.py'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007f898a02a1d8 in __memchr_sse2 () from /lib64/libc.so.6
Missing separate debuginfos, use: dnf debuginfo-install python3-3.8.3-2.fc32.x86_64
(gdb) where
#0 0x00007f898a02a1d8 in __memchr_sse2 () from /lib64/libc.so.6
#1 0x00007f898982caf0 in ffi_call_unix64 () from /lib64/libffi.so.6
#2 0x00007f898982c2ab in ffi_call () from /lib64/libffi.so.6
#3 0x00007f8989851ef1 in _ctypes_callproc.cold () from /usr/lib64/python3.8/lib-dynload/_ctypes.cpython-38-x86_64-linux-gnu.so
#4 0x00007f898985ba2f in PyCFuncPtr_call () from /usr/lib64/python3.8/lib-dynload/_ctypes.cpython-38-x86_64-linux-gnu.so
#5 0x00007f8989d6c7a1 in _PyObject_MakeTpCall () from /lib64/libpython3.8.so.1.0
#6 0x00007f8989d69111 in _PyEval_EvalFrameDefault () from /lib64/libpython3.8.so.1.0
#7 0x00007f8989d62ec4 in _PyEval_EvalCodeWithName () from /lib64/libpython3.8.so.1.0
#8 0x00007f8989dde109 in PyEval_EvalCodeEx () from /lib64/libpython3.8.so.1.0
#9 0x00007f8989dde0cb in PyEval_EvalCode () from /lib64/libpython3.8.so.1.0
#10 0x00007f8989dff028 in run_eval_code_obj () from /lib64/libpython3.8.so.1.0
#11 0x00007f8989dfe763 in run_mod () from /lib64/libpython3.8.so.1.0
#12 0x00007f8989cea81b in PyRun_FileExFlags () from /lib64/libpython3.8.so.1.0
#13 0x00007f8989cea19d in PyRun_SimpleFileExFlags () from /lib64/libpython3.8.so.1.0
#14 0x00007f8989ce153c in Py_RunMain.cold () from /lib64/libpython3.8.so.1.0
#15 0x00007f8989dd1bf9 in Py_BytesMain () from /lib64/libpython3.8.so.1.0
#16 0x00007f8989fb7042 in __libc_start_main () from /lib64/libc.so.6
#17 0x0000557a1f3c407e in _start ()

[trex58]

On AIX:

root@castor4## gdb /opt/freeware/bin/python3
...
(gdb) run -m pdb Pb.py
...
(Pdb) n
b'def'

/home2/freeware/src/packages/BUILD/Python-3.8.5/32bit/Pb.py(35)<module>()
-> print(
(Pdb) n
/home2/freeware/src/packages/BUILD/Python-3.8.5/32bit/Pb.py(36)<module>()
-> CFUNCTYPE(c_char_p, MemchrArgsHack2,
(Pdb)
Thread 2 received signal SIGINT, Interrupt.
[Switching to Thread 1]
0x090000000016426c in __fd_select () from /usr/lib/libc.a(shr_64.o)
(gdb) b ffi_call
Breakpoint 1 at 0x1217918
(gdb) c
...
(Pdb) n

Thread 2 hit Breakpoint 1, 0x0900000001217918 in ffi_call () from /opt/freeware/lib/libffi.a(libffi.so.6)
(gdb) where
#0 0x0900000001217918 in ffi_call () from /opt/freeware/lib/libffi.a(libffi.so.6)
#1 0x0900000001217780 in ffi_prep_cif_machdep () from /opt/freeware/lib/libffi.a(libffi.so.6)
#2 0x0900000001216fb8 in ffi_prep_cif_var () from /opt/freeware/lib/libffi.a(libffi.so.6)
......

(gdb) b memchr
Breakpoint 2 at 0x9000000001b0d60
(gdb) c
Continuing.

Thread 2 hit Breakpoint 2, 0x09000000001b0d60 in memchr () from /usr/lib/libc.a(shr_64.o)
(gdb) i register
r0 0x9000000001b0d60 648518346343124320
r1 0xfffffffffffc8d0 1152921504606832848
r2 0x9001000a008e8b8 648535941212334264
r3 0xa000000003669e0 720575940382845408
r4 0x64 100
r5 0x0 0
r6 0x9001000a04ee730 648535941216921392
r7 0x0 0
...
(gdb) x/s $r3
0xa000000003669e0: "abcdef"

So:

• the string is passed as r3.
• r4 contains "d" = 0x64=100
• but the size 7 is missing

Anyway, it seems that ffi does not pass the pointer, but values. However, the length 7 is missing. Not in r5, and nowhere in the other registers.

[trex58]

On Fedora/x86_64, in order to get the core, one must do:
coredumpctl -o /tmp/core dump /usr/bin/python3.8

[trex58]

On Fedora/PPC64LE, where it is OK, the same debug with gdb gives:

(gdb) where
#0 0x00007ffff7df03b0 in __memchr_power8 () from /lib64/libc.so.6
#1 0x00007fffea167680 in ?? () from /lib64/libffi.so.6
#2 0x00007fffea166284 in ffi_call () from /lib64/libffi.so.6
#3 0x00007fffea1a7fdc in _ctypes_callproc () from /usr/lib64/python3.8/lib-dynload/_ctypes.cpython-38-ppc64le-linux-gnu.so
..........
(gdb) i register
r0 0x7fffea167614 140737120728596
r1 0x7fffffffc490 140737488340112
r2 0x7fffea187f00 140737120861952
r3 0x7fffea33a140 140737122640192
r4 0x6464 25700
r5 0x7 7
r6 0x0 0
r7 0x7fffea33a147 140737122640199
r8 0x7fffea33a140 140737122640192

(gdb) x/s 0x7fffea33a140
0x7fffea33a140: "abcdef"

r3: string
r4 : 0x6464 : "d" ??
r5: 7 : length of the string !!!

[trex58]

On AIX in 32bit, we have:

Thread 2 hit Breakpoint 2, 0xd01407e0 in memchr () from /usr/lib/libc.a(shr.o)
(gdb) where
#0 0xd01407e0 in memchr () from /usr/lib/libc.a(shr.o)
#1 0xd438f480 in ffi_call_AIX () from /opt/freeware/lib/libffi.a(libffi.so.6)
#2 0xd438effc in ffi_call () from /opt/freeware/lib/libffi.a(libffi.so.6)
....
(gdb) i register
r0 0xd01407e0 3490973664
r1 0x2ff20f80 804392832
r2 0xf07a3cc0 4034542784
r3 0xb024c558 2955199832
r4 0x64 100
r5 0x7 7
r6 0x0 0
...

(gdb) x/s 0xb024c558
0xb024c558: "abcdef"

r5 is OK.

[trex58]

AIX: difference between 32bit and 64bit.

After the second print, the stack is:

32bit:
#0 0xd01407e0 in memchr () from /usr/lib/libc.a(shr.o)
#1 0xd438f480 in ffi_call_AIX () from /opt/freeware/lib/libffi.a(libffi.so.6)
#2 0xd438effc in ffi_call () from /opt/freeware/lib/libffi.a(libffi.so.6)
#3 0xd14979bc in ?? ()
#4 0xd148995c in ?? ()
#5 0xd20fd5d8 in _PyObject_MakeTpCall () from /opt/freeware/lib/libpython3.8.so

64bit:
#0 0x09000000001b0d60 in memchr () from /usr/lib/libc.a(shr_64.o)
#1 0x0900000001217f00 in ffi_closure_ASM () from /opt/freeware/lib/libffi.a(libffi.so.6)
#2 0x0900000001217aac in ffi_prep_closure_loc () from /opt/freeware/lib/libffi.a(libffi.so.6)
#3 0x0900000000d30900 in ?? ()
#4 0x0900000000d22b6c in ?? ()
#5 0x0900000000ebbc18 in _PyObject_MakeTpCall () from /opt/freeware/lib64/libpython3.8.so

So, the execution does not run in the same ffi routines in 32bit and in 64bit. Bug ?

It should be interesting to do the same with Python3 and libffi built with -O0 -g maybe.