Formats failing with POCL

[solardiz]

It's surprisingly few now. Running on Debian 13 under qemu-kvm on our ancient Scientific Linux 6.10 on "super", which somehow makes only basic x86-64 SSE2 available in the VM (not even SSSE3 is seen via CPUID inside the VM, let alone AVX). Also the host code built with "Debian clang version 19.1.7 (3+b1)" (./configure CC=clang, no warnings during the build, although Debian 13's gcc 14.2 also works fine).

user@debian:~/john/src$ OMP_NUM_THREADS=31 ../run/john -te=0 2>&1 | tee -a ../run/log-clang
Will run 31 OpenMP threads
Testing: descrypt, traditional crypt(3) [DES 128/128 SSE2]... (31xOMP) PASS
[...]
Testing: HMAC-SHA512 [password is key, SHA512 128/128 SSE2 2x]... (31xOMP) PASS
Device 1: cpu-x86-64-QEMU Virtual CPU version (cpu64-rhel6)
Testing: AndroidBackup-opencl [PBKDF2-SHA1 AES OpenCL 4x]... PASS
Testing: agilekeychain-opencl, 1Password Agile Keychain [PBKDF2-SHA1 AES OpenCL]... PASS
Testing: ansible-opencl, Ansible Vault [PBKDF2-SHA256 HMAC-SHA256 OpenCL]... PASS
Testing: argon2-opencl, Argon2 [BlaMka OpenCL]... FAILED (cmp_all(1))
Testing: axcrypt-opencl [AES SHA1 OpenCL]... PASS
Testing: axcrypt2-opencl, AxCrypt 2.x [PBKDF2-SHA512 AES OpenCL]... PASS
[...]
Testing: KeePass-opencl [AES OpenCL]... PASS
Testing: KeePass-Argon2-opencl [BlaMka OpenCL]... FAILED (cmp_one(1))
Testing: keychain-opencl, Mac OS X Keychain [PBKDF2-SHA1 3DES OpenCL]... PASS
Testing: keyring-opencl, GNOME Keyring [SHA256 AES OpenCL]... PASS
Testing: keystore-opencl, Java KeyStore [SHA1 OpenCL]... PASS
Testing: krb5pa-md5-opencl, Kerberos 5 AS-REQ Pre-Auth etype 23 [MD4 HMAC-MD5 RC4 OpenCL]... PASS
Testing: krb5pa-sha1-opencl, Kerberos 5 AS-REQ Pre-Auth etype 17/18 [PBKDF2-SHA1 (AES-CTS-HMAC-SHA1-96) OpenCL 4x]... PASS
Testing: krb5tgs-opencl, Kerberos 5 TGS-REP etype 23 [MD4 HMAC-MD5 RC4 OpenCL]... FAILED (crypt_all(1) zero return)
Testing: krb5asrep-aes-opencl, Kerberos 5 AS-REP etype 17/18 [PBKDF2-SHA1 AES OpenCL 4x]... PASS
Testing: krb5tgs-sha1-opencl, Kerberos 5 TGS-REP etype 17/18 [PBKDF2-SHA1 AES-CTS OpenCL 4x]... PASS
[...]
Testing: NT-long-opencl [MD4 OpenCL]... PASS
Testing: o5logon-opencl, Oracle O5LOGON protocol [MD5 SHA1 AES OpenCL]... 0: OpenCL CL_INVALID_VALUE (-30) error in opencl_o5logon_fmt_plug.c:301 - Failed writing buffer
PASS
Testing: ODF-opencl, OpenDocument Star/Libre/OpenOffice [PBKDF2-SHA1 BF/AES OpenCL]... PASS
[...]
Testing: PBKDF2-HMAC-SHA512-opencl, GRUB2 / OS X 10.8+ [PBKDF2-SHA512 OpenCL]... PASS
Testing: pdf-opencl, PDF encrypted document [MD5-RC4 / SHA2-AES OpenCL]... 0: OpenCL CL_INVALID_VALUE (-30) error in opencl_pdf_fmt_plug.c:351 - Failed writing buffer
PASS
Testing: pem-opencl, PKCS#8 private key (RSA/DSA/ECDSA) [PBKDF2-SHA1 3DES/AES OpenCL 4x]... PASS
[...]
3 out of 535 tests have FAILED

So it's 3 failing formats (two Argon2 and krb5tgs-opencl), and 2 more reporting an OpenCL error yet passing test (we could want to look into exact meaning of that).

user@debian:~$ clang -v
Debian clang version 19.1.7 (3+b1)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm-19/bin
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/14
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/14
Candidate multilib: .;@m64
Selected multilib: .;@m64

user@debian:~$ dpkg -l | grep -E 'ocl|opencl'
ii  libpocl-dev                           6.0-6                                amd64        development files for the PoCL library
ii  libpocl2-common                       6.0-6                                all          common files for the PoCL library
ii  libpocl2t64:amd64                     6.0-6                                amd64        Portable Computing Language library
ii  ocl-icd-libopencl1:amd64              2.3.3-1                              amd64        Generic OpenCL ICD Loader
ii  ocl-icd-opencl-dev:amd64              2.3.3-1                              amd64        OpenCL development files
ii  opencl-c-headers                      3.0~2024.10.24-2                     all          OpenCL (Open Computing Language) C header files
ii  opencl-clhpp-headers                  3.0~2024.10.24-2                     all          C++ headers for OpenCL development
ii  opencl-headers                        3.0~2024.10.24-2                     all          OpenCL (Open Computing Language) header files
ii  pocl-opencl-icd:amd64                 6.0-6                                amd64        PoCL ICD

[solardiz]

With older POCL pocl-1.7-2.fc34.x86_64 testing current #5884, the failures are partially different:

Testing: argon2-opencl, Argon2 [BlaMka OpenCL]... FAILED (cmp_all(1))
Testing: descrypt-opencl, traditional crypt(3) [DES OpenCL]... FAILED (get_key(0))
Testing: sha1crypt-opencl, (NetBSD) [PBKDF1-SHA1 OpenCL 16x]... FAILED (cmp_all(1))
Testing: KeePass-Argon2-opencl [BlaMka OpenCL]... FAILED (cmp_one(1))
Testing: pgpdisk-opencl, PGP Disk / Virtual Disk [SHA1 AES/TwoFish/CAST OpenCL]... FAILED (cmp_all(1))
5 out of 101 tests have FAILED

(I skipped the PASS lines from this copy-paste.)

None of these failures look new with this PR's changes (also tested these formats with a build from prior to the PR, same failures).

No OpenCL errors are reported.

[solardiz]

Tried #5873 on the Debian 13 built with ASan, got this:

Testing: KeePass-opencl [AES OpenCL]... =================================================================
==73057==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x513000bad517 at pc 0x7faa85cf2936 bp 0x7faa15438b50 sp 0x7faa15438310
READ of size 1063 at 0x513000bad517 thread T51
    #0 0x7faa85cf2935 in memcpy ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors_memintrinsics.inc:115
    #1 0x7faa4fe0daf6 in pocl_exec_command (/lib/x86_64-linux-gnu/libpocl.so.2.13.0+0x94af6) (BuildId: 2cadeb853d356489f81090e75be07a720bd2c3f1)
    #2 0x7faa82b1b73d  (/lib/x86_64-linux-gnu/pocl/libpocl-devices-pthread.so+0x373d) (BuildId: 973825cd305df2873d4052849dfc1fe3c43e67f6)
    #3 0x7faa85c5b1d5 in asan_thread_start ../../../../src/libsanitizer/asan/asan_interceptors.cpp:234
    #4 0x7faa851acb7a  (/lib/x86_64-linux-gnu/libc.so.6+0x92b7a) (BuildId: def5460e3cee00bfee25b429c97bcc4853e5b3a8)
    #5 0x7faa8522a7b7  (/lib/x86_64-linux-gnu/libc.so.6+0x1107b7) (BuildId: def5460e3cee00bfee25b429c97bcc4853e5b3a8)

0x513000bad517 is located 0 bytes after 343-byte region [0x513000bad3c0,0x513000bad517)
allocated by thread T0 here:
    #0 0x7faa85cf4c57 in malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x55eb54d85769 in mem_alloc /home/user/s/john/src/memory.c:92
    #2 0x55eb54f3d4de in keepass_get_salt /home/user/s/john/src/keepass_common.h:440
    #3 0x55eb54d4eb61 in fmt_self_test_body /home/user/s/john/src/formats.c:1211
    #4 0x55eb54d51237 in fmt_self_test /home/user/s/john/src/formats.c:2079
    #5 0x55eb54d2af59 in benchmark_all /home/user/s/john/src/bench.c:883
    #6 0x55eb54d5f9b9 in john_run /home/user/s/john/src/john.c:1695
    #7 0x55eb54d5f9b9 in main /home/user/s/john/src/john.c:2117
    #8 0x7faa85143ca7  (/lib/x86_64-linux-gnu/libc.so.6+0x29ca7) (BuildId: def5460e3cee00bfee25b429c97bcc4853e5b3a8)

Thread T51 created by T0 here:
    #0 0x7faa85cecae1 in pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:245
    #1 0x7faa82b1c1fa  (/lib/x86_64-linux-gnu/pocl/libpocl-devices-pthread.so+0x41fa) (BuildId: 973825cd305df2873d4052849dfc1fe3c43e67f6)

SUMMARY: AddressSanitizer: heap-buffer-overflow ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors_memintrinsics.inc:115 in memcpy
Shadow bytes around the buggy address:
  0x513000bad280: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x513000bad300: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa
  0x513000bad380: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x513000bad400: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x513000bad480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x513000bad500: 00 00[07]fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x513000bad580: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x513000bad600: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x513000bad680: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x513000bad700: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x513000bad780: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd

I doubt it's related to the PR's changes, so posting in here for now, but will re-test without the PR.

[solardiz]

I doubt it's related to the PR's changes, so posting in here for now, but will re-test without the PR.

Indeed, getting the same crash when testing just this one format with current bleeding-jumbo.

[solardiz]

Next one:

Testing: krb5tgs-opencl, Kerberos 5 TGS-REP etype 23 [MD4 HMAC-MD5 RC4 OpenCL]... =================================================================
==89149==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x5190009f78a9 at pc 0x7f95d2af2936 bp 0x7f95753acb50 sp 0x7f95753ac310
READ of size 1185 at 0x5190009f78a9 thread T43
    #0 0x7f95d2af2935 in memcpy ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors_memintrinsics.inc:115
    #1 0x7f959d410af6 in pocl_exec_command (/lib/x86_64-linux-gnu/libpocl.so.2.13.0+0x94af6) (BuildId: 2cadeb853d356489f81090e75be07a720bd2c3f1)
    #2 0x7f95d110373d  (/lib/x86_64-linux-gnu/pocl/libpocl-devices-pthread.so+0x373d) (BuildId: 973825cd305df2873d4052849dfc1fe3c43e67f6)
    #3 0x7f95d2a5b1d5 in asan_thread_start ../../../../src/libsanitizer/asan/asan_interceptors.cpp:234
    #4 0x7f95d1facb7a  (/lib/x86_64-linux-gnu/libc.so.6+0x92b7a) (BuildId: def5460e3cee00bfee25b429c97bcc4853e5b3a8)
    #5 0x7f95d202a7b7  (/lib/x86_64-linux-gnu/libc.so.6+0x1107b7) (BuildId: def5460e3cee00bfee25b429c97bcc4853e5b3a8)

0x5190009f78a9 is located 0 bytes after 1065-byte region [0x5190009f7480,0x5190009f78a9)
allocated by thread T0 here:
    #0 0x7f95d2af4610 in calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:77
    #1 0x559cf2fec86a in mem_calloc /home/user/s/john/src/memory.c:109
    #2 0x559cf2e25d1f in krb5tgs_get_salt /home/user/s/john/src/krb5_tgs_common_plug.c:161
    #3 0x559cf2fb5b61 in fmt_self_test_body /home/user/s/john/src/formats.c:1211
    #4 0x559cf2fb8237 in fmt_self_test /home/user/s/john/src/formats.c:2079
    #5 0x559cf2f91f59 in benchmark_all /home/user/s/john/src/bench.c:883
    #6 0x559cf2fc69b9 in john_run /home/user/s/john/src/john.c:1695
    #7 0x559cf2fc69b9 in main /home/user/s/john/src/john.c:2117
    #8 0x7f95d1f43ca7  (/lib/x86_64-linux-gnu/libc.so.6+0x29ca7) (BuildId: def5460e3cee00bfee25b429c97bcc4853e5b3a8)

Thread T43 created by T0 here:
    #0 0x7f95d2aecae1 in pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:245
    #1 0x7f95d11041fa  (/lib/x86_64-linux-gnu/pocl/libpocl-devices-pthread.so+0x41fa) (BuildId: 973825cd305df2873d4052849dfc1fe3c43e67f6)

SUMMARY: AddressSanitizer: heap-buffer-overflow ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors_memintrinsics.inc:115 in memcpy
Shadow bytes around the buggy address:
  0x5190009f7600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x5190009f7680: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x5190009f7700: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x5190009f7780: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x5190009f7800: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x5190009f7880: 00 00 00 00 00[01]fa fa fa fa fa fa fa fa fa fa
  0x5190009f7900: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x5190009f7980: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x5190009f7a00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x5190009f7a80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x5190009f7b00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa

Looks surprisingly similar. Could be a POCL bug.

[solardiz]

Next:

Testing: o5logon-opencl, Oracle O5LOGON protocol [MD5 SHA1 AES OpenCL]... 0: OpenCL CL_INVALID_VALUE (-30) error in opencl_o5logon_fmt_plug.c:301 - Failed writing buffer
=================================================================
==90042==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x5110066b6278 at pc 0x55b08ec58b95 bp 0x7ffd32f883c0 sp 0x7ffd32f883b8
READ of size 4 at 0x5110066b6278 thread T0
    #0 0x55b08ec58b94 in get_key /home/user/s/john/src/opencl_o5logon_fmt_plug.c:250
    #1 0x55b08e9d25a6 in is_key_right /home/user/s/john/src/formats.c:680
    #2 0x55b08e9dab61 in fmt_self_test_body /home/user/s/john/src/formats.c:1319
    #3 0x55b08e9dc237 in fmt_self_test /home/user/s/john/src/formats.c:2079
    #4 0x55b08e9b5f59 in benchmark_all /home/user/s/john/src/bench.c:883
    #5 0x55b08e9ea9b9 in john_run /home/user/s/john/src/john.c:1695
    #6 0x55b08e9ea9b9 in main /home/user/s/john/src/john.c:2117
    #7 0x7ff95de33ca7  (/lib/x86_64-linux-gnu/libc.so.6+0x29ca7) (BuildId: def5460e3cee00bfee25b429c97bcc4853e5b3a8)
    #8 0x7ff95de33d64 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29d64) (BuildId: def5460e3cee00bfee25b429c97bcc4853e5b3a8)
    #9 0x55b08e55a1b0 in _start (/home/user/s/john/run/john+0x3151b0) (BuildId: 6d13e30b09e9f1503c3ed33086fe60b59ce33e81)

0x5110066b6278 is located 8 bytes before 128-byte region [0x5110066b6280,0x5110066b6300)
allocated by thread T0 here:
    #0 0x7ff95e8f424d in posix_memalign ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x7ff9289e25cb in pocl_aligned_malloc (/lib/x86_64-linux-gnu/libpocl.so.2.13.0+0x695cb) (BuildId: 2cadeb853d356489f81090e75be07a720bd2c3f1)

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/user/s/john/src/opencl_o5logon_fmt_plug.c:250 in get_key
Shadow bytes around the buggy address:
  0x5110066b5f80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x5110066b6000: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x5110066b6080: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x5110066b6100: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x5110066b6180: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x5110066b6200: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa[fa]
  0x5110066b6280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x5110066b6300: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x5110066b6380: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x5110066b6400: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x5110066b6480: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa

[solardiz]

==90042==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x5110066b6278 at pc 0x55b08ec58b95 bp 0x7ffd32f883c0 sp 0x7ffd32f883b8
READ of size 4 at 0x5110066b6278 thread T0
#0 0x55b08ec58b94 in get_key /home/user/s/john/src/opencl_o5logon_fmt_plug.c:250

This one may actually be caused by the PR. This is running against the unmerged PR.

[solardiz]

This one may actually be caused by the PR. This is running against the unmerged PR.

But no, same crash when testing just this format without the PR. Still could be our bug, for a change.

[solardiz]

Next is:

Testing: pdf-opencl, PDF encrypted document [MD5-RC4 / SHA2-AES OpenCL]... 0: OpenCL CL_INVALID_VALUE (-30) error in opencl_pdf_fmt_plug.c:351 - Failed writing buffer
=================================================================
==92219==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x511003fc3efc at pc 0x55b559f7ac5a bp 0x7ffccf1bac30 sp 0x7ffccf1bac28
READ of size 4 at 0x511003fc3efc thread T0
    #0 0x55b559f7ac59 in get_key /home/user/s/john/src/opencl_pdf_fmt_plug.c:290
    #1 0x55b559cb55a6 in is_key_right /home/user/s/john/src/formats.c:680
    #2 0x55b559cbdb61 in fmt_self_test_body /home/user/s/john/src/formats.c:1319
    #3 0x55b559cbf237 in fmt_self_test /home/user/s/john/src/formats.c:2079
    #4 0x55b559c98f59 in benchmark_all /home/user/s/john/src/bench.c:883
    #5 0x55b559ccd9b9 in john_run /home/user/s/john/src/john.c:1695
    #6 0x55b559ccd9b9 in main /home/user/s/john/src/john.c:2117
    #7 0x7f189bf43ca7  (/lib/x86_64-linux-gnu/libc.so.6+0x29ca7) (BuildId: def5460e3cee00bfee25b429c97bcc4853e5b3a8)
    #8 0x7f189bf43d64 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29d64) (BuildId: def5460e3cee00bfee25b429c97bcc4853e5b3a8)
    #9 0x55b55983d1b0 in _start (/home/user/s/john/run/john+0x3151b0) (BuildId: 6d13e30b09e9f1503c3ed33086fe60b59ce33e81)

0x511003fc3efc is located 4 bytes before 128-byte region [0x511003fc3f00,0x511003fc3f80)
allocated by thread T0 here:
    #0 0x7f189caf424d in posix_memalign ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x7f1899bef5cb in pocl_aligned_malloc (/lib/x86_64-linux-gnu/libpocl.so.2.13.0+0x695cb) (BuildId: 2cadeb853d356489f81090e75be07a720bd2c3f1)

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/user/s/john/src/opencl_pdf_fmt_plug.c:290 in get_key
Shadow bytes around the buggy address:
  0x511003fc3c00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x511003fc3c80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x511003fc3d00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x511003fc3d80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x511003fc3e00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x511003fc3e80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa[fa]
  0x511003fc3f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

This is with #5873 including @magnumripper's latest commit changing this kernel. However, I get same crash without this PR.

[solardiz]

Next is, same crash with and without the PR:

Testing: raw-MD4-opencl [MD4 OpenCL]... =================================================================
==92372==ERROR: AddressSanitizer: global-buffer-overflow on address 0x55e4f9a68946 at pc 0x55e4f8d4a2ea bp 0x7ffc51b9df10 sp 0x7ffc51b9df08
READ of size 1 at 0x55e4f9a68946 thread T0
    #0 0x55e4f8d4a2e9 in get_binary /home/user/s/john/src/opencl_rawmd4_fmt_plug.c:299
    #1 0x55e4f8a6c358 in ldr_load_pw_line /home/user/s/john/src/loader.c:976
    #2 0x55e4f8a6fb1b in ldr_init_test_db /home/user/s/john/src/loader.c:1311
    #3 0x55e4f8a2af47 in benchmark_all /home/user/s/john/src/bench.c:880
    #4 0x55e4f8a5f9b9 in john_run /home/user/s/john/src/john.c:1695
    #5 0x55e4f8a5f9b9 in main /home/user/s/john/src/john.c:2117
    #6 0x7f7c0d833ca7  (/lib/x86_64-linux-gnu/libc.so.6+0x29ca7) (BuildId: def5460e3cee00bfee25b429c97bcc4853e5b3a8)
    #7 0x7f7c0d833d64 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29d64) (BuildId: def5460e3cee00bfee25b429c97bcc4853e5b3a8)
    #8 0x55e4f85cf1b0 in _start (/home/user/s/john/run/john+0x3151b0) (BuildId: 6d13e30b09e9f1503c3ed33086fe60b59ce33e81)

0x55e4f9a68946 is located 58 bytes before global variable 'out' defined in 'opencl_rawmd4_fmt_plug.c:294:18' (0x55e4f9a68980) of size 64
0x55e4f9a68946 is located 0 bytes after global variable 'out' defined in 'opencl_rawmd4_fmt_plug.c:282:14' (0x55e4f9a68920) of size 38
  'out' is ascii string '$MD4$8a9d093f14f8701df17732b2bb182c74'
SUMMARY: AddressSanitizer: global-buffer-overflow /home/user/s/john/src/opencl_rawmd4_fmt_plug.c:299 in get_binary
Shadow bytes around the buggy address:
  0x55e4f9a68680: 00 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9
  0x55e4f9a68700: 00 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9
  0x55e4f9a68780: 00 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9
  0x55e4f9a68800: 00 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9
  0x55e4f9a68880: 00 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9
=>0x55e4f9a68900: 00 00 00 00 00 00 00 00[06]f9 f9 f9 f9 f9 f9 f9
  0x55e4f9a68980: 00 00 00 00 00 00 00 00 f9 f9 f9 f9 00 00 00 00

[solardiz]

And indeed also:

Testing: raw-MD5-opencl [MD5 OpenCL]... =================================================================
==92873==ERROR: AddressSanitizer: global-buffer-overflow on address 0x55865aaa4eac at pc 0x558659d8c0ea bp 0x7fff7610a2b0 sp 0x7fff7610a2a8
READ of size 1 at 0x55865aaa4eac thread T0
    #0 0x558659d8c0e9 in get_binary /home/user/s/john/src/opencl_rawmd5_fmt_plug.c:324
    #1 0x558659aa8358 in ldr_load_pw_line /home/user/s/john/src/loader.c:976
    #2 0x558659aabb1b in ldr_init_test_db /home/user/s/john/src/loader.c:1311
    #3 0x558659a66f47 in benchmark_all /home/user/s/john/src/bench.c:880
    #4 0x558659a9b9b9 in john_run /home/user/s/john/src/john.c:1695
    #5 0x558659a9b9b9 in main /home/user/s/john/src/john.c:2117
    #6 0x7f14fe233ca7  (/lib/x86_64-linux-gnu/libc.so.6+0x29ca7) (BuildId: def5460e3cee00bfee25b429c97bcc4853e5b3a8)
    #7 0x7f14fe233d64 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29d64) (BuildId: def5460e3cee00bfee25b429c97bcc4853e5b3a8)
    #8 0x55865960b1b0 in _start (/home/user/s/john/run/john+0x3151b0) (BuildId: 6d13e30b09e9f1503c3ed33086fe60b59ce33e81)

0x55865aaa4eac is located 52 bytes before global variable 'out' defined in 'opencl_rawmd5_fmt_plug.c:319:18' (0x55865aaa4ee0) of size 64
0x55865aaa4eac is located 0 bytes after global variable 'out' defined in 'opencl_rawmd5_fmt_plug.c:302:14' (0x55865aaa4e80) of size 44
  'out' is ascii string '$dynamic_0$5a105e8b9d40e1329780d62ea2265d8a'
SUMMARY: AddressSanitizer: global-buffer-overflow /home/user/s/john/src/opencl_rawmd5_fmt_plug.c:324 in get_binary
Shadow bytes around the buggy address:
  0x55865aaa4c00: 00 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9
  0x55865aaa4c80: 00 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9
  0x55865aaa4d00: 00 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9
  0x55865aaa4d80: 00 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9
  0x55865aaa4e00: 00 00 00 00 00 00 00 00 01 f9 f9 f9 f9 f9 f9 f9
=>0x55865aaa4e80: 00 00 00 00 00[04]f9 f9 f9 f9 f9 f9 00 00 00 00
  0x55865aaa4f00: 00 00 00 00 f9 f9 f9 f9 00 00 00 00 00 00 00 f9

[solardiz]

So it's 3 failing formats (two Argon2 and krb5tgs-opencl), and 2 more reporting an OpenCL error yet passing test (we could want to look into exact meaning of that).

Curiously, with the ASan-crashing formats excluded, only the two Argon2 formats still fail or report any OpenCL error. The 1 other failing format and 2 OpenCL errors coincide with ASan crashes, maybe for good reasons.

[solardiz]

Also, somehow they fail at higher indices when run with --test-full:

Testing: argon2-opencl, Argon2 [BlaMka OpenCL]... FAILED (cmp_all(4))
Testing: KeePass-Argon2-opencl [BlaMka OpenCL]... FAILED (cmp_one(6))

[solardiz]

There are several bugs in host code here (not POCL-specific, I wonder why we didn't detect these before). Figured out the simplest so far:

+++ b/src/opencl_rawmd4_fmt_plug.c
@@ -291,7 +291,7 @@ static char *split(char *ciphertext, int index, struct fmt_main *self)
 
 static void *get_binary(char *ciphertext)
 {
-       static uint32_t out[DIGEST_SIZE];
+       static uint32_t out[DIGEST_SIZE / 4];
        char *p;
        int i;
        p = ciphertext + TAG_LENGTH;

+++ b/src/opencl_rawmd5_fmt_plug.c
@@ -316,7 +316,7 @@ static char *split(char *ciphertext, int index, struct fmt_main *self)
 
 static void *get_binary(char *ciphertext)
 {
-       static uint32_t out[DIGEST_SIZE];
+       static uint32_t out[DIGEST_SIZE / 4];
        char *p;
        int i;
        p = ciphertext + TAG_LENGTH;