Incorrect disassembly of call/jmp reg

[neldredge]

Assemble the following file with nasm -f bin -o foo.bin foo.asm:

bits 64
	call rax

The output is a two-byte file 0xff 0xd0 which I believe is correct. Trying to disassemble this file with ndisasm -b 64 foo.bin outputs:

00000000  FF                db 0xff
00000001  D0                db 0xd0

so ndisasm apparently fails to recognize these bytes as a valid instruction.

The same issue seems to affect call reg as well as jmp reg for any register, and affects disassembly in -b 32 and -b 16 as well.

Memory operands like call [rax] are disassembled correctly.

This is nasm/ndisasm 3.01 running on MacOS ARM64, installed via homebrew.

[neldredge]

It looks like the issue is that insnsd.c contains

    /* 1195 : CALL rm64|near [m:        nw o64 ff /2                      ] LONG,BND,PROT,X86_64 */
        /* 327 324 001(ff) 202 : NWSIZE,LONG,BND,PROT,X86_64 */
        {I_CALL, 1, {RM_GPR|BITS64|NEAR,0,0,0,0}, NO_DECORATOR, nasm_bytecodes+66983, 222},

The NEAR bit seems to get included in the register flags, and register R_RAX doesn't have this flag, so it fails to match in whichreg.

[neldredge]

I bisected on when the NEAR flag appeared in insnsd.c and found ea90c8f. I can't actually run the testcase on that commit, as ndisasm didn't build.

[neldredge]

jmpe is also affected, just for completeness.