Test every non tested opcode

[barotto]

A comment from VOGONS thread https://www.vogons.org/viewtopic.php?f=9&t=60095:

XLAT instructions (both 16-bit and 32-bit versions) were faulty: it was checking memory against a write(for the memory operand) instead of a proper read.

[superfury]

Currently left(ignoring those still pending in the other issue(pushf(d)/popf(d)), as well as protected-mode instructions and related instruction for protected mode-specific functionality(0F beginning range instructions)):

• Logical operations in the 00-3F range.
• Logical operations in the 80-83 range.
• INC reg in the 41-47 range.
• DEC reg in the 49-4F range.
• String input/output from port in the 6C-6F range, probably testable using e.g. the temporary DMA registers(unused registers which aren't used on the DMA chipset of the DMA Page Registers).
• TEST instructions(opcode 84-85).
• XCHG instructions(opcode 86-87).
• MOV instructions(opcode 88-8B).
• XCHG instructions(opcode 90-97).
• SAHF/LAHF instructions (opcode 9E-9F).
• MOV with immediate offset(opcodes A0-A3).
• 32-bit address-size string instructions(B/W/D).
• TEST AL/AX/EAX,imm(opcodes A8&A9).
• Different variants of SHLD/SHRD with operand-sizes.
• MOV reg,imm (opcodes B0-BF).
• RET near (opcode C2).
• MOV r/m,imm(opcodes C6-C7).
• ENTER/LEAVE(opcodes C6&C7).
• XLAT (opcode D7).
• LOOP/J(E)CXZ instructions 32-bit operand size(opcodes E0-E3).
• IN/OUT instructions(opcodes E4-E7, EC-EF), see string i/o instructions.
• JMP instructions(opcodes E9-EB).
• REP/REPE/REPNE (opcodes F0-F3).
• Opcode F6/F7 TEST r/m,imm
• Basic flag operations (opcodes F8-FD).
• INC r/m8/16/32 instructions(opcodes FE-FF).
• 32-bit relative jumps(0F80-0F8F instruction range).

Side note: opcode 82h is missing from the intel-opcodes list, which is an alias for opcode 80h( http://ref.x86asm.net/coder32.html#x82 ).

[superfury]

Just created a little 'testsuite' that executes many of the basic instructions mentioned in my previous post(although it requires to be logged and verified manually by looking for register/memory reads/writes/changes):
https://bitbucket.org/superfury/unipcemu/src/9c54c037466f0e079e76a35987584b0687d42af8/UniPCemu/assembly/?at=master

Although I've coded opcodes 00-3F by using raw binary statements to make sure the nasm assembler doesn't actually create opcodes outside of that range somehow(2-byte versions of them).

The only test that actually verifies itself using assembly instructions is the ret imm testing, much in the way your stack tests run.

Edit: https://bitbucket.org/superfury/unipcemu/src/6c46742934fc8ffba70de4e139671244164270e9/UniPCemu/assembly/?at=master

Fixed some bugs in the basic *D opcodes for the 00-3F range being the wrong opcode byte.

[superfury]

So, combining both testsuite results, that only leaves possible errors in:

• Logical operations in the 80-83 range**.
• String input/output from port in the 6C-6F range, probably testable using e.g. the temporary DMA registers(unused registers which aren't used on the DMA chipset of the DMA Page Registers).
• TEST instructions(opcode 84-85).
• MOV instructions(opcode 88-8B).
• SAHF/LAHF instructions (opcode 9E-9F).
• 32-bit address-size string instructions(B/W/D).
• Different variants of SHLD/SHRD with operand-sizes.
• MOV r/m,imm(opcodes C6-C7).
• ENTER/LEAVE(opcodes C8&C9).
• XLAT (opcode D7).
• LOOP/J(E)CXZ instructions 32-bit operand size(opcodes E0-E3)**.
• IN/OUT instructions(opcodes E4-E7, EC-EF), see string i/o instructions.
• JMP instructions(opcodes E9-EB).
• REP/REPE/REPNE (opcodes F0-F3).
• Basic flag operations (opcodes F8-FD).
• INC r/m8/16/32 instructions(opcodes FE-FF).
• 32-bit relative jumps(0F80-0F8F instruction range).

** partially tested and verified in both testsuites

[superfury]

Just found a little 'bug' in my emulator(which seems to be only half documented in the 80386 programmer's reference manual's far return instruction description). (E)SP is increased before popping SS:(E)SP during a stack switch to an outer privilege level(resulting CS.RPL>CPL), which is documented, but also after popping the SS:(E)SP(thus, increasing(popping) the caller's stack variables. The latter isn't documented within the 80386 programmer's reference manual as far as I can see.

[superfury]

Having fixed the far return using the immediate on both stack locations(both on source and destination stacks), the Extended Memory Tester v3.0 now properly detects the video card and extended memory, continuing to test memory:D

This app now runs properly: https://archive.org/details/msdos_TESTEXT3_shareware

[superfury]

Hmmmm..... One basic device that can be used to test the ISNS/OUTS instructions is the IDE/ATA harddisk's buffer(Command E4 to read the buffer, Command E8 to write the buffer). First fill it with a pattern manually(512 bytes), then read it back and verify the pattern. That should be able to verify all in(s)/out(s) instructions.

Edit: Although I know some of them work properly(8-bit plain, 8-bit string and 16-bit string) work already, since they're used during POST(8-bit plain), during disk reads(8-bit or 16-bit reads), disk writes(8-bit or 16-bit writes) and CD-ROM access(16-bit reads and writes). The only one of those still untested are string and normal 32-bit variants and normal 16/32-bit single input/output.

[superfury]

I see you're busy on some more (protected mode) tests. Great!

That now just leaves:

• String input/output from port in the 6C-6F range, probably testable using e.g. the temporary DMA registers(unused registers which aren't used on the DMA chipset of the DMA Page Registers) and hard disk buffer(using the read/write buffer commands).
• MOV instructions(opcode 88-8B).
• SAHF/LAHF instructions (opcode 9E-9F).
• MOV r/m,imm(opcodes C6-C7).
• ENTER/LEAVE(opcodes C8&C9).
• XLAT (opcode D7).
• LOOP/J(E)CXZ instructions 32-bit operand size(opcodes E0-E3)**.
• IN/OUT instructions(opcodes E4-E7, EC-EF), see string i/o instructions.
• JMP instructions(opcodes E9-EB).
• Basic flag operations (opcodes F8-FD).
• 32-bit relative jumps(0F80-0F8F instruction range).

So mostly various kinds of move instructions and miscellaneous instructions.

And of course all remaining protected-mode functionality itself(task switching etc.) is still left to test.

I'm currently wondering if protected-mode functionality of UniPCemu still has bugs(concerning call gates and interrupts etc. pushing data on the stack with(out) stack switch) in various cases.

[barotto]

I've found a pretty big bug on my emulator comparing its memory dump with Bochs' after the execution of the test suite: the Dirty bit of a PTE is not properly set on a write when the same page has been previously accessed by a read.
So there are a lot of behavioural tests to be implemented beside the various opcodes...

[superfury]

I've just ran Bochs 2.6.9 and dumped the 640K memory. Then I did the same with UniPCemu's MMU functionality(running the debugger, press and hold circle and then tap square) when reaching the HLT at the end of the POST 0xFF(when reaching the HLT status).

It reveals there's quite a lot wrong, apparently:

00000401 C2 00
00000402 10 00
00000405 8E 00
000020B8 67 07
000023E0 27 07
000023E4 27 07
000023E8 27 07
0000FFFC F5 00
0000FFFD FF 00
0000FFFF F0 00
0001FFB7 05 46
0001FFBB 05 32
0001FFBF 56 00
0001FFC2 00 80
0001FFC3 D3 00
0001FFC4 72 40
0001FFC7 58 DB
0001FFC8 73 FF
0001FFCB 38 00
0001FFCC AD 00
0001FFCF 0F 14
0001FFD0 AB 00
0001FFD1 00 FF
0001FFD2 00 FF
0001FFD3 01 00
0001FFD4 00 C0
0001FFD7 EB 05
0001FFD8 FF 00
0001FFDB 04 0D
0001FFDC AD 6F
0001FFDF 20 32
0001FFE2 80 00
0001FFE3 07 00
0001FFE4 08 00
0001FFE6 00 80
0001FFE7 94 00
0001FFE8 72 40
0001FFED FF 00
0001FFEE FF 00
0001FFEF 07 00
0001FFF0 08 01
0001FFF1 00 01
0001FFF3 08 84
0001FFF4 AD 00
0001FFF5 00 FF
0001FFF6 00 FF
0001FFF7 0D FF
0001FFF8 00 FF
0001FFF9 00 FF
0001FFFA 00 FF
0001FFFB 13 6E
0001FFFC 00 75
0001FFFD 00 57
0002E000 01 00
0002E001 FF 00
0002E002 FF 00
0002E003 FF 00
0009F003 00 50

So that means:

• First IDT entry has an issue.
• Page table entry issues? Seems to be the Accessed bit not being updated correctly?
• Stack overflow? (FFFC-FFFF)
• More stack issues? (1FFB7-1FFFD)
• Normal test issues? (2E000-2E003 and 9F003)

[barotto]

First IDT entry has an issue.

That's the IDT area, but it's not necessarily a problem with the IDT management, could be something that writes garbage at the wrong address.

Page table entry issues?

Possible.

Stack overflow? (FFFC-FFFF)

Maybe, or again you could be writing garbage to the wrong addresses (see below).

More stack issues? (1FFB7-1FFFD)

Maybe.

Normal test issues? (2E000-2E003 and 9F003)

Can't tell right now what the segment at 0x2e000 is used for, but dword at 0x9F000 should be 0x50465046, for some reason you're missing the last byte 0x50 at 0x9F003, or you have overwritten it with 00.

It seems like you have problems with the MMU.

[superfury]

When debugging the direct memory writes by the Paging unit, I see:

• Write to address 0x000023e0 value 0x000f8027.
• Write to address 0x000023e4 value 0x000f9027.
• Write to address 0x000023e8 value 0x000fa027.
• Write to address 0x000020b8 value 0x0002e067 (at step EE).

So the Paging unit requests the memory mapping to actually write the correct values there at some point. Maybe something else is somehow overwriting it?
Edit: Or maybe there's an issue in the writeback to memory after said breakpoint... Hmmmm...
Edit: Already confirmed that the 0x23E0 memory location receives and stores value 0x27. So it's another bug that's somehow overwriting this value?
Edit: Just added those addresses hardcoded to the MMU unit. Let's see if they're overwritten with different values normally(thus not an general emulator bug).
Edit: Looking directly in the capture of the physical RAM(with memory gaps having been removed when dumping the RAM), the values written are actually in RAM at their supposed locations?

Edit: Looking again at my own created logs(made using MinGW-w64 using the following script), from UniPCemu's ROM directory(which contains the Bochs dump as well as the ROMs):

cp "..\captures\memory.dat" "fullmemory.dat"
dd if=/dev/null of=fullmemory.dat bs=1 count=1 seek=655360
cmp -l fullmemory.dat bochs-640k-memdump.bin | gawk '{printf "%08X %02X %02X\n", ($1-1), strtonum(0$2), strtonum(0$3)}'>memory.cmp.txt

It copies the memory.dat to the ROM directory for processing, then edits it to become 640K large, finally executing the difference dump as per your documentation.

I thought that the left was the Bochs dump, while the right value was my own. Looking at it again(as well as reading your documentation on the results of the gawk command again), it's reversed.

So those three pages aren't supposed to be accessed(since bits 5(PTE&PDE)&6(PDE) are supposed to be cleared instead of set in the end of logging). But they're accessed anyways, which isn't supposed to happen? So those paged accesses on those locations are instructions addressing a wrong point in memory, or incorrect mapping somehow?

[barotto]

PTE at 0x23e0 is for the page at 0xF8000. Its Accessed bit should be 0, not 1 (byte at 0x23e0 should be 0x07 not 0x27). It seems like you're fetching the NOPs that are present in that area of the test code. Or maybe your MMU is updating the wrong PTE.

I also think the dword at 0x9F000 is interesting. See what is wrinting 00 at 0x9F003, overwriting the 0x50.