Report of issues encountered during HLK testing across various system versions

[zjmletang]

hi, I have been recently conducting HLK tests on several Windows versions using AutoHCK. During this process, I encountered a few issues

1. Missing download_url in HLK11_23H2.json
   I found that the configuration file HLK11_23H2.json is missing the download_url field. This prevents the automated workflow from downloading the necessary HLK setup files for Windows 11 23H2.

2. the boot disk cannot be detected in UEFI mode
   I am experiencing an issue where the boot disk cannot be detected during installation/boot for the following UEFI-based targets:

Windows Server 2016 x64
Windows Server 2025 x64
Windows 11 24H2 x64
I have tried applying the ISO patches as suggested in the AutoHCK documentation, but the issue persists on these specific versions.

3. Proposed addition: Win2025x64_bios
   As a temporary workaround for the UEFI storage issue mentioned above, I have locally added a Win2025x64_bios target. I noticed this is currently not present in the codebase. This allows us to complete the tests on Windows Server 2025. Can we add this 2025 BIOS configuration? I'm not sure if there were other considerations for why it’s currently missing.
4. Win2016x64_bios test issue
   When testing with the Win2016x64_bios target, the process fails to proceed to the actual test execution phase. Taking netkvm as an example, the execution consistently hangs at the following step:
   Adding target Red Hat VirtIO Ethernet Adapter...

Finally, Merry Christmas.

[kostyanf14]

Hi!

Missing download_url in HLK11_23H2.json

Expected. MSFT removed the download link for HLK11_23H2 from the public portal when we updated URLs. The old installer does not work. Only one possibility to use HLK 23H2, download it manually from untrusted sources.

The boot disk cannot be detected in UEFI mode

Can you please share some screenshots from the VM? This is strange. I reinstalled 24H2 a week ago, and it works.

Proposed addition: Win2025x64_bios

We can add it, but we don't see usage for BIOS in 2025. Also, MSFT recommended using UEFI. This is why we change defaults to UEFI and don't add new BIOS platforms.

Win2016x64_bios test issue

Can't answer now. We haven't tested Server 2016 for a long time. If you can run AutoHCK with verbose logs and share the full log it can be helpful.

Merry Christmas to you, too

[zjmletang]

@kostyanf14 thanks
Can you please share some screenshots from the VM? This is strange. I reinstalled 24H2 a week ago, and it works.
Here's my problem: my ISO file supports both BIOS and UEFI, and sometimes I mess up the settings when creating the bootable drive

Can't answer now. We haven't tested Server 2016 for a long time. If you can run AutoHCK with verbose logs and share the full log it can be helpful.
I’ve found that this issue is essentially the same as issue #823. It occurs intermittently and is fundamentally caused by an error during installation. You can refer to the logs in that previous issue for more details."

[zjmletang]

@kostyanf14 I'd like to ask: have you encountered situations where, when running multiple tests concurrently, a few of them tend to get stuck and remain in the "inQueue" state indefinitely? I'm wondering if our AutoHCK has any limitations regarding concurrent test execution.

[zjmletang]

Hi @kostyanf14

While reviewing the current HLK configuration, I noticed that the following two test items are being skipped:

[NDISTest 6.0 - [1 Machine] - 1c_Mini6RSSOids]
[DF - Embedded Signature Verification Test (Tuning and Validation)]
Regarding the first one, [NDISTest 6.0 - [1 Machine] - 1c_Mini6RSSOids], my testing shows that it passes consistently if we simply increase the vCPU count for the guest. As for the [DF - Embedded Signature Verification Test (Tuning and Validation)], it is currently passing and is explicitly included in the playlist（At the very least, they are included in the Windows Server 2022 playlist）

Given that both are playlist items, I’m concerned that skipping them might result in a rejection from Microsoft during the driver signing submission. Could you share the rationale for skipping these two cases? I want to ensure I’m not overlooking any specific environment constraints or historical issues.

[kostyanf14]

I'd like to ask: have you encountered situations where, when running multiple tests concurrently, a few of them tend to get stuck and remain in the "inQueue" state indefinitely? I'm wondering if our AutoHCK has any limitations regarding concurrent test execution.

Yes and no. AutoHCK can wait for multiple tests concurrently, but I know several places where "NullReferenceException" can occur if you manually run tests via VNC.

skipped: [NDISTest 6.0 - [1 Machine] - 1c_Mini6RSSOids]

1. In the default configuration, we have 4/8 CPUs, while NetKVM requires a minimum of 32
2. Removed for Server 2025

skipped: [DF - Embedded Signature Verification Test (Tuning and Validation)]

We see failures sometimes for some self-signed drivers. As this is the default for HCK-CI testing in virtio-win, we decided to disable this for ALL drivers. In Red Hat, we use an attestation-signed driver if this test fails.

[kostyanf14]

@zjmletang, I would ask you to close this task about multiple issues and open a new one for the independent error that still exists. I want to make each issue independently small.

[zjmletang]

@kostyanf14 ok. I will open a new one for the independent error that still exists