Problem with D3D11TextureKmtBit

[sven-axm]

First of all, your project is a great demonstration of how to do interop between DirectX and Vulkan!

But I ran into an issue when running the WPF example on my laptop. My laptop has a dedicated Nvidia GPU and an integrated Intel GPU. When selecting the Nvidia GPU as the preferred card in the Nvidia Control Panel, it works fine. But when selecting the integrated GPU it crashes (device lost). I enabled the validation layer and found the problem that my integrated GPU does not support the D3D11TextureKmtBit as a ExternalMemoryHandleTypeFlag. It does support D3D11TextureBit though. But I can't just change the flag because then it also crahes (out of device memory, also with the Nvidia GPU).

Do you have any ideas about whether it can work with the D3D11TextureBit and if so, how to make it work?

[malstraem]

out of device memory

Sounds like import needs dedicated allocation. I'll investigate this once I get my hands on a laptop with an integrated intel card 😅

[sven-axm]

Thanks! I think that the problem should be fixed if you can change the flag to VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_BIT and get it to run on a normal dedicated GPU. Then it should also run on the integrated GPU. At least I hope so ^^
Some more thoughts about the problem:
In your example you are using IDXGIResource::GetSharedHandle to obtain the shared handle. According to the specification this should work with the VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_KMT_BIT. But for the VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_BIT you are supposed to use IDXGIResource1::CreateSharedHandle and get a NT handle. Maybe this requires some other changes, too.

The Kmt flag seems to be the old handle type which might not be supported by modern drivers as stated in the second issue here. This would explain why it does not work on the integrated Intel GPU. I don't know if it is possible to use the NT shared handle type with DirectX 9.

[malstraem]

@sven-axm, Vulkan documentation is quite confusing and hard to navigate, but here are a few points:

1. CompatibleHandleTypes of VkExternalMemoryProperties should be checked to contains VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_BIT and VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_KMT_BIT. Depending on the value its need to select a paths with get or create shared handles. KMT flag indicates that D3D11 texture should be created with D3D11_RESOURCE_MISC_SHARED_NTHANDLE additionaly to D3D11_RESOURCE_MISC_SHARED and imported using created handle.

2. Docs says that ExternalMemoryFeatures of VkExternalMemoryProperties should be checked for VK_EXTERNAL_MEMORY_FEATURE_DEDICATED_ONLY_BIT to procces dedicated allocation logic, but I see Nvidia RTX3060/1060 and Intel HD 630 works without. Docs also says (does this mean that dedicated allocations can be ignored on all devices?):

Because their semantics in external APIs roughly align with that of an image or buffer with a dedicated allocation in Vulkan, implementations are required to report VK_EXTERNAL_MEMORY_FEATURE_DEDICATED_ONLY_BIT for the following external handle types:
VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_BIT
VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_KMT_BIT
...

3. Sample needs force adapter for correct work - logic should be rewritten to chooses DXGI adapter based on the Vulkan choice. In order for Vulkan and DirectX to use the same physical device, the LUID must be checked.

I pushed hard coded paths and now you can check behavior on your card. WinUI sample should be work, but WPF is broken cause D3D9 texture is not capable with IDXGIResource1::CreateSharedHandle. Need to find out is there any workarounds to create D3D9 texture according to D3D11_RESOURCE_MISC_SHARED_NTHANDLE. If there is not, logic can be reversed - create D311 texture with needed flags and open it on the D3D9 side.

[sven-axm]

That sounds like a good way of handling it in WinUI. For WPF and D3D9 I also can't get it to work. Even if I reverse the creation order of the textures (first creating D3D11 texture and opening it with D3D9 by passing the handle in the CreateTexture method), it fails when calling CreateTexture with "Value does not fall within the expected range".

Here's what I did:

    var texture2DDesc = new Texture2DDesc()
{
	BindFlags = (uint)(BindFlag.RenderTarget | BindFlag.ShaderResource),
	Format = Silk.NET.DXGI.Format.FormatB8G8R8A8Unorm,
	Width = width,
	Height = height,
	MipLevels = 1,
	SampleDesc = new SampleDesc(1, 0),
	Usage = Usage.Default,
	MiscFlags = (uint)(ResourceMiscFlag.SharedNthandle | ResourceMiscFlag.SharedKeyedmutex),								 
	CPUAccessFlags = 0, //(uint) D3D11.CpuAccessFlag.None,
	ArraySize = 1
};

var hr = d3d11device.CreateTexture2D(texture2DDesc, (SubresourceData*)IntPtr.Zero, ref renderTargetTexture);
SilkMarshal.ThrowHResult(hr);

// get DXGI resource
IDXGIResource1* pDXGIResource;
var dxgiResourceGuid = IDXGIResource1.Guid;
renderTargetTexture.QueryInterface(ref dxgiResourceGuid, (void**)&pDXGIResource);
ComPtr<IDXGIResource1> DXGIResource = new ComPtr<IDXGIResource1>(pDXGIResource);


// DXGI create shared handle
void* sharedNtHandle = null;
hr = pDXGIResource->CreateSharedHandle(null, DXGI.SharedResourceRead | DXGI.SharedResourceWrite, (char*)null, ref sharedNtHandle);
SilkMarshal.ThrowHResult(hr);



// create D3D9 texture
void* d3d9shared = sharedNtHandle;
ThrowHResult(d3d9device.CreateTexture
(
	width,
	height,
	1u,
	D3D9.UsageRendertarget,
	Silk.NET.Direct3D9.Format.A8R8G8B8,
	Pool.Default,
	ref backbufferTexture,
	ref d3d9shared
)); // Fails here

d3d9surface = new ComPtr<IDirect3DSurface9>();
backbufferTexture.GetSurfaceLevel(0, ref d3d9surface);

renderTargetSharedHandle = (nint)sharedNtHandle;

Maybe it is not possible to open this texture on a D3D9 device. Or I am using a wrong flag or a flag is missing.

The last resort would be to not use a shared texture but instead just copy the rendered image into the WPF app. I think SharpEngine does this too. When I run the WPF sample with my integrated Intel GPU, it reports "WriteableBitmap" instead of "SharedTexture" as the used PresentationType.

[malstraem]

@sven-axm, I am preparing changes with processing correct paths for VkExternalMemoryHandleTypeFlagBits, will be pushed soon. I think WPF case with VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_BIT only need additional call to copy resource.

[jack-day]

Hey, apologies if I'm reviving a bit of an old issue. I've been looking into Vulkan interop with DirectX recently and this repo is an amazing resource. But, I was running into the same problem as in this issue, my latptop's integrated graphics doesn't work with VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_KMT_BIT, so I did some looking into things myself. Here's what I found.

Firstly, there's some important info about the handle types that can be found in the VkExternalMemoryHandleTypeFlagBits docs. It explains that VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_BIT is for NT handles returned by CreateSharedHandle() and VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_KMT_BIT is for global share handles (KMT handles) returned by GetSharedHandle().

Knowing that, regarding this issue as a whole, @sven-axm is right in thinking that the KMT handle type is old, getting increasingly lesser support, which is why it won't work on either of our integrated GPUs. They're also right in thinking NT handles aren't supported with D3D9. This means that for the WPF app, unfortunately, it's simply not going to work if the GPU doesn't support KMT handles, at least with the D3DImage. I've made PR #3 to add a check in the Vulkan interop to ensure the device supports KMT handles.

In regards to swapping to NT handles, whilst we can't do it in WPF, we can do it in WinUI. Microsoft recommends using NT handles over KMT handles for sharing resources, and they also recommend using a keyed mutex with shared NT handles, there's info in the misc flag docs.

Additionally, @malstraem mentions the following about dedicated allocation:

Docs says that ExternalMemoryFeatures of VkExternalMemoryProperties should be checked for VK_EXTERNAL_MEMORY_FEATURE_DEDICATED_ONLY_BIT to procces dedicated allocation logic, but I see Nvidia RTX3060/1060 and Intel HD 630 works without. Docs also says (does this mean that dedicated allocations can be ignored on all devices?)

I agree that the documentation is confusing, and seems to indicate that these handle types are a special case, but the
wording makes it difficult to interpret whether we must use dedicated allocation. Nonetheless, I think I've found the reason why the VK_EXTERNAL_MEMORY_FEATURE_DEDICATED_ONLY_BIT flag is given, but everything still works without dedicated allocation. Looking at the Valid Usage cases in the VkMemoryDedicatedAllocateInfo docs,
we can find:

If image is not VK_NULL_HANDLE and VkMemoryAllocateInfo defines a memory import
operation with handle type VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_BIT,
VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_KMT_BIT,
VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_BIT,
VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_KMT_BIT,
VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D12_HEAP_BIT,
or VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D12_RESOURCE_BIT,
and the external handle was created by the Vulkan API, then the memory being
imported must also be a dedicated image allocation and image must be identical
to the image associated with the imported memory.

Those docs specify that, for those handle types, only in the case of the external handle being created by the Vulkan API, does the memory allocation need to be dedicated. So for example, if vkGetMemoryWin32HandleKHR() is used to create an external handle in the Vulkan API, memory imported using this handle must use a dedicated memory allocation. Conversely, if we add VkImportMemoryWin32HandleInfoKHR to VkMemoryAllocateInfo to import an external Windows handle, memory imported using this handle does not need to use a dedicated memory allocation if the handle type matches those we listed ealier from the VkMemoryDedicatedAllocateInfo docs. Otherwise, it must use dedicated memory allocation. That's the only reasonable explanation I could find at least, could just be that it's not a fatal error or it's undefined behaviour to use a handle type that gives VK_EXTERNAL_MEMORY_FEATURE_DEDICATED_ONLY_BIT without dedicated allocation. Nonetheless, dedicated allocation wouldn't hurt.

@malstraem also touches on adding a check for the LUID. I also found that the VkExternalMemoryHandleTypeFlagBits docs say:

Some external memory handle types can only be shared within the same underlying physical device and/or the same driver version, as defined in the following table:

So an LUID check would be good to check the devices are the same across the interop.

For all of those changes mentioned, I've made PR #4.

Let me know if I've made a mistake or missed anything. I'm hoping this should be enough to tie things up for this issue.

[malstraem]

@jack-day, your fixes have been merged, much thanks. Since it looks good, I think it would be more valuable if the example worked for both handle types. I'll try to free up some time to make two switchable branches.

I apologize for the long silence, these months have been busy 🤓

However, I still want to figure out the path for WPF and the NT handle or state that it is impossible or the cost is comparable to copy to CPU e.g. WriteableBitmap.

I am stuck on the fact that although the import of the NT handle on the Vulkan side is successful, and the D3D9 device successfully shares the texture with D3D11 (all three have the same LUID), I do not see the image after resource copy call.

[jack-day]

@malstraem got a code snippet or a branch with your changes I could take a look at?