Welcome to the Neural Graphics SDK for Game Engines

Neural Graphics SDK for Game Engines is Arm’s unified graphics software development kit designed to support multiple rendering use cases across diverse engines and platforms. It is derived from AMD's FFX SDK 1.1.3(https://github.com/GPUOpen-LibrariesAndSDKs/FidelityFX-SDK). It consolidates technologies like NSS (Neural Super Sampling), ASR(Accuracy Super Resolution), and NFRU (Neural Frame Rate Upsampling) into a modular, engine-agnostic framework.

Currently, Neural Graphics SDK for Game Engines supports NSS.

'*' means not supported yet.

*ASR: Not supported yet. But it will be a fallback for NSS, for some devices cannot support ML extensions for Vulkan.

api_layer: we are compliant with AMD FidelityFX API 1.1.3.

components: Currently we support NSS. All components are implemented independently in this layer.

backend: All components share one backend. It will access deeper layers via Vulkan APIs.

Vulkan Emulation Layer: Emulation layers for supporting the ML extensions and Vulkan headers before the real device is ready.(https://github.com/arm/ai-ml-sdk-for-vulkan)

System requirement

Cmake: minimun version 3.21, maxmum version 3.31

Vulkan SDK: recommend version 1.4.321.0. You can download from Lunarg(https://www.lunarg.com/vulkan-sdk/).

Vulkan Emulation Layer

After the SDK has been integrated, you are ready to run Neural Graphics technologies, like NSS in your project. But before real devices are ready for ML extensions for Vulkan, you need to enable Vulkan emulation layers.

You can use "Vulkan configurator" in your vulkan sdk:

Or add Vulkan emulation layer libs' path to your system environment. For example: Windows:

set "VK_ADD_LAYER_PATH=path\to\VulkanML"
set "VK_INSTANCE_LAYERS=VK_LAYER_ML_Graph_Emulation;VK_LAYER_ML_Tensor_Emulation"

Linux:

export VK_ADD_LAYER_PATH="path/to/VulkanMLLib"
export LD_LIBRARY_PATH="path/to/VulkanMLLib"
export VK_INSTANCE_LAYERS="VK_LAYER_ML_Graph_Emulation:VK_LAYER_ML_Tensor_Emulation"

Whatever method you use, make sure VK_LAYER_ML_Graph_Emulation is enabled before VK_LAYER_ML_Tensor_Emulation.

Build SDK

Follow these steps to build the SDK:

1. Ensure you have python installed.
2. Run the build.py cross-platform script to build the SDK.
3. Run the script with -h to view detailed usage information.

If the SDK builds successfully, the libraries for the SDK are generated in the ./bin folder.

Enable extensions

When creating vulkan physical devices, you must enable the necessary extensions:

• VulkanExtensions::kVK_ARM_tensors
• VulkanExtensions::kVK_ARM_data_graph.

Sample code:

VkDeviceCreateInfo ci = {};
ci.sType = VK_STRUCTURE_TYPE_DEVICE_CREATE_INFO;
... //Other create info settings
 
VkPhysicalDeviceTensorFeaturesARM tensorFeature = {};
{
    tensorFeature.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_TENSOR_FEATURES_ARM;
 
    VkPhysicalDeviceFeatures2 features2 = {};
    features2.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2;
    features2.pNext = &tensorFeature;
    vkGetPhysicalDeviceFeatures2(m_physicalDevice, &features2);
 
    tensorFeature.pNext = const_cast<void*>(ci.pNext);
    ci.pNext = &tensorFeature;
}
 
VkPhysicalDeviceDataGraphFeaturesARM dataGraphFeature = {};
{
    dataGraphFeature.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_DATA_GRAPH_FEATURES_ARM;
 
    VkPhysicalDeviceFeatures2 features2 = {};
    features2.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2;
    features2.pNext = &dataGraphFeature;
    vkGetPhysicalDeviceFeatures2(m_physicalDevice, &features2);
 
    dataGraphFeature.pNext = const_cast<void*>(ci.pNext);
    ci.pNext = &tensorFeature; 
}
 
vkCreateDevice(m_physicalDevice, &ci, nullptr, &m_vkdevice)

Limitations

The SDK only supports vulkan backend.

The SDK does not implement automatic GPU synchronization - it relies on the application to properly synchronize GPU work before context destruction.

Trademarks and Copyrights

AMD is a trademark of Advanced Micro Devices, Inc.

AMD FidelityFX™ is a trademark of Advanced Micro Devices, Inc.

Arm® is a registered trademark of Arm Limited (or its subsidiaries) in the US and/or elsewhere.

Vulkan is a registered trademark and the Vulkan SC logo is a trademark of the Khronos Group Inc.

Visual Studio, Windows are registered trademarks or trademarks of Microsoft Corporation in the US and other jurisdictions.