Skip to content

Commit d555266

Browse files
Update learning path for Model Training Gym
1 parent f06c7e5 commit d555266

File tree

5 files changed

+27
-12
lines changed

5 files changed

+27
-12
lines changed

content/learning-paths/mobile-graphics-and-gaming/model-training-gym/1-introduction.md

Lines changed: 17 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -8,29 +8,37 @@ layout: learningpathall
88

99
## What is Neural Graphics?
1010

11-
Neural graphics is an intersection of graphics and machine learning. Rather than relying purely on traditional GPU pipelines, neural graphics integrates learned models directly into the rendering stack. The techniques are particularly powerful on mobile devices, where battery life and performance constraints limit traditional compute-heavy rendering approaches. The goal is to deliver high visual fidelity without increasing GPU cost. This is achieved by training and deploying compact neural networks optimized for the device's hardware.
11+
Neural graphics is an intersection of graphics and machine learning. Rather than relying purely on traditional GPU pipelines, neural graphics integrates learned models directly into the rendering stack. These techniques are particularly powerful on mobile devices, where battery life and performance constraints limit traditional compute-heavy rendering approaches. Your goal is to deliver high visual fidelity without increasing GPU cost. You achieve this by training and deploying compact neural networks optimized for your device's hardware.
1212

1313
## How does Arm support neural graphics?
1414

15-
Arm enables neural graphics through the [**Neural Graphics Development Kit**](https://developer.arm.com/mobile-graphics-and-gaming/neural-graphics): a set of open-source tools that let developers train, evaluate, and deploy ML models for graphics workloads.
15+
16+
Arm enables neural graphics through the [**Neural Graphics Development Kit**](https://developer.arm.com/mobile-graphics-and-gaming/neural-graphics): a set of open-source tools that let you train, evaluate, and deploy ML models for graphics workloads.
17+
1618

1719
At its core are the ML Extensions for Vulkan, which bring native ML inference into the GPU pipeline using structured compute graphs. These extensions (`VK_ARM_tensors` and `VK_ARM_data_graph`) allow real-time upscaling and similar effects to run efficiently alongside rendering tasks.
1820

1921

20-
You can develop neural graphics models using well-known ML frameworks like PyTorch, then export them for deployment with Arm's hardware-aware pipeline. The workflow converts your model to `.vgf` via the TOSA intermediate representation, making it possible to tailor model development for your game use case. In this Learning Path, you will focus on **Neural Super Sampling (NSS)** as the primary example for training, evaluating, and deploying neural models using the [**Neural Graphics Model Gym**](https://github.com/arm/neural-graphics-model-gym). To learn more about NSS, see the [resources on Hugging Face](https://huggingface.co/Arm/neural-super-sampling). Arm has also developed a set of Vulkan Samples to help you get started. The `.vgf` format is introduced in the `postprocessing_with_vgf` sample. For a broader overview of neural graphics developer resources including the Vulkan Samples, visit the [introductory Learning Path](/learning-paths/mobile-graphics-and-gaming/vulkan-ml-sample/).
22+
23+
You can develop neural graphics models using well-known ML frameworks like PyTorch, then export them for deployment with Arm's hardware-aware pipeline. The workflow converts your model to `.vgf` via the TOSA intermediate representation, making it possible to tailor model development for your game use case. In this Learning Path, you will focus on **Neural Super Sampling (NSS)** as the primary example for training, evaluating, and deploying neural models using the [**Neural Graphics Model Gym**](https://github.com/arm/neural-graphics-model-gym). To learn more about NSS, see the [resources on Hugging Face](https://huggingface.co/Arm/neural-super-sampling). Arm has also developed a set of Vulkan Samples to help you get started. The `.vgf` format is introduced in the `postprocessing_with_vgf` sample. For a broader overview of neural graphics developer resources, including the Vulkan Samples, visit the [introductory Learning Path](/learning-paths/mobile-graphics-and-gaming/vulkan-ml-sample/).
24+
2125

2226

2327
Starting in 2026, Arm GPUs will feature dedicated neural accelerators, optimized for low-latency inference in graphics workloads. To help you get started early, Arm provides the ML Emulation Layers for Vulkan that simulate future hardware behavior, so you can build and test models now.
2428

2529
## What is the Neural Graphics Model Gym?
2630

31+
2732
The Neural Graphics Model Gym is an open-source toolkit for fine-tuning and exporting neural graphics models. It is designed to streamline the entire model lifecycle for graphics-focused use cases, like NSS.
2833

29-
Model Gym gives you:
34+
With Model Gym, you can:
35+
36+
- Train and evaluate models using a PyTorch-based API
37+
- Export models to `.vgf` using ExecuTorch for real-time use in game development
38+
- Take advantage of quantization-aware training (QAT) and post-training quantization (PTQ) with ExecuTorch
39+
- Use an optional Docker setup for reproducibility
40+
41+
You can choose to work with Python notebooks for rapid experimentation or use the command-line interface for automation. This Learning Path will walk you through the demonstrative notebooks and prepare you to start using the CLI for your own model development.
3042

31-
- A training and evaluation API built on PyTorch
32-
- Model export to .vgf using ExecuTorch for real-time use in game development
33-
- Support for quantization-aware training (QAT) and post-training quantization (PTQ) using ExecuTorch
34-
- Optional Docker setup for reproducibility
3543

36-
The toolkit supports workflows via both Python notebooks (for rapid experimentation) and command-line interface. This Learning Path will walk you through the demonstrative notebooks, and prepare you to start using the CLI for your own model development.
44+
You have completed the introduction! You're ready to set up your environment and start working with neural graphics models. Keep going!

content/learning-paths/mobile-graphics-and-gaming/model-training-gym/2-devenv.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,4 +55,5 @@ print("Torch version:", torch.__version__)
5555
print("Model Gym version:", ng_model_gym.__version__)
5656
```
5757

58-
You’re now ready to start walking through the training and evaluation steps.
58+
You’ve completed your environment setup - great work! You’re now ready to start walking through the training and evaluation steps.
59+

content/learning-paths/mobile-graphics-and-gaming/model-training-gym/3-model-training.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,8 @@ neural-graphics-model-gym-examples/tutorials/nss/model_evaluation_example.ipynb
6262

6363
At the end you should see a visual comparison of the NSS upscaling and the ground truth image.
6464

65-
Proceed to the final section to view the model structure and explore further resources.
65+
66+
You’ve completed the training and evaluation steps. Proceed to the final section to view the model structure and explore further resources.
67+
6668

6769

content/learning-paths/mobile-graphics-and-gaming/model-training-gym/4-model-explorer.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,4 +46,4 @@ Use the file browser to open the `.vgf` model exported earlier in your training
4646

4747
Through this Learning Path, you’ve learned what neural graphics is and why it matters for game performance. You’ve stepped through the process of training and evaluating an NSS model using PyTorch and the Model Gym, and seen how to export that model into VGF (.vgf) for real-time deployment. You’ve also explored how to visualize and inspect the model’s structure using Model Explorer.
4848

49-
As a next step, you can head over to the [Model Training Gym repository](https://github.com/arm/neural-graphics-model-gym/tree/main) documentation to explore integration into your own game development workflow. You’ll find resources on fine-tuning, deeper details about the training and export process, and everything you need to adapt to your own content and workflows.
49+
You’ve completed the Learning Path. You now know how to train, export, and visualize neural graphics models on Arm. Explore the Model Training Gym repository for deeper integration and to keep building your skills.

content/learning-paths/mobile-graphics-and-gaming/model-training-gym/_index.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,10 @@ further_reading:
5050
title: NSS on HuggingFace
5151
link: https://huggingface.co/Arm/neural-super-sampling
5252
type: website
53+
- resource:
54+
title: Vulkan ML Sample Learning Path
55+
link: /learning-paths/mobile-graphics-and-gaming/vulkan-ml-sample/
56+
type: learningpath
5357

5458

5559
### FIXED, DO NOT MODIFY

0 commit comments

Comments
 (0)