Merge pull request #1964 from ArmDeveloperEcosystem/main

Prod update for stable audio

Author: pareenaverma

content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/1-prerequisites.md

@@ -19,7 +19,7 @@ Your first task is to prepare a development environment with the required softwa
 
 ### Create workspace directory
 
-Create a separate directory for all dependencies and repositories that this Learning Path uses.
+Create a separate directory for all the dependencies and repositories that this Learning Path uses.
 
 Export the `WORKSPACE` variable to point to this directory, which you will use in the following steps:
 
@@ -74,7 +74,7 @@ See the [CMake install guide](/install-guides/cmake/) for troubleshooting instru
 
 ### Install Bazel
 
-Bazel is an open-source build tool which we will use to build LiteRT libraries.
+Bazel is an open-source build tool which you will use to build LiteRT libraries.
 
 {{< tabpane code=true >}}
   {{< tab header="Linux">}}
@@ -98,22 +98,24 @@ wget https://dl.google.com/android/repository/android-ndk-r25b-linux.zip
 unzip android-ndk-r25b-linux.zip
   {{< /tab >}}
   {{< tab header="MacOS">}}
-brew install --cask android-studio temurin
+wget https://dl.google.com/android/repository/android-ndk-r25b-darwin.zip
+unzip android-ndk-r25b-darwin
+mv android-ndk-r25b-darwin ~/Library/Android/android-ndk-r25b
   {{< /tab >}}
 {{< /tabpane >}}
 
-For easier access and execution of Android NDK tools, add these to the `PATH` and set the `ANDROID_NDK` variable:
+For easier access and execution of Android NDK tools, add these to the `PATH` and set the `NDK_PATH` variable:
 
 {{< tabpane code=true >}}
   {{< tab header="Linux">}}
-export ANDROID_NDK=$WORKSPACE/android-ndk-r25b/
-export PATH=$ANDROID_NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/:$PATH
+export NDK_PATH=$WORKSPACE/android-ndk-r25b/
+export PATH=$NDK_PATH/toolchains/llvm/prebuilt/linux-x86_64/bin/:$PATH
   {{< /tab >}}
   {{< tab header="MacOS">}}
-export ANDROID_NDK=~/Library/Android/sdk/ndk/27.0.12077973/
-export PATH=$PATH:$ANDROID_NDK/toolchains/llvm/prebuilt/darwin-x86_64/bin
+export NDK_PATH=~/Library/Android/android-ndk-r25b
+export PATH=$PATH:$NDK_PATH/toolchains/llvm/prebuilt/darwin-x86_64/bin
 export PATH=$PATH:~/Library/Android/sdk/cmdline-tools/latest/bin
   {{< /tab >}}
 {{< /tabpane >}}
 
-Now that your development environment is ready and all pre-requisites installed, you can test the Audio Stable Open model.
+Now that your development environment is ready and all the prerequisites are installed, you can move on to test the Stable Audio Open Small model.

content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/2-testing-model.md

@@ -8,12 +8,12 @@ layout: learningpathall
 
 ## Download the model
 
-Stable Audio Open is an open-source model optimized for generating short audio samples, sound effects, and production elements using text prompts.
+Stable Audio Open Small is an open-source model optimized for generating short audio samples, sound effects, and production elements using text prompts.
 
 [Log in](https://huggingface.co/login) to HuggingFace and navigate to the model landing page:
 
 ```bash
-https://huggingface.co/stabilityai/stable-audio-open-small
+https://huggingface.co/stabilityai/stable-audio-open-small/tree/main
 ```
 
 You may need to fill out a form with your contact information to use the model:
@@ -26,15 +26,11 @@ Download and copy the configuration file `model_config.json` and the model itsel
 ls $WORKSPACE/model_config.json $WORKSPACE/model.ckpt
 ```
 
-## Test the model
+You can learn more about this model [here](https://huggingface.co/stabilityai/stable-audio-open-small).
 
-To test the model, use the Stable Audio demo site, which lets you experiment directly through a web-based interface:
+### Good prompting practices
 
-```bash
-https://stableaudio.com/
-```
-
-Use the UI to enter a prompt. A good prompt can include:
+A good prompt for the Stable Audio Open Small model can include the following elements:
 
 * Music genre and subgenre.
 * Musical elements (texture, rhythm and articulation).
@@ -45,5 +41,5 @@ The order of prompt parameters matters. For more information, see the [Prompt st
 
 You can explore training and inference code for audio generation models in the [Stable Audio Tools repository](https://github.com/Stability-AI/stable-audio-tools).
 
-Now that you've downloaded and tested the model, continue to the next section to convert the model to LiteRT.
+Now that you've downloaded the model, you're ready to convert it to LiteRT format in the next step.
 

content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/3-converting-model.md

@@ -1,31 +1,37 @@
 ---
-title: Convert Open Stable Audio Small model to LiteRT
+title: Convert Stable Audio Open Small model to LiteRT
 weight: 4
 
 ### FIXED, DO NOT MODIFY
 layout: learningpathall
 ---
+In this section, you will learn about the audio generation model. You will then clone a repository that contains the scripts required to convert the model submodules into LiteRT format and generate the inference application.
 
-## Stable Audio Open Small Model
+## Stable Audio Open Small
+
+The open-source model consists of three main submodules. They are described in the table below, and come together through the pipeline shown in the image.
 
 |Submodule|Description|
 |------|------|
 |Conditioners| Includes a T5-based text encoder for the input prompt and a numerical duration encoder. These components convert the inputs into embeddings passed to the DiT model. |
 |Diffusion Transformer (DiT)| Denoises random noise over multiple steps to produce structured latent audio, guided by conditioner embeddings. |
 |AutoEncoder| Compresses audio waveforms into a latent representation for processing by the DiT model, and decompresses the output back into audio. |
 
-The submodules work together to provide the pipeline as shown below:
+
 ![Model structure#center](./model.png)
 
-As part of this section, you will covert each of the three submodules into [LiteRT](https://ai.google.dev/edge/litert) format, using two separate conversion routes:
-1. Conditioners submodule - ONNX to LiteRT using [onnx2tf](https://github.com/PINTO0309/onnx2tf) tool.
-2. DiT and AutoEncoder submodules - PyTorch to LiteRT using Google AI Edge Torch tool.
+In this section, you will explore two different conversion routes, to convert the submodules to [LiteRT](https://ai.google.dev/edge/litert) format. Both methods will be run using Python wrapper scripts from the examples repository.
+
+1. **ONNX to LiteRT**: using the `onnx2tf` tool. This is the traditional two-step approach (PyTorch -> ONNX -> LiteRT). You will use it to convert the Conditioners submodule.
+
+2. **PyTorch to LiteRT**: using the Google AI Edge Torch tool. You will use this tool to convert the DiT and AutoEncoder submodules.
+
 
-### Create virtual environment and install dependencies
+## Download the sample code
 
 The Conditioners submodule is made of the T5Encoder model. You will use the ONNX to TFLite conversion for this submodule.
 
-To avoid dependency issues, create a virtual environment. In this guide, we will use `virtualenv`:
+To avoid dependency issues, create a virtual environment. For example, you can use the following command:
 
 ```bash
 cd $WORKSPACE
@@ -37,11 +43,11 @@ Clone the examples repository:
 
 ```bash
 cd $WORKSPACE
-git clone https://github.com/ARM-software/ML-examples/tree/main/kleidiai-examples/audiogen
-cd audio-stale-open-litert
+git clone https://github.com/ARM-software/ML-examples.git
+cd ML-examples/kleidiai-examples/audiogen/
 ```
 
-We now install the needed python packages for this, including *onnx2tf* and *ai_edge_litert*
+Install the required Python packages for this, including *onnx2tf* and *ai_edge_litert*
 
 ```bash
 bash install_requirements.sh
@@ -61,20 +67,19 @@ ImportError: cannot import name 'AttrsDescriptor' from 'triton.compiler.compiler
 ($WORKSPACE/env/lib/python3.10/site-packages/triton/compiler/compiler.py)
 ```
 
-Install the following dependency and rerun the script:
+Reinstall the following dependency:
 ```bash
 pip install triton==3.2.0
-bash install_requirements.sh
 ```
 
 {{% /notice %}}
 
 ### Convert Conditioners Submodule
 
-The Conditioners submodule is based on the T5Encoder model. We convert it first to ONNX, then to LiteRT.
+The Conditioners submodule is based on the T5Encoder model. First, convert it to ONNX, then to LiteRT.
 
-For this conversion we include the following steps:
-1. Load the Conditioners submodule from the Stable Audio Open model configuration and checkpoint.
+For this conversion, the following steps are required:
+1. Load the Conditioners submodule from the Stable Audio Open Small model configuration and checkpoint.
 2. Export the Conditioners submodule to ONNX via *torch.onnx.export()*.
 3. Convert the resulting ONNX file to LiteRT using *onnx2tf*.
 
@@ -84,28 +89,29 @@ You can use the provided script to convert the Conditioners submodule:
 python3 ./scripts/export_conditioners.py --model_config "$WORKSPACE/model_config.json" --ckpt_path "$WORKSPACE/model.ckpt"
 ```
 
-After successful conversion, you now have a `conditioners.onnx` model in your current directory.
+After successful conversion, you now have a `tflite_conditioners` directory containing models with different precisions (e.g., float16, float32).
+
+You will be using the float32.tflite model for on-device inference.
 
 ### Convert DiT and AutoEncoder
 
-To convert the DiT and AutoEncoder submodules, use the [Generative API](https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/generative/) provided by the ai-edge-torch tools. This enables you to export a generative PyTorch model directly to tflite using three main steps:
+To convert the DiT and AutoEncoder submodules, use the [Generative API](https://github.com/google-ai-edge/ai-edge-torch/tree/main/ai_edge_torch/generative/) provided by the ai-edge-torch tools. This enables you to export a generative PyTorch model directly to `.tflite` using three main steps:
 
 1. Model re-authoring.
 2. Quantization.
 3. Conversion.
 
-Convert the DiT and AutoEncoder submodules using the provided python script:
+Convert the DiT and AutoEncoder submodules using the provided Python script:
+
 ```bash
-CUDA_VISIBLE_DEVICES="" python3 ./scripts/export_dit_autoencoder.py --model_config "$WORKSPACE/model_config.json" --ckpt_path "$WORKSPACE/model.ckpt"
+python3 ./scripts/export_dit_autoencoder.py --model_config "$WORKSPACE/model_config.json" --ckpt_path "$WORKSPACE/model.ckpt"
 ```
 
-After successful conversion, you now have `dit_model.tflite` and `autoencoder_model.tflite` models in your current directory and can deactivate the virtual environment:
+After successful conversion, you now have `dit_model.tflite` and `autoencoder_model.tflite` models in your current directory.
 
-```bash
-deactivate
-```
+A more detailed explanation of the above scripts is available [here](https://github.com/ARM-software/ML-examples/blob/main/kleidiai-examples/audiogen/scripts/README.md).
 
-For easier access, we add all needed models to one directory:
+For easy access, add all the required models to one directory:
 
 ```bash
 export LITERT_MODELS_PATH=$WORKSPACE/litert-models
@@ -115,7 +121,7 @@ cp dit_model.tflite $LITERT_MODELS_PATH
 cp autoencoder_model.tflite $LITERT_MODELS_PATH
 ```
 
-With all three submodules converted to LiteRT format, you're ready to build LiteRT and run the model on a mobile device in the next step.
+With all three submodules now converted to LiteRT format, you're ready to build the runtime and run Stable Audio Open Small directly on an Android device in the next step.
 
 
 

content/learning-paths/mobile-graphics-and-gaming/run-stable-audio-open-small-with-lite-rt/4-building-litert.md

@@ -8,7 +8,7 @@ layout: learningpathall
 
 ## LiteRT
 
-LiteRT (short for Lite Runtime), formerly known as TensorFlow Lite, is Google's high-performance runtime for on-device AI.
+LiteRT (short for Lite Runtime), formerly known as TensorFlow Lite, is Google's high-performance runtime for on-device AI. Designed for low-latency, resource-efficient execution, LiteRT is optimized for mobile and embedded environments — making it a natural fit for Arm CPUs running models like Stable Audio Open Small. You’ll build the runtime using the Bazel build tool.
 
 ## Build LiteRT libraries
 
@@ -20,55 +20,51 @@ git clone https://github.com/tensorflow/tensorflow.git tensorflow_src
 cd tensorflow_src
 ```
 
-We will use a specific commit of tensorflow for build so you can checkout and set the `TF_SRC_PATH`:
+Check out the specified commit of TensorFlow, and set the `TF_SRC_PATH`:
 ```bash
 git checkout 84dd28bbc29d75e6a6d917eb2998e4e8ea90ec56
 export TF_SRC_PATH=$(pwd)
 ```
 
-We can use `bazel` to build LiteRT libraries, first we use configure script to create a custom configuration for this:
+A script is available to configure the `bazel` build environment. Run it to create a custom TFLite build for Android:
 
-You can now create a custom TFLite build for android:
-
-Ensure the `ANDROID_NDK` variable is set to your previously installed Android NDK:
+{{% notice Reminder %}}
+Ensure the `NDK_PATH` variable is set to your previously installed Android NDK:
 {{< tabpane code=true >}}
   {{< tab header="Linux">}}
-export ANDROID_NDK=$WORKSPACE/android-ndk-r25b/
-export PATH=$ANDROID_NDK/toolchains/llvm/prebuilt/linux-x86_64/bin/:$PATH
+export NDK_PATH=$WORKSPACE/android-ndk-r25b/
+export PATH=$NDK_PATH/toolchains/llvm/prebuilt/linux-x86_64/bin/:$PATH
   {{< /tab >}}
   {{< tab header="MacOS">}}
-export TF_CXX_FLAGS="-DTF_MAJOR_VERSION=0 -DTF_MINOR_VERSION=0 -DTF_PATCH_VERSION=0 -DTF_VERSION_SUFFIX=''"
-export ANDROID_NDK=~/Library/Android/sdk/ndk/27.0.12077973/
-export PATH=$PATH:$ANDROID_NDK/toolchains/llvm/prebuilt/darwin-x86_64/bin
-export PATH=$PATH:~/Library/Android/sdk/cmdline-tools/latest/bin
+export NDK_PATH=~/Library/Android/android-ndk-r25b
+export PATH=$PATH:$NDK_PATH/toolchains/llvm/prebuilt/darwin-x86_64/bin
   {{< /tab >}}
 {{< /tabpane >}}
+{{% /notice  %}}
 
-Set the TensorFlow version
+The configuration script is interactive. Run it using the command below, and use the table to set the parameters for this Learning Path use-case.
 
 ```bash
-export TF_CXX_FLAGS="-DTF_MAJOR_VERSION=0 -DTF_MINOR_VERSION=0 -DTF_PATCH_VERSION=0 -DTF_VERSION_SUFFIX=''"
-```
-
-
-Now you can configure TensorFlow. Here you can set the custom build parameters needed as follows:
-
-```bash { output_lines = "2-14" }
 python3 ./configure.py
-Please specify the location of python. [Default is $WORKSPACE/bin/python3]:
-Please input the desired Python library path to use. Default is [$WORKSPACE/lib/python3.10/site-packages]
-Do you wish to build TensorFlow with ROCm support? [y/N]: n
-Do you wish to build TensorFlow with CUDA support? [y/N]: n
-Do you want to use Clang to build TensorFlow? [Y/n]: n
-Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: y
-Please specify the home path of the Android NDK to use. [Default is /home/user/Android/Sdk/ndk-bundle]: /home/user/Workspace/tools/ndk/android-ndk-r25b
-Please specify the (min) Android NDK API level to use. [Available levels: [16, 17, 18, 19, 21, 22, 23, 24, 26, 27, 28, 29, 30, 31, 32, 33]] [Default is 21]: 30
-Please specify the home path of the Android SDK to use. [Default is /home/user/Android/Sdk]:
-Please specify the Android SDK API level to use. [Available levels: ['31', '33', '34', '35']] [Default is 35]:
-Please specify an Android build tools version to use. [Available versions: ['30.0.3', '34.0.0', '35.0.0']] [Default is 35.0.0]:
 ```
 
-Once the bazel configuration is complete, you can build TFLite as follows:
+|Question|Input|
+|---|---|
+|Please specify the location of python. [Default is $WORKSPACE/bin/python3]:| Enter (default) |
+|Please input the desired Python library path to use[$WORKSPACE/lib/python3.10/site-packages] | Enter |
+|Do you wish to build TensorFlow with ROCm support? [y/N]|N (No)|
+|Do you wish to build TensorFlow with CUDA support?|N|
+|Do you want to use Clang to build TensorFlow? [Y/n]|N|
+|Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]|y (Yes) |
+|Please specify the home path of the Android NDK to use. [Default is /home/user/Android/Sdk/ndk-bundle]| Enter |
+|Please specify the (min) Android NDK API level to use. [Default is 21] | 27 |
+|Please specify the home path of the Android SDK to use. [Default is /home/user/Android/Sdk]| Enter |
+|Please specify the Android SDK API level to use.  [Default is 35]| Enter |
+|Please specify an Android build tools version to use.  [Default is 35.0.0]| Enter |
+|Do you wish to build TensorFlow with iOS support? [y/N]:| n |
+
+Once the Bazel configuration is complete, you can build TFLite as follows:
+
 ```console
 bazel build -c opt --config android_arm64 //tensorflow/lite:libtensorflowlite.so \
     --define tflite_with_xnnpack=true \
@@ -77,7 +73,15 @@ bazel build -c opt --config android_arm64 //tensorflow/lite:libtensorflowlite.so
     --define tflite_with_xnnpack_qu8=true
 ```
 
-This will produce a `libtensorflowlite.so` shared library for android with XNNPack enabled, which we will use to build the example next.
+The final step is to build flatbuffers used by the application:
+```
+cd $WORKSPACE/tensorflow_src
+mkdir flatc-native-build && cd flatc-native-build
+cmake ../tensorflow/lite/tools/cmake/native_tools/flatbuffers
+cmake --build .
+```
+
+Now that LiteRT and FlatBuffers are built, you're ready to compile and deploy the Stable Audio Open Small inference application on your Android device.