Refactor program-data separation example

Author: lucylq

program-data-separation/README.md

@@ -1,6 +1,10 @@
 # Program Data Separation Examples
 
-This directory provides an example of the Program Data Separation APIs in ExecuTorch.
+This directory provides an example of the Program Data Separation APIs in ExecuTorch. Specifically, it showcases:
+1. Simple program data separation examples using the portable operators and XNNPACK.
+2. LoRA inference example with a LoRA and non-LoRA model sharing foundation weights.
+
+## Program Data Separation
 
 The program-data separation APIs allow users to generate a separate data file when exporting and lowering a model. i.e., generate a PTE file containing the model execution program, and one (or more) [PTD](https://github.com/pytorch/executorch/blob/main/extension/flat_tensor/README.md) file/s containing only weights.
 
@@ -9,13 +13,6 @@ PTD files are used to store data outside of the PTE file. Some use-cases:
 - Deduplication: sharing model weights between multiple executable PTE files. This can significantly reduce binary file size and runtime memory usage.
 - Flexible deployment: allow async updates between program and data, especially if they are updated with different cadences.
 
-## LoRA
-A major use-case that program-data separation enables is inference with multiple LoRA adapters. LoRA is a fine-tuning technique introduced in [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685). LoRA fine-tuning produces lightweight 'adapter' weights that can be applied to an existing model to adapt it to a new task. LoRA adapters are typically small in comparison to LLM foundation weights. They are generally on the order of KB,MB, depending on the finetuning setup and model size.
-
-With program-data separation, users can generate a PTE file containing the program and LoRA weights, and save the original foundation weights to a separate PTD file. Provided they are based on the same underlying model, multiple LoRA-adapted PTE files can share the same foundation weights. This means adding a model adapted to a new task incurs minimal binary size and runtime memory overhead; the cost of the lora adapter weights.
-
-An example of this usage is coming soon.
-
 ## Virtual environment setup
 Create and activate a Python virtual environment:
 ```bash
@@ -27,23 +24,20 @@ conda create -yn executorch-ptd python=3.10.0 && conda activate executorch-ptd
 ```
 
 Install dependencies:
-
-[Please install ExecuTorch pip package from source](https://docs.pytorch.org/executorch/stable/using-executorch-building-from-source.html#install-executorch-pip-package-from-source), until executorch==0.7.0 is released.
-
 ```
 pip install executorch==0.7.0
 ```
 
 ## Export a model with program-data separation
 To export a non-delegated linear model, into the current directory:
 ```python
-python export.py --outdir .
+python export_linear.py --outdir .
 ```
 Expect the files 'linear.pte' and 'linear.ptd'.
 
 To export a linear model delegated to XNNPACK, into the current directory:
 ```python
-python export.py --outdir . --xnnpack
+python export_linear.py --outdir . --xnnpack
 ```
 Expect the files 'linear_xnnpack.pte' and 'linear_xnnpack.ptd'.
 
@@ -53,38 +47,16 @@ Note:
 
 For more information on the PTD data format, please see the [flat_tensor](https://github.com/pytorch/executorch/blob/main/extension/flat_tensor/README.md) directory.
 
-## Runtime (cpp)
-The cpp/ directory contains the executorch submodule along with a main.cpp file that demonstrates how to load the PTE and PTD files and execute the program.
-
-First, export your PTE and PTD files using the instructions above.
-
-**Build instructions**
-
-Change to the cpp directory.
-```
-cd cpp
-```
-
-Create build directory if it doesn't exist.
-```
-mkdir -p build
-cd build
-```
+Please see [program-data-separation/cpp](cpp/) for instructions on running the exported models.
 
-Configure CMake.
-```
-cmake -DCMAKE_BUILD_TYPE=Release ..
-```
+## Export a model with LoRA
+A major use-case that program-data separation enables is inference with multiple LoRA adapters. LoRA is a fine-tuning technique introduced in [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685). LoRA fine-tuning produces lightweight 'adapter' weights that can be applied to an existing model to adapt it to a new task. LoRA adapters are typically small in comparison to LLM foundation weights, on the order of KB-MB depending on the finetuning setup and model size.
 
-Build the project.
-```
-cmake --build . -j$(nproc)
-echo "Build complete! Executable located at: ./bin/executorch_program_data_separation"
-```
+To enable LoRA, we generate:
+- PTE file/s: containing program and LoRA adapter weights.
+- PTD file: containing foundation weights.
 
-Run the executable.
-```
-./bin/executorch_program_data_separation --model-path ../../linear.pte --data-path ../../linear.ptd
+Multiple LoRA-adapted PTE files can share the same foundation weights and adding a model adapted to a new task incurs minimal binary size and runtime memory overhead.
 
-./bin/executorch_program_data_separation --model-path ../../linear_xnnpack.pte --data-path ../../linear_xnnpack.ptd
-```
+### Requirements
+LoRA is currently supported on executorch main. [Please install ExecuTorch pip package from source](https://docs.pytorch.org/executorch/stable/using-executorch-building-from-source.html#install-executorch-pip-package-from-source), until executorch==1.0 is released.

program-data-separation/cpp/README.md

@@ -0,0 +1,37 @@
+# ExecuTorch Program Data Separation Demo C++.
+
+This directory contains the C++ code to run the examples generated in [program-data-separation](../program-data-separation/README.md).
+
+## Build instructions
+0. Export the model/s. See [program-data-separation](../program-data-separation/README.md) for instructions.
+1. The ExecuTorch repository is configured as a git submodule at `~/executorch-examples/program-data-separation/cpp/executorch`.  To initialize it:
+   ```bash
+    cd ~/executorch-examples/
+    git submodule sync
+    git submodule update --init --recursive
+   ```
+2. Install dev requirements for ExecuTorch
+
+    ```bash
+    cd ~/executorch-examples/mv2/cpp/executorch
+    pip install -r requirements-dev.txt
+    ```
+
+## Program-data separation demo
+**Build instructions**
+
+Build the executable:
+```bash
+cd ~/executorch-examples/program-data-separation/cpp
+chmod +x build_example.sh
+./build_example.sh
+```
+
+Run the executable.
+```
+./bin/executorch_program_data_separation --model-path ../../linear.pte --data-path ../../linear.ptd
+
+./bin/executorch_program_data_separation --model-path ../../linear_xnnpack.pte --data-path ../../linear_xnnpack.ptd
+```
+
+## LoRA demo

program-data-separation/cpp/build_example.sh

@@ -0,0 +1,14 @@
+#!/bin/bash
+set -e
+
+# Create build directory if it doesn't exist
+mkdir -p build
+cd build
+
+# Configure CMake
+cmake -DCMAKE_BUILD_TYPE=Release ..
+
+# Build the project
+cmake --build . -j$(nproc)
+
+echo "Build complete! Executable located at: ./bin/executorch_program_data_separation"