refactor

Author: lucylq

program-data-separation/README.md

@@ -13,41 +13,10 @@ PTD files are used to store data outside of the PTE file. Some use-cases:
 - Deduplication: sharing model weights between multiple executable PTE files. This can significantly reduce binary file size and runtime memory usage.
 - Flexible deployment: allow async updates between program and data, especially if they are updated with different cadences.
 
-## Virtual environment setup
-Create and activate a Python virtual environment:
-```bash
-python3 -m venv .venv && source .venv/bin/activate && pip install --upgrade pip
-```
-Or alternatively, [install conda on your machine](https://conda.io/projects/conda/en/latest/user-guide/install/index.html)
-```bash
-conda create -yn executorch-ptd python=3.10.0 && conda activate executorch-ptd
-```
-
-Install dependencies:
-```
-pip install executorch==0.7.0
-```
-
-## Export a model with program-data separation
-To export a non-delegated linear model, into the current directory:
-```python
-python export_linear.py --outdir .
-```
-Expect the files 'linear.pte' and 'linear.ptd'.
-
-To export a linear model delegated to XNNPACK, into the current directory:
-```python
-python export_linear.py --outdir . --xnnpack
-```
-Expect the files 'linear_xnnpack.pte' and 'linear_xnnpack.ptd'.
-
-Note:
-- PTE: contains the program execution logic.
-- PTD: contains the constant tensors used by the PTE.
-
 For more information on the PTD data format, please see the [flat_tensor](https://github.com/pytorch/executorch/blob/main/extension/flat_tensor/README.md) directory.
 
-Please see [program-data-separation/cpp](cpp/) for instructions on running the exported models.
+## Export a model with program-data separation
+For a demo of the program-data separation APIs using a linear model, please see [program-data-separation/cpp/linear_example](linear_example/). This example generates and runs a program-data separated linear model, with weights and bias in a separate .ptd file.
 
 ## Export a model with LoRA
 A major use-case that program-data separation enables is inference with multiple LoRA adapters. LoRA is a fine-tuning technique introduced in [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685). LoRA fine-tuning produces lightweight 'adapter' weights that can be applied to an existing model to adapt it to a new task. LoRA adapters are typically small in comparison to LLM foundation weights, on the order of KB-MB depending on the finetuning setup and model size.

program-data-separation/cpp/CMakeLists.txt

@@ -17,7 +17,7 @@ option(EXECUTORCH_BUILD_XNNPACK "" ON)
 # Add ExecuTorch subdirectory
 add_subdirectory("executorch")
 
-set(DEMO_SOURCES main.cpp)
+set(DEMO_SOURCES linear_example/main.cpp)
 
 # Create executable
 add_executable(executorch_program_data_separation ${DEMO_SOURCES})

program-data-separation/cpp/README.md

@@ -0,0 +1,74 @@
+# ExecuTorch Program Data Separation Demo C++.
+
+This directory contains the C++ code to run the examples generated in [program-data-separation](../program-data-separation/README.md).
+
+
+## Virtual environment setup.
+Create and activate a Python virtual environment:
+```bash
+python3 -m venv .venv && source .venv/bin/activate && pip install --upgrade pip
+```
+Or alternatively, [install conda on your machine](https://conda.io/projects/conda/en/latest/user-guide/install/index.html)
+```bash
+conda create -yn executorch-ptd python=3.10.0 && conda activate executorch-ptd
+```
+
+Install dependencies:
+```bash
+pip install executorch==0.7.0
+```
+
+## Export the model/s.
+
+Change into the program-data-separation directory and create a directory to hold exported artifacts.
+```bash
+cd ~/executorch-examples/program-data-separation
+mkdir models
+```
+
+Export models into the `models` directory. The first command will generated undelegated model/data files, and the second will generate XNNPACK-delegated model/data files.
+```bash
+python export_linear.py --outdir models/
+python export_linear.py --outdir models/ --xnnpack
+```
+Expect the files `linear.pte` and `linear.ptd`, `linear_xnnpack.pte` and `linear_xnnpack.ptd`.
+
+Note:
+- PTE: contains the program execution logic.
+- PTD: contains the constant tensors used by the PTE.
+
+See [program-data-separation](../../program-data-separation/README.md) for instructions.
+
+## Install runtime dependencies.
+The ExecuTorch repository is configured as a git submodule at `~/executorch-examples/program-data-separation/cpp/executorch`.  To initialize it:
+```bash
+cd ~/executorch-examples/
+git submodule sync
+git submodule update --init --recursive
+```
+Install dev requirements for ExecuTorch
+
+```bash
+cd ~/executorch-examples/program-data-separation/cpp/executorch
+pip install -r requirements-dev.txt
+```
+
+## Build the runtime.
+Build the executable:
+```bash
+cd ~/executorch-examples/program-data-separation/cpp/linear_example
+chmod +x build_example.sh
+./build_example.sh
+```
+
+## Run the executable.
+```
+./build/bin/executorch_program_data_separation --model-path ../../models/linear.pte --data-path ../../models/linear.ptd
+
+./build/bin/executorch_program_data_separation --model-path ../../models/linear_xnnpack.pte --data-path ../../models/linear_xnnpack.ptd
+```
+
+## Clean up.
+rm -rf build
+cd ~/executorch-examples/program-data-separation
+rm -rf models

program-data-separation/cpp/build_example.sh

@@ -0,0 +1,15 @@
+#!/bin/bash
+set -e
+
+# Clean and create build directory if it doesn't exist
+rm -rf build
+mkdir -p build
+cd build
+
+# Configure CMake
+cmake -DCMAKE_BUILD_TYPE=Release ../..
+
+# Build the project
+cmake --build . -j$(nproc)
+
+echo "Build complete! Executable located at: ./build/bin/executorch_program_data_separation"