[ggma] Add gyu (ggma yielding utility) tool

Implement gyu CLI tool to automate GGMA model package creation:

- Merge prefill.py and decode.py into unified export.py
- Create modular gpm tool structure:
  - gyu/init.py: Setup venv, install deps (CPU-only torch), clone TICO, extract o2o tools
  - gyu/import.py: Download complete model from HuggingFace
  - gyu/export.py: Run conversion pipeline and create .ggma package
  - gyu/common.py: Shared utilities and constants
  - gyu/clean.py: Remove building directory
  - gyu/gyu: Bash wrapper to dispatch commands

Documentation:
- Rename README.md → DEVELOPER.md (technical guide)
- Add USER.md (user-facing guide)

Author: Sanggyu Lee

…me/ggma/examples/generate_text/README.md …ggma/examples/generate_text/DEVELOPER.mdruntime/ggma/examples/generate_text/README.md renamed to runtime/ggma/examples/generate_text/DEVELOPER.md runtime/ggma/examples/generate_text/README.md renamed to runtime/ggma/examples/generate_text/DEVELOPER.md

@@ -1,6 +1,6 @@
-# TinyLlama Text Generation Example
+# TinyLlama Text Generation Developer Guide
 
-This document provides a step‑by‑step guide for generating and processing a TinyLlama text‑generation model.
+This document provides a detailed technical guide for generating, processing, and optimizing the TinyLlama text-generation model. For basic usage, see [USER.md](USER.md).
 
 ## Summary
 
@@ -22,14 +22,6 @@ source _/bin/activate
 pip install -r requirements.txt
 ```
 
-### 3. Install TICO (Torch IR to Circle ONE)
-```bash
-# Clone the repository
-git clone https://github.com/Samsung/TICO.git
-# Install it in editable mode
-pip install -e TICO
-```
-
 ### 4. Get [o2o](https://github.com/Samsung/ONE/pull/16233) in PATH
 *Requires the GitHub CLI (`gh`).*
 ```bash
@@ -41,8 +33,8 @@ export PATH=../../../../tools/o2o:$PATH
 
 ### 1. Create the prefill and decode Circle model files
 ```bash
-python prefill.py   # Generates prefill.circle
-python decode.py    # Generates decode_.circle
+python tinyllama.py --mode prefill   # Generates prefill.circle
+python tinyllama.py --mode decode    # Generates decode_.circle
 ```
 
 Verify the generated files:

runtime/ggma/examples/generate_text/USER.md

@@ -0,0 +1,48 @@
+# TinyLlama Text Generation User Guide
+
+This guide shows how to create a GGMA package for the TinyLlama model using the `gyu` (Generate Your Utility) tool.
+
+## Quick Start
+
+```bash
+cd runtime/ggma/examples/generate_text/
+
+# 1. Initialize environment (one-time setup)
+gyu/gyu init
+
+# 2. Import model from HuggingFace
+gyu/gyu import Maykeye/TinyLLama-v0
+
+# 3. Export to GGMA package
+gyu/gyu export
+
+# 4. Your package is ready at: build/tinyllama-v0.ggma/
+```
+
+## Commands
+
+- **`gyu init`**: Set up environment (venv, dependencies, tools)
+- **`gyu import <model_id>`**: Download model from HuggingFace to `build/`
+- **`gyu export`**: Convert model to Circle format and create `.ggma` package in `build/`
+- **`gyu clean`**: Remove `build/` directory
+
+## Package Structure
+
+The final package at `build/tinyllama-v0.ggma/` contains:
+- `model.circle` - Optimized model for GGMA runtime
+- `tokenizer.json` - Tokenizer configuration
+- `tokenizer.model` - Tokenizer model
+- `config.json` - Model configuration
+
+## Running the Model
+
+```bash
+# Build ggma_run (from ONE root)
+make -j$(nproc)
+make install
+
+# Run the model
+Product/out/bin/ggma_run build/tinyllama-v0.ggma
+```
+
+For detailed developer instructions, see [DEVELOPER.md](DEVELOPER.md).

runtime/ggma/examples/generate_text/decode.py

@@ -0,0 +1,15 @@
+#!/usr/bin/env python3
+import shutil
+import os
+
+def main():
+    build_dir = "build"
+    if os.path.exists(build_dir):
+        print(f"Removing {build_dir} directory...")
+        shutil.rmtree(build_dir)
+        print("Clean complete.")
+    else:
+        print(f"{build_dir} directory does not exist.")
+
+if __name__ == "__main__":
+    main()

runtime/ggma/examples/generate_text/gyu/clean.py

@@ -0,0 +1,11 @@
+import subprocess
+
+# Constants
+VENV_DIR = "venv"
+PR_WORKTREE = "_pr_16233"
+PR_BRANCH = "pr-16233"
+PR_REF = "refs/pull/16233/head"
+
+def run_command(cmd, cwd=None, env=None, check=True):
+    print(f"Running: {cmd}")
+    subprocess.run(cmd, shell=True, cwd=cwd, env=env, check=check)

runtime/ggma/examples/generate_text/gyu/common.py

@@ -0,0 +1,92 @@
+#!/usr/bin/env python3
+import os
+import shutil
+import json
+from common import VENV_DIR, run_command
+
+def main():
+    # Change to build directory
+    build_dir = "build"
+    if not os.path.exists(build_dir):
+        print(f"Error: {build_dir} directory does not exist. Run 'gyu import' first.")
+        return
+    
+    os.chdir(build_dir)
+    
+    # Find model directory by config.json
+    model_dir = None
+    model_id = None
+    for d in os.listdir("."):
+        config_path = os.path.join(d, "config.json")
+        if os.path.isdir(d) and os.path.exists(config_path):
+            model_dir = d
+            # Read model ID from config.json
+            with open(config_path, "r") as f:
+                config = json.load(f)
+                model_id = config.get("_name_or_path", d)
+            print(f"Using local model directory: {model_dir}")
+            print(f"Model ID from config: {model_id}")
+            break
+    
+    if not model_dir:
+        raise ValueError("No local model directory found (directory with config.json)")
+    
+    # Add o2o tools to PATH
+    env = os.environ.copy()
+    o2o_path = os.path.abspath("../o2o")
+    env["PATH"] = f"{o2o_path}:{env['PATH']}"
+    
+    python_bin = os.path.join("..", VENV_DIR, "bin", "python3")
+    export_script = os.path.join("..", "tinyllama.py")
+    
+    # 1. Generate prefill and decode circles
+    print("Running tinyllama.py (prefill)...")
+    run_command(f"{python_bin} {export_script} --mode prefill --model {model_dir}", env=env)
+    
+    print("Running tinyllama.py (decode)...")
+    run_command(f"{python_bin} {export_script} --mode decode --model {model_dir}", env=env)
+    
+    # 2. Pipeline
+    pipeline_cmd = (
+        f"fuse.attention.py < decode_.circle "
+        f"| fuse.bmm_lhs_const.py "
+        f"| reshape.io.py input --by_shape [1,16,30,4] [1,16,32,4] "
+        f"| transpose.io.kvcache.py > decode.circle"
+    )
+    run_command(pipeline_cmd, env=env)
+    
+    # 3. Merge
+    merge_cmd = (
+        f"merge.circles.py prefill.circle decode.circle "
+        f"| downcast.input_ids.py "
+        f"| gc.py > model.circle"
+    )
+    run_command(merge_cmd, env=env)
+    
+    # 4. Create package directory and copy files
+    # Find source directory with tokenizer.json
+    source_dir = None
+    for d in os.listdir("."):
+        if os.path.isdir(d) and os.path.exists(os.path.join(d, "tokenizer.json")):
+            source_dir = d
+            break
+    
+    if source_dir:
+        package_dir = f"{source_dir}.ggma"
+        print(f"Creating package directory {package_dir}...")
+        os.makedirs(package_dir, exist_ok=True)
+        
+        # Copy tokenizer and config files
+        for filename in ["tokenizer.json", "tokenizer.model", "config.json"]:
+            src = os.path.join(source_dir, filename)
+            if os.path.exists(src):
+                shutil.copy2(src, package_dir)
+        
+        # Move model.circle
+        print(f"Moving model.circle to {package_dir}...")
+        shutil.move("model.circle", os.path.join(package_dir, "model.circle"))
+    else:
+        print("Warning: Could not find source directory (directory with tokenizer.json). Leaving model.circle in current dir.")
+
+if __name__ == "__main__":
+    main()

runtime/ggma/examples/generate_text/gyu/export.py

@@ -0,0 +1,14 @@
+#!/bin/bash
+SCRIPT_DIR=$(dirname "$0")
+COMMAND="$1"
+shift  # Remove command from arguments
+
+if [ "$COMMAND" == "init" ]; then
+    python3 "$SCRIPT_DIR/$COMMAND.py" "$@"
+else
+    if [ ! -f "venv/bin/python3" ]; then
+        echo "Error: Environment not initialized. Run 'gyu init' first."
+        exit 1
+    fi
+    venv/bin/python3 "$SCRIPT_DIR/$COMMAND.py" "$@"
+fi

runtime/ggma/examples/generate_text/gyu/gyu

@@ -0,0 +1,25 @@
+#!/usr/bin/env python3
+import sys
+import os
+from huggingface_hub import snapshot_download
+
+def main():
+    if len(sys.argv) < 2:
+        print("Usage: gyu import <model_id>")
+        sys.exit(1)
+    
+    model_id = sys.argv[1]
+    model_basename = model_id.split("/")[-1].lower()
+    
+    # Create build directory
+    build_dir = "build"
+    os.makedirs(build_dir, exist_ok=True)
+    
+    # Download into build directory
+    target_dir = os.path.join(build_dir, model_basename)
+    print(f"Downloading model files for {model_id} into {target_dir}...")
+    
+    snapshot_download(repo_id=model_id, local_dir=target_dir)
+
+if __name__ == "__main__":
+    main()

runtime/ggma/examples/generate_text/gyu/import.py

@@ -0,0 +1,79 @@
+#!/usr/bin/env python3
+import os
+import shutil
+import subprocess
+import venv
+from common import VENV_DIR, PR_WORKTREE, PR_BRANCH, PR_REF, run_command
+
+def main():
+    # 1. Create virtual environment
+    if not os.path.exists(VENV_DIR):
+        print(f"Creating virtual environment in {VENV_DIR}...")
+        venv.create(VENV_DIR, with_pip=True)
+    
+    # 2. Install torch cpu first to avoid nvidia packages
+    pip_cmd = os.path.join(VENV_DIR, "bin", "pip")
+    run_command(f"{pip_cmd} install torch --index-url https://download.pytorch.org/whl/cpu")
+
+    # 3. Install requirements
+    if os.path.exists("requirements.txt"):
+        run_command(f"{pip_cmd} install -r requirements.txt")
+        
+    # 4. Clone and Install TICO (Shallow)
+    if not os.path.exists("TICO"):
+        run_command("git clone --depth 1 https://github.com/Samsung/TICO.git")
+    
+    # Patch TICO to support None condition
+    tico_record_input = os.path.join("TICO", "tico", "utils", "record_input.py")
+    if os.path.exists(tico_record_input):
+        with open(tico_record_input, "r") as f:
+            content = f.read()
+        if "self.condition = condition" in content and "self.condition = condition if condition" not in content:
+            print("Patching TICO/tico/utils/record_input.py...")
+            content = content.replace("self.condition = condition", "self.condition = condition if condition is not None else lambda args_dict: True")
+            with open(tico_record_input, "w") as f:
+                f.write(content)
+
+    run_command(f"{pip_cmd} install -e TICO --extra-index-url https://download.pytorch.org/whl/cpu")
+
+    # 5. Git worktree for PR and o2o extraction
+    if not os.path.exists("o2o"):
+        if not os.path.exists(PR_WORKTREE):
+            # Fetch PR only if worktree doesn't exist
+            try:
+                 run_command(f"git fetch https://github.com/Samsung/ONE.git {PR_REF}:{PR_BRANCH}")
+            except subprocess.CalledProcessError:
+                 print("Fetch failed, possibly branch already exists. Continuing...")
+
+            # Create worktree with no checkout
+            run_command(f"git worktree add --no-checkout -f {PR_WORKTREE} {PR_BRANCH}")
+            
+            # Configure sparse checkout
+            cwd = os.getcwd()
+            try:
+                os.chdir(PR_WORKTREE)
+                run_command("git sparse-checkout init --cone")
+                run_command("git sparse-checkout set tools/o2o")
+                # Populate files
+                run_command(f"git checkout {PR_BRANCH}")
+            finally:
+                os.chdir(cwd)
+        
+        # Move o2o to top level
+        print("Moving o2o tools to ./o2o...")
+        if os.path.exists(os.path.join(PR_WORKTREE, "tools", "o2o")):
+             shutil.move(os.path.join(PR_WORKTREE, "tools", "o2o"), "o2o")
+        
+        # Remove worktree
+        print("Removing temporary worktree...")
+        run_command(f"git worktree remove --force {PR_WORKTREE}")
+            
+    else:
+        print("o2o tools already exist.")
+        
+    # Make tools executable
+    if os.path.exists("o2o"):
+        run_command("chmod +x o2o/*.py")
+
+if __name__ == "__main__":
+    main()