feat(dev): add Ansible playbook for ComfyStream setup (#114)

This commit adds an Ansible playbook for automated ComfyStream setup, switches Caddy release to `stable`, and prevents cloud-init from duplicating import lines. Also introduces a `--bare-vm` option for deploying a clean TensorDock instance without ComfyStream and refines cloud-init template behavior for better reliability.

Author: rickstaa

README.md

@@ -10,6 +10,7 @@ This repo also includes a WebRTC server and UI that uses comfystream to support
     - [Docker Image](#docker-image)
       - [RunPod](#runpod)
       - [Tensordock](#tensordock)
+      - [Other Cloud Providers](#other-cloud-providers)
   - [Download Models](#download-models)
   - [Install package](#install-package)
     - [Custom Nodes](#custom-nodes)
@@ -42,11 +43,15 @@ The RunPod template [livepeer-comfystream](https://runpod.io/console/deploy?temp
 
 #### Tensordock
 
-We also have a python script that can be used to spin up a ComfyStream instance on a [Tensordock server](https://tensordock.com/). Refer to [scripts/README.md](scripts/README.md) for instructions.
+We also have a python script that can be used to spin up a ComfyStream instance on a [Tensordock server](https://tensordock.com/). Refer to [scripts/README.md](./scripts/README.md#tensordock-auto-setup-fully-automated) for instructions.
+
+#### Other Cloud Providers
+
+We also provide an [Ansible playbook](https://docs.ansible.com/ansible/latest/installation_guide/index.html) for deploying ComfyStream on any cloud provider. Refer to [scripts/README.md](./scripts/README.md#cloud-agnostic-automated-setup-ansible-based-deployment) for instructions.
 
 ## Download Models
 
-Refer to [src/comfystream/scripts/README.md](src/comfystream/scripts/README.md) for instructions to download commonly used models.
+Refer to [scripts/README.md](src/comfystream/scripts/README.md) for instructions to download commonly used models.
 
 ## Install package
 
@@ -83,7 +88,7 @@ pip install git+https://github.com/yondonfu/comfystream.git
 
 ### Custom Nodes
 
-comfystream uses a few auxiliary custom nodes to support running workflows. 
+Comfystream uses a few auxiliary custom nodes to support running workflows.
 
 **Note:** If you are using comfystream as a custom node in ComfyUI, you can skip the following steps.
 

scripts/README.md

@@ -1,48 +1,108 @@
 # ComfyUI Helper Scripts
 
-This directory contains various helper scripts designed to streamline working with ComfyStream:
+This directory contains helper scripts to simplify the deployment and management of **ComfyStream**:
 
-- `spinup_comfystream_tensordock.py`: Spins up a ComfyStream instance on a [Tensordock server](https://tensordock.com/), installs dependencies, and starts the server.
+- **Ansible Playbook (`ansible/plays/setup_comfystream.yml`)** – Deploys ComfyStream on any cloud provider.  
+- **`spinup_comfystream_tensordock.py`** – Fully automates VM creation and ComfyStream setup on a [TensorDock server](https://tensordock.com/).  
 
 ## Usage Instructions
 
-### Spinup TensorDock ComfyStream Instance
+### Ansible Playbook (Cloud-Agnostic Deployment)
 
-1. **Create a Tensordock Account**: Register at [Tensordock](https://dashboard.tensordock.com/register), attach a credit card, and create an API key and token.
-2. **Set Up a Python Virtual Environment**:
-   Create and activate a virtual environment using [Conda](https://docs.anaconda.com/miniconda/) and install the required dependencies:
+This repository includes an [Ansible playbook](https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_intro.html) to deploy ComfyStream on any cloud provider. Follow the steps below:
+
+1. **Create a VM**:  
+   Deploy a VM on a cloud provider such as [TensorDock](https://marketplace.tensordock.com/deploy?gpu=geforcertx4090-pcie-24gb&gpuCount=1&ramAmount=16&vcpuCount=4&storage=100&os=Ubuntu-22.04-LTS), AWS, Google Cloud, or Azure with the following minimum specifications:
+   - **GPU**: 20GB VRAM  
+   - **RAM**: 16GB  
+   - **CPU**: 4 vCPUs  
+   - **Storage**: 100GB
+
+   > [!TIP]
+   > You can use the `spinup_comfystream_tensordock.py --bare-vm` script to create a [compatible VM on TensorDock](https://marketplace.tensordock.com/deploy?gpu=geforcertx4090-pcie-24gb&gpuCount=1&ramAmount=16&vcpuCount=4&storage=100&os=Ubuntu-22.04-LTS)
+
+2. **Open Required Ports**:  
+   Ensure the following ports are open **inbound and outbound** on the VM's firewall/security group:
+   - **SSH (Port 22)** – Remote access  
+   - **HTTPS (Port 8189)** – ComfyStream access  
+  
+3. **Install Ansible**:  
+   Follow the [official Ansible installation guide](https://docs.ansible.com/ansible/latest/installation_guide/index.html).
+
+4. **Configure the Inventory File**:
+   Add the VM’s public IP to `ansible/inventory.yml`.
+
+5. **Change the ComfyUI Password:**  
+   Open the `ansible/plays/setup_comfystream.yaml` file and replace the `comfyui_password` value with your own secure password.
+
+6. **Run the Playbook**:  
+   Execute:
 
     ```bash
-    conda create -n comfystream python=3.8
-    conda activate comfystream
-    pip install -r requirements.txt
+    ansible-playbook -i ansible/inventory.yaml ansible/plays/setup_comfystream.yaml
     ```
 
-3. **Check Available Options** *(Optional but Recommended)*:  
-   To see all available script options, run:
+   > [!IMPORTANT]  
+   > When using a non-sudo user, add `--ask-become-pass` to provide the sudo password or use an Ansible vault for secure storage.
+
+7. Access the Server:
+   Once the **Ansible playbook** completes successfully, the **ComfyStream** container will start, downloading models and building TensorRT engines. When ready, access **ComfyUI** at `https://<VM_IP>:<PORT_THAT_FORWARDS_TO_8189>`.
+
+> [!IMPORTANT]
+> If you encounter a `toomanyrequests` error while pulling the Docker image, either wait a few minutes or provide your Docker credentials when running the playbook:  
+>
+> ```bash
+> ansible-playbook -i ansible/inventory.yaml ansible/plays/setup_comfystream.yaml -e "docker_hub_username=your_dockerhub_username docker_hub_password=your_dockerhub_pat"
+> ```
+
+> [!TIP]
+> The [ComfyStream Docker image](https://hub.docker.com/r/livepeer/comfystream/tags) is **~20GB**. To check download progress, SSH into the VM and run:
+>
+> ```bash
+> docker pull livepeer/comfystream:latest
+> ```
+
+### TensorDock Spinup Script (Fully Automated)
+
+The `spinup_comfystream_tensordock.py` script automates VM provisioning, setup, and server launch on [TensorDock](https://tensordock.com/). Follow the steps below:
+
+1. **Create a TensorDock Account**: Sign up at [Tensordock](https://dashboard.tensordock.com/register), add a payment method, and generate API credentials.
+
+2. **Set Up a Python Virtual Environment**:
+   To prevent dependency conflicts, create and activate a virtual environment with [Conda](https://docs.anaconda.com/miniconda/) and install the required dependencies:
+
+    ```bash
+   conda create -n comfystream python=3.8
+   conda activate comfystream
+   pip install -r requirements.txt
+   ```
+
+3. **View Available Script Options** *(Optional)*:  
+   To see all available options, run:
 
     ```bash
     python spinup_comfystream_tensordock.py --help
     ```
 
 4. **Run the Script**:  
-   Execute the following command to spin up a ComfyStream instance:
+   Execute the following command to provision a VM and set up ComfyStream automatically:
 
     ```bash
     python spinup_comfystream_tensordock.py --api-key <API_KEY> --api-token <API_TOKEN>
     ```
 
 5. **Access the Server**:  
-   The script will set up the instance, install dependencies, and start the server. This process will take a few minutes. Once completed, you can access the ComfyStream server and UI using the provided URLs.
+   Once the setup is complete, the script will display the URLs to access ComfyStream.
 
-6. **Stop the Server**:  
-   To stop the server, run:
+6. **Stop & Delete the VM** *(When No Longer Needed)*:
+   To stop and remove the instance, run:
 
     ```bash
     python spinup_comfystream_tensordock.py --delete <VM_ID>
     ```
 
-   Replace `<VM_ID>` with the ID of the VM you want to delete. You can find the VM ID in the script logs or the [Tensordock dashboard](https://dashboard.tensordock.com/).
+   Replace `<VM_ID>` with the VM ID found in the script logs or the [TensorDock dashboard](https://dashboard.tensordock.com/instances).
 
 > [!WARNING]
-> If you see `max retries exceeded with url` errors, the VM was likely created but is inaccessible. Check the [TensorDock dashboard](https://dashboard.tensordock.com/instances), delete the VM, wait 2-3 minutes, then run the script again.
+> If you encounter `max retries exceeded with URL` errors, the VM might have been created but is inaccessible.  
+> Check the [TensorDock dashboard](https://dashboard.tensordock.com/instances), delete the VM manually, wait **2-3 minutes**, then rerun the script.

scripts/ansible/inventory.yaml

@@ -0,0 +1,8 @@
+---
+all:
+  hosts:
+    comfystream_vm:
+      ansible_host: 123.234.102.170                     # Replace with your VM's IP address
+      ansible_port: 22                                  # Replace with your VM's SSH port
+      ansible_user: user                                # Replace with your SSH user
+      ansible_ssh_private_key_file: ~/.ssh/private_key  # Path to your SSH key