Skip to content

Latest commit

 

History

History
150 lines (109 loc) · 3.82 KB

README.md

File metadata and controls

150 lines (109 loc) · 3.82 KB

LTX-2 Generate Video (RunPod Serverless)

A RunPod Serverless worker template that generates videos using LTX-2 via ComfyUI + ComfyUI-LTXVideo.

This template supports:

  • Text-to-video (mode="t2v")
  • Image-to-video (mode="i2v")

It returns the generated MP4 as Base64 (data:video/mp4;base64,...) in the response JSON.

The overall design mirrors the Wan2.2 template in wlsdml1114/generate_video, but swaps the workflow and models for LTX-2.

Deployment on RunPod

  1. Create a new Serverless Endpoint on RunPod.
  2. Point it at this repo/folder as the Docker context (or push this folder as its own repo and use that URL).
  3. Build the image. The Dockerfile will:
    • Install ComfyUI
    • Install ComfyUI-LTXVideo
    • Copy the LTX-2 workflows from workflows/
  4. Attach a Network Volume with your LTX-2 weights:
    • Default paths are configured in extra_model_paths.yaml.

Environment variables (optional):

  • COMFY_HOST (default: 127.0.0.1)
  • COMFY_PORT (default: 8188)
  • COMFY_DIR (default: /workspace/ComfyUI)
  • WORKFLOW_T2V_PATH (default: /app/workflows/ltx2_t2v_full.json)
  • WORKFLOW_I2V_PATH (default: /app/workflows/ltx2_i2v_full.json)

API

Input

Send a RunPod job with an input object.

Common fields:

  • mode: "t2v" or "i2v" (required)
  • prompt: string (required)
  • negative_prompt: string (optional)
  • seed: int (optional)
  • steps: int (optional)
  • cfg: float (optional)
  • width: int (optional)
  • height: int (optional)
  • fps: float (optional)
  • num_frames: int (optional)

For mode="i2v", provide one image input:

  • image_path (string) OR
  • image_url (string) OR
  • image_base64 (string, can be data:image/...;base64,...)

Output

Success:

{ "video": "data:video/mp4;base64,AAAA..." }

Error:

{ "error": "message" }

Network Volume layout (recommended)

This template expects your ComfyUI models on a mounted volume. You can adapt paths in extra_model_paths.yaml, but a typical structure is:

ComfyUI/
  models/
    checkpoints/
      ltx-2-19b-dev-fp8.safetensors
    text_encoders/
      gemma_3_12B_it_fp4_mixed.safetensors
    loras/
      ltx-2-19b-distilled-lora-384.safetensors
    latent_upscale_models/
      ltx-2-spatial-upscaler-x2-1.0.safetensors

Attach this directory as a Network Volume to your Serverless endpoint and make sure it is mounted under /workspace/ComfyUI/models (or update extra_model_paths.yaml to match your layout).

Python client

See generate_video_client.py for a convenience wrapper.

Basic text-to-video usage:

from generate_video_client import GenerateVideoClient

client = GenerateVideoClient(
    runpod_endpoint_id="your-endpoint-id",
    runpod_api_key="your-runpod-api-key",
)

result = client.create_video_t2v(
    prompt="cinematic shot of a neon-lit rainy street at night",
    width=1280,
    height=720,
    num_frames=121,
    fps=24,
    seed=42,
)

if "video" in result:
    client.save_video_result(result, "./ltx2_t2v_demo.mp4")

Basic image-to-video usage:

from generate_video_client import GenerateVideoClient

client = GenerateVideoClient(
    runpod_endpoint_id="your-endpoint-id",
    runpod_api_key="your-runpod-api-key",
)

result = client.create_video_i2v(
    image_path="./input.png",
    prompt="slow dolly-in shot, cinematic lighting",
    num_frames=121,
    fps=24,
)

if "video" in result:
    client.save_video_result(result, "./ltx2_i2v_demo.mp4")

The client uses the RunPod /runsync API under the hood and returns the handler output directly.

References

  • LTX-2 ComfyUI tutorial: https://docs.comfy.org/tutorials/video/ltx/ltx-2
  • LTX docs (ComfyUI integration): https://docs.ltx.video/open-source-model/integration-tools/comfy-ui