Skip to content

Conversation

@q5sys
Copy link

@q5sys q5sys commented Jan 8, 2026

Problem:
Running LTX-2 inference with identical inputs (same seed, prompt, settings) produced different video outputs between runs. This violated the expectation that a fixed seed should yield reproducible results.
See links below for example output proving this.

Root Cause:
Best I can tell, PyTorch and CUDA prioritize performance over bitwise determinism by default: scaled_dot_product_attention uses non-deterministic fused kernels, and cuBLAS may use non-deterministic algorithms.

Solution:
Added TI2VidTwoStagesPipelineDeterministic class with enable_deterministic_mode() that enforces reproducibility by:

  • Setting torch.manual_seed() and CUDA seeds from the user-provided --seed
  • Enabling torch.backends.cudnn.deterministic = True
  • Disabling torch.backends.cudnn.benchmark
  • Setting torch.use_deterministic_algorithms(True, warn_only=True)
  • Configuring CUBLAS_WORKSPACE_CONFIG for deterministic matrix ops

Performance Impact (RTX 5090 benchmark):

Original Deterministic Slowdown
Stage 1: 1.24 it/s 1.16 it/s ~6%
Stage 2: 1.18 s/it 1.22 s/it ~3%
Stage 3: 1.47 it/s 1.18 it/s ~20%

Overall ~1-2 seconds additional time for a ~27 second generation. The original TI2VidTwoStagesPipeline is preserved for users who prioritize speed.

I thought about making alterations to the original pipeline and enabling a --deterministic or --reproducible flag which would then enable this capability, but I figured it was safer and cleaner to just create a new pipeline since I'm not aware of what additional development you all have on your internal roadmap, and I wanted this to be as clean and simple for you to merge this into your existing development work.

Reference Files for PR: (See for scripts that were run and the output video)

With the Upstream Code:
Script: https://github.com/q5sys/hotdog-determinism/blob/main/run-ltx-hotdog.sh
Shell Log: https://github.com/q5sys/hotdog-determinism/blob/main/ltx-hotdog-shell-output.txt
Output 1: https://github.com/q5sys/hotdog-determinism/blob/main/hotdog-gpu0-1.mp4
Output 2: https://github.com/q5sys/hotdog-determinism/blob/main/hotdog-gpu0-2.mp4

With the Deterministic Pipeline:
Script: https://github.com/q5sys/hotdog-determinism/blob/main/run-ltx-hotdog-deterministic.sh
Shell Log: https://github.com/q5sys/hotdog-determinism/blob/main/ltx-hotdog-deterministic-shell-output.txt
Output 1: https://github.com/q5sys/hotdog-determinism/blob/main/hotdog-gpu0-deterministic-1.mp4
Output 2: https://github.com/q5sys/hotdog-determinism/blob/main/hotdog-gpu0-deterministic-2.mp4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant