QVAC-19998 feat(ltx): full-GPU LTX-2.3 video support (Metal)#13
QVAC-19998 feat(ltx): full-GPU LTX-2.3 video support (Metal)#13aegioscy wants to merge 7 commits into
Conversation
Port of 6ded9e6 onto 2026-06-04 base: use ggml_rope_flux fused op for q/k in the Flux attention path (rope.hpp) and for v in the flash-attn wrapper (ggml_extend.hpp) when the backend supports it, with a CPU fallback to the existing permute/reshape path. Bumps ggml submodule to aegioscy/qvac-ext-ggml@c40a0fc which provides GGML_OP_ROPE_FLUX with CPU + Metal kernels. Co-authored-by: Cursor <cursoragent@cursor.com>
ggml is now provided entirely by the vcpkg ggml port (tetherto/qvac-ext-ggml). The submodule is not needed and was only present for non-vcpkg local builds. Co-authored-by: Cursor <cursoragent@cursor.com>
The ../ggml/src/ggml-impl.h path was a remnant of the git submodule. All used symbols (GGML_MAX_DIMS, GGML_MAX_SRC, GGML_ASSERT, ggml_graph_n_nodes, ggml_graph_node) are in the public ggml.h. Co-authored-by: Cursor <cursoragent@cursor.com>
ggml_cgraph is now opaque (ggml-impl.h no longer vendored). Replace direct member access with the public leaf accessors exported by qvac-ext-ggml 2026-06-06, fixing 'member access into incomplete type ggml_cgraph'. Co-authored-by: Cursor <cursoragent@cursor.com>
SDContextParams::to_sd_ctx_params_t left preferred_gpu_backend out of the sd_ctx_params_t aggregate initializer, so it zero-initialized to SD_BACKEND_PREF_CPU and the whole LTX pipeline ran on CPU even on Metal machines. Add SD_BACKEND_PREF_GPU explicitly; it is only honored when --backend is unset, so existing --backend overrides are unaffected. Co-authored-by: Cursor <cursoragent@cursor.com>
With the LTX-2 additions, the stable-diffusion.cpp translation unit exceeds MSVC's COFF 2^16 section limit and fails to compile on Windows with fatal error C1128. /bigobj raises the limit; clang/gcc are unaffected. Co-authored-by: Cursor <cursoragent@cursor.com>
|
The default build path looks broken on this branch. Verified on a clean checkout of df47d5e: cmake .. with default flags (the flow docs/build.md documents) — exit 1: CMake Error at CMakeLists.txt:312 (add_subdirectory): Questions: Is consuming this repo directly (without the vcpkg overlay) unsupported on this branch? If yes, can we either flip SD_USE_SYSTEM_GGML default to ON, or replace the add_subdirectory(ggml) fallback with a message(FATAL_ERROR ...) pointing users at -DSD_USE_SYSTEM_GGML=ON + the qvac-ext-ggml package? |
Addresses review feedback on PR #13. After removing the ggml submodule, a plain `cmake ..` defaulted SD_USE_SYSTEM_GGML=OFF and hit add_subdirectory(ggml) on a now-missing directory, failing with a confusing CMake error. - Default SD_USE_SYSTEM_GGML to ON (system/vcpkg ggml is the only supported path on this branch). - Replace the add_subdirectory(ggml) fallback with explicit FATAL_ERROR messages pointing at the qvac-ext-ggml vcpkg port and the vcpkg toolchain file (both for missing system ggml and for an explicit -DSD_USE_SYSTEM_GGML=OFF with no submodule present). - Update docs/build.md: drop the --recursive / git submodule update instructions and document the system/vcpkg ggml workflow. Note that the port exports GGML_MAX_NAME=128 as a PUBLIC compile definition so consumers inherit it automatically. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Thanks @dev-nid — all four points addressed in 6a13b9c:
|
Summary
Splits the LTX / custom-ggml delta out of the
2026-06-04fork base.2026-06-04(base) = upstream1f9ee88+ the 5 general, upstream-ggml-compatible patches: vcpkg port infra, Flux qkv assert, ESRGAN device API, upscaler defaults, Wan I2V VAE tiling bypass. Itsggmlsubmodule stays at stock leejet/ggml0ce7ad3.This PR adds the 6 commits that depend on / switch to the unified qvac ggml:
feat: use fused Flux RoPE when available— requiresGGML_OP_ROPE_FLUX(not in stock leejet ggml); bumps submodule to qvac-ext-ggmlchore: remove ggml git submodule — use vcpkg SD_USE_SYSTEM_GGMLfix: remove ggml-impl.h include — symbols are in public ggml.hggml_graph_cut: use public ggml_graph_leaf/n_leafs/add_leaf APIcli: default preferred_gpu_backend to GPU— fixes the LTX pipeline running on CPUcmake: add /bigobj for MSVC— build fix for the larger LTX/system-ggml objectsLTX-2.3 model support itself comes from upstream; these are the qvac-side build/packaging reconciliation + the custom-ggml dependency.
Notes
2026-06-04-ltxtip (df47d5e) is unchanged and byte-for-byte identical to the original fork tip.Test plan
2026-06-04-ltx