checkpoint #3357
Open
checkpoint #3357
+665
−2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Upgraded TVM submodule to FFI bump commit (f68651f035) - Fixed script printer namespace mismatch (node->script) - Added conditional script printer imports with dummy fallbacks - Resolved CMake compatibility issues in tokenizers-cpp submodules - Added mlc_llm console script entry point to pyproject.toml - Established virtual environment isolation for clean builds - TVM v0.22 now imports successfully without errors - MLC-LLM CLI functional with TVM v0.22 backend - Ready for Phase 2: DLPack Type System Migration Technical fixes: - C++: script_printer.cc namespace registration fix - Python: Optional Scriptable import with comprehensive fallback - Build: TVM Python package separate installation requirement - Environment: Virtual environment isolation for reproducibility
- Updated tokenizers-cpp to commit 405aa4fa - Updated TVM to commit 52a49c82 -hotfix for TVM cpp dependency.
Phase 1 completed basic TVM v0.22 integration, but model compilation fails with segmentation fault during convert_weight operation. Root Cause: DLPack type system incompatibility - TVM v0.22 changed DLTensor → DLNDArray - TVM v0.22 changed DLManagedTensor → DLManagedNDArray - MLC-LLM still uses old DLPack types for tensor operations Impact: Cannot compile Gemma-3-270M or any models Solution: Phase 2 DLPack migration required immediately Validation: The refactor.md complexity assessment was accurate - Phase 1 alone insufficient for full functionality.
… not DLPack Root cause identified: - Segfault occurs during TIR static initialization for Gemma3 sliding window attention - NOT DLPack type incompatibility as initially assumed - Issue is in TIR code generation for sliding window attention mechanisms - Confirmed: Even q0f16 (no quantization) still segfaults - Hypothesis: Missing TIR bitwise operations using powers of 2 for sliding window masks Phase 1: ✅ TVM basic integration successful Phase 2: 🔴 TIR sliding window operations required (not DLPack migration) User insight: 'bitwise stuff happens in quantization' - correct, but issue is broader - TIR generation for sliding window attention patterns fails.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.