Merged
Conversation
…ions have not changed
…with split_alignment > 1
* Modify the static solver so that each segment of input_split_size is divisible by the same number * modify chunk logic in static solver
* add merge_with_split_alignment method in AttnRanges * support split alignment in dynamic solver
* relaxed the buffer size up to INT_MAX limit for internode * tested over INT_MAX buffer size in exp/grpcoll tests * minor fixed * added docstring for config funcs * added minimium num bytes check for native grpcoll * fixed tma bytes and num warps for internode cache notify kernel * raised up default num_rdma_bytes * further fixed internode cache notify kernel for group reduce * removed the temp debug code to make benchmark mask split-aligned
* added num_heads_q,kv,group to comm meta for dynamic solver; added seperate split alignment for kv/qo * added num_heads_q/kv to comm meta for dynamic solver * supported split alignment varying from dtype * added native_grpcoll_split_alignment to test_pipeline/test_pipeline_sdpa * tested through dynamic split alignment for pipeline ut; added world size offset for seed * added some comments * added MAGI_ATTENTION_NATIVE_GRPCOLL_SPLIT_ALIGNMENT to docs
* updated and polished api for required num_heads_q, num_heads_kv, head_dim * adjusted the calls in ut for updated APIs * adjusted the calls in examples for updated APIs * adjusted the calls in exps for updated APIs * adjusted the calls in docs and readme for updated APIs, as well as deleting the magi_attn_varlen_dipatch and magi_attn_flex_dispatch deprecated APIs * minor updated tests/test_api/test_interface.py * minor updated benchmark dockerfile
* added head dim to comm meta * supported auto split alignment w/o varying from dtypes * minor updated repr and utils * added strategy for calc_split_alignment
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.