NPU adaption for FLUX #10465
NPU adaption for FLUX #10465
Conversation
|
@sayakpaul Please take a look at this Flux adaption. Thank you! |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| self.norm1_context = AdaLayerNormZero(dim) | ||
|
|
||
| if hasattr(F, "scaled_dot_product_attention"): | ||
| processor = FluxAttnProcessor2_0() |
There was a problem hiding this comment.
let's keep the default logic simple i.e. remove the changes from this file
you can use NPU with set_attn_processor, no?
There was a problem hiding this comment.
@yiyixuxu In the attention processor, it has NPU FA, but only with AttnProcessorNPU. Or do you want me to add a function in attention_processor.py to change the processor there? I think right now the selection is only in the init, so it shouldn't be any logic changes. Please let me know what you think so that I can modify based on that. Thanks
In the attention processor, it has NPU FA, but only with AttnProcessorNPU. Or do you want me to add a function in attention_processor.py to change the processor there? I think right now the selection is only in the init, so it shouldn't be any logic changes. Please let me know what you think so that I can modify based on that. Thanks
|
@yiyixuxu Please take a look at this, thanks |
|
hi @leisuzz I think we are not going to change default attention processor for now, so we won't be able to accept this change at the moment. Sorry about the back and forth! |
|
this means user just has to manually run this to use NPU for now pipe.set_attn_processor(FusedFluxAttnProcessor2_0_NPU()) |
4 participants
What does this PR do?
Added Flash Attention for FLUX
Fixed issue in Flux Controlnet when using enable flash attention
Before submitting
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.