Support flash-attn 2.7.0 in FPDT attention - #8022
Conversation
flash-attn 2.7.0 split the window_size argument into separate window_size_left/window_size_right ints, and reduced the _flash_attn_forward return from 8 values to 4. Add a 2.7.0+ branch to both _FPDTGPUAttentionImpl_ and _FPDTGPUOffloadingAttentionImpl_ (forward + backward) so FPDT keeps working with newer flash-attn releases while remaining compatible with 2.6.x and earlier. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> Signed-off-by: binchengxiong <binchengxiong@alibaba-inc.com>
|
If we don't add the uploaded commit, we would come across the following error message when running the test command using flash-attn 2.7.0 version:
|
|
@YJHMITWEB Can you review this PR? |
Hi @tohtana , this PR looks good to me, as it only includes a version check. We can bring it in with a rebase. |
tohtana
left a comment
There was a problem hiding this comment.
Thank you @xbcReal! Let me merge this.
@YJHMITWEB Thank you for your review!
## Summary - flash-attn 2.7.0 split the `window_size` argument into `window_size_left`/`window_size_right` ints, and reduced `_flash_attn_forward`'s return from 8 values to 4. Calling FPDT with flash-attn >= 2.7 currently errors out. - Add a `>= 2.7.0` branch to both `_FPDTGPUAttentionImpl_` and `_FPDTGPUOffloadingAttentionImpl_` (forward + backward) so FPDT keeps working with newer flash-attn releases while remaining compatible with 2.6.x and earlier. - Update `FPDT_Attention`'s ImportError message so it no longer pins 2.6.3. Signed-off-by: binchengxiong <binchengxiong@alibaba-inc.com> Co-authored-by: binchengxiong <binchengxiong@alibaba-inc.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: Masahiro Tanaka <81312776+tohtana@users.noreply.github.com>
## Summary - flash-attn 2.7.0 split the `window_size` argument into `window_size_left`/`window_size_right` ints, and reduced `_flash_attn_forward`'s return from 8 values to 4. Calling FPDT with flash-attn >= 2.7 currently errors out. - Add a `>= 2.7.0` branch to both `_FPDTGPUAttentionImpl_` and `_FPDTGPUOffloadingAttentionImpl_` (forward + backward) so FPDT keeps working with newer flash-attn releases while remaining compatible with 2.6.x and earlier. - Update `FPDT_Attention`'s ImportError message so it no longer pins 2.6.3. Signed-off-by: binchengxiong <binchengxiong@alibaba-inc.com> Co-authored-by: binchengxiong <binchengxiong@alibaba-inc.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: Masahiro Tanaka <81312776+tohtana@users.noreply.github.com> Signed-off-by: nathon-lee <leejianwoo@gmail.com>
## Summary - flash-attn 2.7.0 split the `window_size` argument into `window_size_left`/`window_size_right` ints, and reduced `_flash_attn_forward`'s return from 8 values to 4. Calling FPDT with flash-attn >= 2.7 currently errors out. - Add a `>= 2.7.0` branch to both `_FPDTGPUAttentionImpl_` and `_FPDTGPUOffloadingAttentionImpl_` (forward + backward) so FPDT keeps working with newer flash-attn releases while remaining compatible with 2.6.x and earlier. - Update `FPDT_Attention`'s ImportError message so it no longer pins 2.6.3. Signed-off-by: binchengxiong <binchengxiong@alibaba-inc.com> Co-authored-by: binchengxiong <binchengxiong@alibaba-inc.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: Masahiro Tanaka <81312776+tohtana@users.noreply.github.com>
Summary
window_sizeargument intowindow_size_left/window_size_rightints, and reduced_flash_attn_forward's return from 8 values to 4. Calling FPDT with flash-attn >= 2.7 currently errors out.>= 2.7.0branch to both_FPDTGPUAttentionImpl_and_FPDTGPUOffloadingAttentionImpl_(forward + backward) so FPDT keeps working with newer flash-attn releases while remaining compatible with 2.6.x and earlier.FPDT_Attention's ImportError message so it no longer pins 2.6.3.