Skip to content

SDPA f32 #15064

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 110 commits into from
Closed

SDPA f32 #15064

wants to merge 110 commits into from

Conversation

@wine99
Copy link

@wine99 wine99 commented Aug 4, 2025

No description provided.

YangleiZouIntel and others added 30 commits July 30, 2025 13:06
@YangleiZouIntel @ravi9
Add ggml-openvino base files
388a133
@zhanmyz @ravi9
add openvino as optional backend for Llama.cpp ggml
315f6af
@zhanmyz @ravi9
* Configure the device(default CPU) that uses OpenVINO to compile th…
2e0c4cf 
…e model

 * Add OpenVINO ADD operator to Llama.cpp. The output is somewhat abnormal and needs further debugging.
@zhanmyz @ravi9
Solve the issue of abnormal model output caused by using OpenVINO ADD…
d363ff6 
… operator
@zhanmyz @ravi9
Add OpenVINO MUL operator to GGML of Llama.cpp.
0d36b77
@zhanmyz @ravi9
Add compile options
76a273b
@zhanmyz @ravi9
add OpenVINO frontend convert process steps
fa4a6ab
@zhanmyz @ravi9
add get openvino available ops function
13a8828
@yumengbo @ravi9
Add PoC of integration of openvino frontend. Main changes: ggml-ov-fr…
0a96956 
…ontend-utils, GraphIterator, Decoder
@yumengbo @ravi9
Implement GgmlOvDecoder. Add dump functions.
9f976bf
@yumengbo @ravi9
Convert subgraph with add, sub, mul, div op to ov model and do infer …
b052547 
…on openvino device
@yumengbo @ravi9
Add GGML_OV_FRONTEND option. Add readme.
09ba65b
@yumengbo @ravi9
Change output for infer request to set output tensor. Support scale, …
cde1059 
…view op.
@zhanmyz @ravi9
add GET_ROWS operator of OpenVINO to GGML of llama.cpp
3546a85
@zhanmyz @ravi9
Update build.md and add operation mapping(GGML to OpenVINO)
e9668e4
@zhanmyz @ravi9
add the rms_norm operator implemented using OpenVINO to the GGML back…
694d9c9 
…end of llama.cpp
@yumengbo @ravi9
Fix issue for output memory copy of infer request
d0426f4
@yumengbo @ravi9
Change to implementation following pytorch frontend
ad958b0
@yumengbo @ravi9
Add support for UNARY SILU op . Fix pytorch impl bugs.
e0749e7
@yumengbo @ravi9
Support Softmax op
2ba30f7
@yumengbo @ravi9
Support Softmax op
ec40661
@yumengbo @ravi9
Support ROPE op.
c6afac5
@zhanmyz @ravi9
Add support for RMS_NORM OP
62ca2d5
@zhanmyz @ravi9
Add MUL_MAT,CPY,CONT as operators implemented in OpenVINO for GGML ba…
8c9a672 
…ckend
@zhanmyz @ravi9
Move CPY from GGML OV Backend to OV Frontend
f591d22
@zhanmyz @ravi9
add implementation of MUL_MAT, CPY, CONT of GGML ops using OV ops
db56d18
@zhanmyz @ravi9
add implementation of CPY when the output tensor is non-contiguous
4751087
@zhanmyz @ravi9
add tmp source code files
dedd2fa
@zhanmyz @ravi9
Execute singel CONT operator is OK
c73594c
@zhanmyz @ravi9
Execute CONT & VIEW operators in OV Frontend is OK
79d3824
@wine99 wine99 requested review from ggerganov and ngxson as code owners August 4, 2025 05:19
@wine99
Copy link
Author

wine99 commented Aug 4, 2025

Sorry this PR is created by mistake

@wine99 wine99 closed this Aug 4, 2025
@github-actions github-actions bot added documentation Improvements or additions to documentation build Compilation issues testing Everything test related devops improvements to build systems and github actions ggml changes relating to the ggml tensor library for machine learning labels Aug 4, 2025
@wine99 wine99 deleted the sdpa_f32 branch October 15, 2025 06:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ngxson ngxson Awaiting requested review from ngxson ngxson is a code owner

@ggerganov ggerganov Awaiting requested review from ggerganov ggerganov is a code owner

Assignees

No one assigned

Labels

build Compilation issues devops improvements to build systems and github actions documentation Improvements or additions to documentation ggml changes relating to the ggml tensor library for machine learning testing Everything test related

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@wine99 @YangleiZouIntel @zhanmyz @yumengbo @virajwad @ravi9 @cavusmustafa