-
Notifications
You must be signed in to change notification settings - Fork 12.2k
musa: enable fp16 mma (all) and cublas on qy2 #13842
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
3 tasks
JohannesGaessler
approved these changes
May 29, 2025
Let's hold off on merging this until I upgrade to the upcoming MUSA SDK and verify that everything works as expected. |
Signed-off-by: Xiaodong Ye <[email protected]>
Co-authored-by: Johannes Gäßler <[email protected]>
Signed-off-by: Xiaodong Ye <[email protected]>
Signed-off-by: Xiaodong Ye <[email protected]>
Signed-off-by: Xiaodong Ye <[email protected]>
Rebased onto |
gabe-l-hart
added a commit
to gabe-l-hart/llama.cpp
that referenced
this pull request
Jun 27, 2025
* mamba2-sync: (22 commits) recurrent : call balloc split_reset() in init_batch() (ggml-org#14414) ggml : add ggml_set_rows (ggml-org#14274) convert : fix broken sentencepiece vocab (ggml-org#14416) mamba : fix mismatched new and delete size for llm_build_mamba model : gemma3n text-only (ggml-org#14400) cmake: regen vulkan shaders when shaders-gen sources change (ggml-org#14398) llama : return mistral-v7-tekken as default template only (ggml-org#14390) metal : add special-case mat-vec mul for ne00 == 4 (ggml-org#14385) metal : batch rows copy in a single threadgroup (ggml-org#14384) docs: update s390x documentation + add faq (ggml-org#14389) musa: enable fp16 mma (all) and cublas on qy2 (ggml-org#13842) ggml-cpu: enable IBM NNPA Vector Intrinsics (ggml-org#14317) ggml : do not output unprintable characters on GGUF load failure (ggml-org#14381) sycl: GGML_SYCL_DISABLE_OPT on by default for all Intel Devices (ggml-org#13973) opencl: ref count `ggml_backend_opencl_context` and refactor profiling (ggml-org#14254) batch : fix check for empty sequences in memory (ggml-org#14364) cmake : use LLAMA_BUILD_NUMBER when defining LLAMA_INSTALL_VERSION (ggml-org#14362) server : move no API key doc to /health (ggml-org#14352) main : honor --verbose-prompt on interactive prompts (ggml-org#14350) jinja : Add Mistral-Small-3.2-24B-Instruct-2506.jinja (ggml-org#14349) ...
Nexesenex
added a commit
to Nexesenex/croco.cpp
that referenced
this pull request
Jun 28, 2025
This reverts commit 716301d.
Nexesenex
added a commit
to Nexesenex/croco.cpp
that referenced
this pull request
Jun 28, 2025
This reverts commit 716301d.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
ggml
changes relating to the ggml tensor library for machine learning
Nvidia GPU
Issues specific to Nvidia GPUs
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Make sure to read the contributing guidelines before submitting a PR
This PR is a rework of #13149.
Regarding the
cublasGemmBatchedEx
issue on the MTT S4000, I will continue investigating it with our MUBLAS team to identify a proper solution.Testing Done
All tests below were performed on the MTT S80.
./build/bin/test-backend-ops
passedDeepSeek-R1-Distill-Qwen-7B-Q4_K_M.gguf
,qwen3_8b_q4_k_m.gguf
,nvidia-llama-3_1-nemotron-nano-8b-v1-q4_k_m.gguf
with or without the-fa
flag