Commit 7f2b67a

authored

Fused Metal SDPA with masking! (#1225)

* Metal SDPA with masking * Much faster quantization on metal! * Check if actually metal * Materialize the mask

1 parent e7371c1 commit 7f2b67aCopy full SHA for 7f2b67a

17 files changed

+233

-128

lines changed

Cargo.lock
Cargo.toml
mistralrs-core
- Cargo.toml
- src
  - attention.rs
  - pipeline
    - isq.rs
mistralrs-quant/src
- bitsandbytes
  - mod.rs
- blockwise_fp8
  - mod.rs
- distributed
  - layers.rs
- dummy
  - mod.rs
- fp8
  - mod.rs
- gguf
  - mod.rs
- gptq
  - gptq_cpu.rs
  - gptq_cuda.rs
- hqq
  - mod.rs
- lib.rs
- unquantized
  - mod.rs
- utils
  - isq.rs

17 files changed

+233

-128

lines changed

`‎Cargo.lock`

Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Comments

(0)