Skip to content

Use Q4_K for attn_v for Q2_K_S when n_gqa >= 4#4996

Merged
ggerganov merged 1 commit intomasterfrom
ik/better_q2_k_s
Jan 17, 2024
Merged

Use Q4_K for attn_v for Q2_K_S when n_gqa >= 4#4996
ggerganov merged 1 commit intomasterfrom
ik/better_q2_k_s

Commits

Commits on Jan 17, 2024