Johannes Gäßler
|
3b099bcd9c
|
CUDA: fix MMQ writeback for int8 tensor cores (#8100)
|
2024-06-24 22:15:33 +02:00 |
|
Johannes Gäßler
|
a818f3028d
|
CUDA: use MMQ instead of cuBLAS by default (#8075)
|
2024-06-24 17:43:42 +02:00 |
|
Johannes Gäßler
|
9a590c8226
|
CUDA: optimize MMQ int8 tensor core performance (#8062)
* CUDA: optimize MMQ int8 tensor core performance
* only a single get_mma_tile_x_k function
* simplify code, make functions constexpr
|
2024-06-24 12:41:23 +02:00 |
|
Johannes Gäßler
|
d50f8897a7
|
CUDA: stream-k decomposition for MMQ (#8018)
* CUDA: stream-k decomposition for MMQ
* fix undefined memory reads for small matrices
|
2024-06-20 14:39:21 +02:00 |
|
Johannes Gäßler
|
76d66ee0be
|
CUDA: faster q2_K, q3_K MMQ + int8 tensor cores (#7921)
* CUDA: faster q2_K, q3_K MMQ + int8 tensor cores
* try CI fix
* try CI fix
* try CI fix
* fix data race
* rever q2_K precision related changes
|
2024-06-14 18:41:49 +02:00 |
|
Johannes Gäßler
|
bdcb8f4222
|
CUDA: int8 tensor cores for MMQ (q4_K, q5_K, q6_K) (#7860)
|
2024-06-11 08:26:07 +02:00 |
|
Johannes Gäßler
|
1f0dabda8d
|
CUDA: use tensor cores for MMQ (#7676)
* CUDA: int8 tensor cores for MMQ (legacy quants)
* fix out-of-bounds writes
* __builtin_assume -> GGML_CUDA_ASSUME
* fix writeback returning too early
|
2024-06-10 11:45:13 +02:00 |
|
Johannes Gäßler
|
42b53d192f
|
CUDA: revise q8_1 data layout for mul_mat_q (#7824)
|
2024-06-09 09:42:25 +02:00 |
|
Johannes Gäßler
|
7d1a378b8f
|
CUDA: refactor mmq, dmmv, mmvq (#7716)
* CUDA: refactor mmq, dmmv, mmvq
* fix out-of-bounds write
* struct for qk, qr, qi
* fix cmake build
* mmq_type_traits
|
2024-06-05 16:53:00 +02:00 |
|
slaren
|
ae1f211ce2
|
cuda : refactor into multiple files (#6269)
|
2024-03-25 13:50:23 +01:00 |
|