Kawrakow 38de86a711 llama : multi-threaded quantization (#1075) ... * Multi-threading quantization. Not much gain for simple quantizations, bit it will be important for quantizations that require more CPU cycles. * Multi-threading for quantize-stats It now does the job in ~14 seconds on my Mac for Q4_0, Q4_1 and Q4_2. Single-threaded it was taking more than 2 minutes after adding the more elaborate version of Q4_2. * Reviewer comments * Avoiding compiler confusion After changing chunk_size to const int as suggested by @ggerganov, clang and GCC starting to warn me that I don't need to capture it in the lambda. So, I removed it from the capture list. But that makes the MSVC build fail. So, making it a constexpr to make every compiler happy. * Still fighting with lambda captures in MSVC --------- Co-authored-by: Iwan Kawrakow <iwan.kawrakow@gmail.com> Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>		2023-04-20 20:42:27 +03:00
..
CMakeLists.txt	Add quantize-stats command for testing quantization (#728)	2023-04-08 00:09:18 +02:00
quantize-stats.cpp	llama : multi-threaded quantization (#1075)	2023-04-20 20:42:27 +03:00