3744 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Georgi Gerganov	b467702b87	readme : fix typos	2023-03-29 19:38:31 +03:00
Georgi Gerganov	516d88e75c	readme : add GPT4All instructions (close #588)	2023-03-29 19:37:20 +03:00
Georgi Gerganov	53635c081c	py : add GPT4All conversion script ... For now: copy-paste Too much time for me to deduplicate the python code	2023-03-29 19:29:52 +03:00
Maël Kerbiriou	41318d708e	llama : use the same threshold for OpenBLAS and ggml thread limiting (#577)	2023-03-29 19:10:07 +03:00
Tobias Lütke	a6956b25a1	add example of re-act pattern (#583) ... * add example of re-act pattern * spelling... * fixed whitespace in reverse prompt issue	2023-03-29 10:10:24 -05:00
anzz1	83df5639eb	Fix GCC warning about binary literal (#595) ... 0b10101010 -> 0xAA /* 0b10101010 */	2023-03-29 13:20:07 +00:00
anzz1	a5c42c4b13	Fix typo in llama.h (#593)	2023-03-29 13:19:29 +00:00
anzz1	5a5f8b1501	Enable Fused-Multiply-Add (FMA) and F16C/CVT16 vector extensions on MSVC (#375) ... * Enable Fused-Multiply-Add (FMA) instructions on MSVC __FMA__ macro does not exist in MSVC * Enable F16C/CVT16 vector extensions on MSVC __F16C__ macro does not exist in MSVC, but is implied with AVX2/AVX512 * MSVC cvt intrinsics * Add __SSE3__ macro for MSVC too because why not even though it's not currently used for anything when AVX is defined	2023-03-28 22:44:29 +03:00
anzz1	f1217055ea	CI: fix subdirectory path globbing (#546) ... - Changes in subdirectories will now be detecter properly - (Windows-MSVC) AVX512 tests temporarily disabled	2023-03-28 22:43:25 +03:00
anzz1	7f4c5c6651	llama : fix linkage with mingw (#551) ... * Revert 7e53955 (#542) Still needs to be fixed properly * Fix linking on mingw32	2023-03-28 21:23:09 +03:00
slaren	2a98bc18ea	ggml : add AVX2 implementation of quantize_row_q4_1 (#515) ... * Add AVX2 implementation of quantize_row_q4_1 * Actually use AVX2 * Make quantize_row_q4_1 static Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2023-03-28 21:06:03 +03:00
thement	d0aaff571c	py : add temporary script to convert old ggml files to newer version (#539) ... Co-authored-by: Jakub Horak <jakub.horak@ibawizard.net>	2023-03-28 20:55:42 +03:00
Tai Duc Nguyen	d0330fd783	py : add capabiliy to convert from ggml back to torch or hf format for further consumption/training/finetuning (#403)	2023-03-28 20:51:29 +03:00
Stephan Walter	99c5b27654	ggml : refactor quantized processing functions (#509) ... * Refactor quantized processing functions * ggml : minor --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2023-03-28 20:13:01 +03:00
DooWoong Lee (David)	692ce3164e	py : removed unused model variable and verified that the code functions correctly with vocab_only setting. Also confirmed that the code works as expected after running with reduced memory usage due to deletion of no-longer-needed variable. (#547)	2023-03-28 20:02:34 +03:00
Georgi Gerganov	96f9c0506f	ci : make ctest verbose, hopefully we see what is wrong with the sanitizer	2023-03-28 20:01:09 +03:00
Georgi Gerganov	d502bc7c9d	tests : free llama context at the end of the test	2023-03-28 19:51:55 +03:00
Stephan Walter	436e561931	all : be more strict about converting float to double (#458) ... * Be more strict about converting float to double * Test equivalence of round, SILU implementations Test module is commented out in CMakeLists.txt because the tests may take a long time, depending on how much the compiler optimizes. * Fix softmax in perplexity.cpp * all : prefer float over double where appropriate * perplexity : add <cmath> --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2023-03-28 19:48:20 +03:00
Jed Fox	20e1e84884	deploy : add a Package.swift for SwiftPM support (#393) ... * Add a Package.swift for SwiftPM support * Swap from exclusions to allowlist	2023-03-28 19:39:01 +03:00
Stephan Walter	c1f885067c	ggml : introduce structs for the q4 data blocks (#356) ... * Introduce structs for the q4 data blocks * ggml : rename quant struct variables + fix ARM_NEON --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2023-03-28 18:56:03 +03:00
Georgi Gerganov	e0670260fb	gitignore : add "embedding"	2023-03-28 18:34:35 +03:00
dotpy314	28ba975aea	Check the existence of f16_model_path_base in quantize.py (#574) ... Co-authored-by: Jincheng Miao <jincheng.miao@gmail.com>	2023-03-28 18:06:28 +03:00
slaren	a6bdc47cba	Fix usage of F16C intrinsics in AVX code (#563) ... * Fix usage of F16C intrinsics in AVX code when F16C is not defined	2023-03-28 17:26:55 +03:00
anzz1	7b8dbcb78b	main.cpp fixes, refactoring (#571) ... - main: entering empty line passes back control without new input in interactive/instruct modes - instruct mode: keep prompt fix - instruct mode: duplicate instruct prompt fix - refactor: move common console code from main->common	2023-03-28 17:09:55 +03:00
RJ Adriaansen	4b8efff0e3	Add embedding example to Makefile (#540)	2023-03-28 09:11:09 +03:00
Marco Matthies	7e5395575a	Fix missing ggml link in cmake for examples/* on w64-mingw32 (#542)	2023-03-27 07:55:26 +03:00
Erik Scholz	34c1072e49	ci: add debug build to sanitizer build matrix (#527)	2023-03-26 15:48:40 +00:00
Stephan Walter	939ad2d3a5	Fix undefined variables in debug build, remove unused variables (#531)	2023-03-26 15:34:02 +00:00
Juan Calderon-Perez	8c2ec5e21d	Add support for linux/arm64 platform during Docker Builds (#514) ... * Add support for linux/arm64 platform * Add platform to versioned builds	2023-03-26 14:48:42 +00:00
Stephan Walter	b391579db9	Update README and comments for standalone perplexity tool (#525)	2023-03-26 16:14:01 +03:00
anzz1	7a87d31f4f	[main] fix infinite generation (-n == -1) (#523)	2023-03-26 16:06:10 +03:00
Georgi Gerganov	348d6926ee	Add logo to README.md	2023-03-26 10:20:49 +03:00
Harald Fernengel	33e35b8fe8	Exit from interactive mode if input stream is bad (#491) ... Allow exiting the interactive prompt also with CTRL-D on Unix and CTRL-Z on Windows.	2023-03-26 08:25:46 +03:00
anzz1	19726169b3	CI: Run other sanitizer builds even if one fails (#511) ... applies only to sanitizer builds so they wont be cancelled	2023-03-26 00:13:28 +02:00
jp-x-g	f732695cd5	Clarify console output in convert-pth-to-ggml.py (#512) ... "Processing part 1 of 3" instead of "Processing part 0"	2023-03-25 23:53:55 +02:00
anzz1	2f7bf7dd7c	CMake / CI additions (#497) ... * CMake: Add AVX512 option * CI: Add AVX/AVX512 builds (Windows) (AVX512 tests can only be run when the worker happens to support it, building works anyway) * CMake: Fix sanitizer linkage ( merged #468 ) * CI: Add sanitizer builds (Ubuntu) * CI: Fix release tagging (change @zendesk/action-create-release to @anzz1/action-create-release until upstream PR Added commitish as input zendesk/action-create-release#32 is merged)	2023-03-25 23:38:11 +02:00
anzz1	34ab526843	(Windows) Set console to UTF-8 on init (#420) ... Sets console codepage to 65001 (CP_UTF8) on start for both input and output, should fix problems with UTF-8 characters.	2023-03-25 22:29:22 +02:00
Georgi Gerganov	c2b25b6912	Fix colors enabling on WIN32	2023-03-25 21:53:39 +02:00
Georgi Gerganov	79b2b266db	If n_predict == -1, generate forever	2023-03-25 21:51:41 +02:00
Georgi Gerganov	e2d490dafd	Inifinite generation via context swapping (#71)	2023-03-25 21:36:22 +02:00
Georgi Gerganov	03f7e33560	Cleanup STL headers + fix embedding examples + minor stuff	2023-03-25 20:51:14 +02:00
Georgi Gerganov	55ad42af84	Move chat scripts into "./examples"	2023-03-25 20:37:09 +02:00
slaren	459e93cce0	Add AVX2 implementation of dequantize_row_q4_1 (#505)	2023-03-25 20:31:48 +02:00
Georgi Gerganov	a316a425d0	Overhaul the examples structure ... - main -> examples - utils -> examples (renamed to "common") - quantize -> examples - separate tools for "perplexity" and "embedding" Hope I didn't break something !	2023-03-25 20:26:40 +02:00
Georgi Gerganov	ecbe466a36	Retire the ggml_mul_mat() branch for transposed src0 (#500) ... * Retire the ggml_mul_mat() for transposed src0 - It can always be made contiguous with ggml_cpy() - The code is now simplified - The results are deterministic in respect to num threads * SIMD-ify dequantize_row_q4_0() for ARM_NEON (#502) * Attempt to SIMD-ify dequantize_row_q4_0() for ARM_NEON * Fix dequantization - forgot to interleave the quants	2023-03-25 19:47:21 +02:00
Georgi Gerganov	502a400192	Disable prompt verbosity by default and add option to enable (#480)	2023-03-25 17:17:16 +02:00
slaren	09aecbf628	Add AVX2 implementation of dequantize_row_q4_0 (#467)	2023-03-25 17:06:49 +02:00
Georgi Gerganov	4640eff23d	Don't interefe with BLAS for large prompts by running only 1 thread	2023-03-25 17:03:10 +02:00
Georgi Gerganov	ab77d76312	Add longer DAN prompt for testing big batch numbers	2023-03-25 16:49:09 +02:00
slaren	29b7baab67	Add timings for the prompt evaluation (#478)	2023-03-25 16:34:23 +02:00