303 Commits

Author	SHA1	Message	Date
Olivier Chafik	db4bf93812	Merge remote-tracking branch 'origin/master' into tool-call	2024-10-22 14:37:30 +01:00
Molly Sophia	4ff7fe1fb3	llama : add chat template for RWKV-World + fix EOT (#9968) ... * Add chat template for RWKV-World Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * RWKV: Fix the chat template not being used Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * RWKV v6: Set EOT token to ``\n\n`` Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * readme: add rwkv into supported model list Signed-off-by: Molly Sophia <mollysophia379@gmail.com> --------- Signed-off-by: Molly Sophia <mollysophia379@gmail.com>	2024-10-22 13:33:37 +03:00
ochafik	b53362a148	Update test-tool-call.cpp	2024-10-22 10:54:48 +01:00
Georgi Gerganov	55e47786e3	llama : default sampling changes + greedy update (#9897) ... * llama : deprecate softmax sampler + fix dist sampler ggml-ci * tests : replace macros with functions ggml-ci * sampling : change temperature sampler logic For t <= 0.0f, keep the max logit intact and set the rest to -inf * cont : no need for special "greedy" logic top-k == 1 is the same * tests : init prob correctly * llama : handle temp <= 0.0 in the temp_ext sampler too ggml-ci * cont : avoid extra loop in temperature sampler for sub-zero temp ggml-ci	2024-10-21 09:46:40 +03:00
Joe Eli McIlvain	66c2c93082	grammar : fix JSON Schema for string regex with top-level alt. (#9903) ... Prior to this commit, using a JSON Schema containing a string with `pattern` regular expression that uses top-level alternation (e.g. `"pattern": "^A|B|C|D$"`) would result in invalid JSON output from the constrained sampling grammar, because it ended up creating a grammar rule like this for the string: ``` thing ::= "\"" "A" | "B" | "C" | "D" "\"" space ``` Note that this rule will only match a starting quote for the "A" case, and will only match an ending quote for the "D" case, so this rule will always produce invalid JSON when used for sampling (that is, the JSON will always be lacking the starting quote, the ending quote, or both). This was fixed in a simple way by adding parentheses to the generated rule (for all string pattern rules, to keep it simple), such that the new generated rule looks like this (correct): ``` thing ::= "\"" ("A" | "B" | "C" | "D") "\"" space ```	2024-10-16 19:03:24 +03:00
MaggotHATE	fbc98b748e	sampling : add XTC sampler (#9742) ... * Initial XTC commit Adds XTC sampler, not activated by default, but recommended settings by default. * Cleanup * Simplified chances calculation To be more inline with the original implementation, chance is calculated once at the beginning. * First fixes by comments Still need to look into sorting * Fixed trailing backspaces * Fixed RNG to be reproduceable Thanks to @slaren for directions * Fixed forgotten header * Moved `min_keep` Moved from conditions to a simple check at the end. * Fixed broken randomization Thanks to @slaren for explanation * Swapped sorting for a custom algorithm Shifts tokens to remove the penalized ones, then puts the penalized at the back. Should make `min_keep` still viable. * Algorithm rework 1. Scan token from top till the first non-penalizable 2. Remove the last captured token (the least probable above threshold) 3. Shift all tokens to override the remaining penalizable 4. Penalize and put them at the the bottom. * Added XTC to `test-sampling` * Simplified algorithm and more tests * Updated info in common and args * Merged back lost commits in common and arg * Update dump info in common * Fixed incorrect min_keep check * Added XTC to README * Renamed parameters, fixed info and defaults * probability is at 0 by default, but XTC is included in sampling queue * threshold higher than 0.5 switches XTC off * Initial server support * Added XTC to server UIs * Fixed labels in old server UI * Made algorithm safer and more readable * Removed xtc_threshold_max * Fixed arg after update * Quick fixes by comments * Simplified algorithm since threshold_max is removed * Renamed random distribution * Fixed tests and outdated README * Small fixes	2024-10-15 12:54:55 +02:00
Diego Devesa	7eee341bee	common : use common_ prefix for common library functions (#9805) ... * common : use common_ prefix for common library functions --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2024-10-10 22:57:42 +02:00
Diego Devesa	dca1d4b58a	ggml : fix BLAS with unsupported types (#9775) ... * ggml : do not use BLAS with types without to_float * ggml : return pointer from ggml_internal_get_type_traits to avoid unnecessary copies * ggml : rename ggml_internal_get_type_traits -> ggml_get_type_traits it's not really internal if everybody uses it	2024-10-08 14:21:43 +02:00
Diego Devesa	6374743747	ggml : add backend registry / device interfaces to BLAS backend (#9752) ... * ggml : add backend registry / device interfaces to BLAS backend * fix mmap usage when using host buffers	2024-10-07 21:55:08 +02:00
ochafik	332506910f	tool-call: accept {"type": "function", "name": "fn" for llama 3.x	2024-10-07 02:23:37 +01:00
Olivier Chafik	366efc8a18	tool-call: fix llama 3.x tc parsing when there are spaces before "name"	2024-10-03 21:46:41 +01:00
Olivier Chafik	ece12b074f	antiprompts: ensure partial match is at end of string (or else server stops sending replies)	2024-10-03 19:23:08 +01:00
Johannes Gäßler	fabdc3bda3	ggml/ex: calculate accuracy in graph, adapt MNIST (ggml/980)	2024-10-03 21:17:26 +03:00
Diego Devesa	c83ad6d01e	ggml-backend : add device and backend reg interfaces (#9707) ... Co-authored-by: Johannes Gäßler <johannesg@5d6.de>	2024-10-03 01:49:47 +02:00
Olivier Chafik	9e502e89a5	tool-call: promote getting chat templates w/ dedicated script rather than rely on test resources	2024-10-02 15:03:08 +01:00
Olivier Chafik	c76b14501e	tool-call: fix Makefile	2024-10-02 00:06:42 +01:00
Olivier Chafik	c36a196f53	tool-call: prepare possible externalization of minja + factor tool call style out of template	2024-10-01 23:12:24 +01:00
Johannes Gäßler	e98c1c188e	test: fix OPT_STEP_ADAMW for test-backend-ops (ggml/974)	2024-10-01 16:07:40 +03:00
Johannes Gäßler	7254cdf7e8	ggml: fix gradient allocation logic (ggml/966) ... * ggml: fix gradient allocation logic * gradient allocation in ggml_build_backward_expand * fixup * fix test-backend-ops grad * suggestions by slaren * fix test1.c * fix legacy opt API * fix test-grad0 * remove keep arg	2024-10-01 16:07:38 +03:00
ochafik	d9451fd647	antiprompts: avoid c++20 struct initializers in test	2024-09-30 04:08:55 +01:00
ochafik	0fc5ad7ae1	minja: avoid c++20 struct initializers in test	2024-09-30 03:51:48 +01:00
ochafik	9ac4b04aa2	tool-call: add fs_list_files to common, w/ win32 impl for msys2 build	2024-09-29 00:42:52 +01:00
ochafik	cb7912ee74	chat-template: add phi-3.5-vision-instruct	2024-09-29 00:33:19 +01:00
ochafik	c87c12168a	tool-call: fix memory leak in test	2024-09-28 23:44:28 +01:00
ochafik	22493c8e9e	tests: fix test-chat-template run from build	2024-09-28 23:31:23 +01:00
ochafik	ad6719e2a7	tests: fix typo	2024-09-28 23:26:19 +01:00
ochafik	a072f30a8d	tests: attempt to find assets for tests run from build subfolder	2024-09-28 23:15:36 +01:00
ochafik	bc3e0c0830	tool-call: Qwen 2.5 Instruct also requires object arguments	2024-09-28 23:05:35 +01:00
ochafik	dbda025f87	tool-call: test messages -> template -> grammar -> tool call parser	2024-09-28 22:32:47 +01:00
ochafik	9358d1f62c	minja: fix gcc8 build of test	2024-09-28 19:50:08 +01:00
ochafik	6e0053a81b	chat-template: enumerate files w/ C API rather than private using std::__fs::filesystem	2024-09-28 18:47:11 +01:00
ochafik	c657857e21	tool-call: cleanup tools.py	2024-09-28 18:33:40 +01:00
ochafik	d983516f40	tool-call: let the tool call handler expand chat template, moving builtin_tools down as extra_context	2024-09-28 17:46:36 +01:00
ochafik	0c85bc7a8f	tool-call: test tool call style detection	2024-09-28 17:43:09 +01:00
slaren	1b2f992cd2	test-backend-ops : use flops for some performance tests (#9657) ... * test-backend-ops : use flops for some performance tests - parallelize tensor quantization - use a different set of cases for performance and correctness tests - run each test for at least one second	2024-09-28 14:32:46 +02:00
Zhenwei Jin	6102037bbb	vocab : refactor tokenizer to reduce init overhead (#9449) ... * refactor tokenizer * llama : make llm_tokenizer more private ggml-ci * refactor tokenizer * refactor tokenizer * llama : make llm_tokenizer more private ggml-ci * remove unused files * remove unused fileds to avoid unused filed build error * avoid symbol link error * Update src/llama.cpp * Update src/llama.cpp --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2024-09-28 15:10:58 +03:00
ochafik	887951beb0	minja: generate chat goldens w/ fixed date to support Llama-3.2-3B-Instruct (uses strftime_now)	2024-09-27 19:52:15 +01:00
ochafik	701b664551	minja: add indent filter to support command-r-plus's chat templates	2024-09-27 19:00:14 +01:00
ochafik	0093a5e527	minja: fix identifiers parsing (when start w/ not/is/etc) and lstrip_blocks corner case (needed by DeepSeek-V2.5	2024-09-27 18:30:44 +01:00
ochafik	1e5c0e747e	chat-template: fix jinja tests (make safe a passthrough)	2024-09-27 03:50:04 +01:00
ochafik	f9c1743bb5	minja: fix iterables	2024-09-27 03:36:49 +01:00
ochafik	10f9fe8d49	tool-call: fix tool call return format	2024-09-26 21:01:04 +01:00
ochafik	2926089c5d	fix lints	2024-09-26 19:06:29 +01:00
ochafik	5840e10069	tool-call: merge & fix jinja template tests into test-chat-template	2024-09-26 19:05:00 +01:00
ochafik	50685f837f	minja: add str.title()	2024-09-26 19:03:59 +01:00
ochafik	296331bba3	minja: update chat template goldens w/ llama.3.1 arguments workaround	2024-09-26 18:10:27 +01:00
ochafik	cf7bece6a7	tool-call: factor chat template away from legacy API	2024-09-26 17:19:29 +01:00
ochafik	0c870133d8	tool-call: test/fix functionary-medium-v3.1's template (can "look" like llama3.1 template)	2024-09-26 05:56:15 +01:00
ochafik	8e4a9bad8a	minja: allow none input to selectattr, and add safe passthrough filter	2024-09-26 05:53:12 +01:00
ochafik	2eb29bf8b8	tool-call: update chat templates/goldens	2024-09-26 04:00:10 +01:00