321 Commits

Author	SHA1	Message	Date
ochafik	b35aa4ae1c	tool-call: add LLAMA_UPDATE_GOLDENS env for test-chat-template	2024-10-31 13:53:33 +00:00
ochafik	f5b7825595	tool-call: code_interpreter & system + tool call support for all jinja templates!	2024-10-31 13:52:46 +00:00
ochafik	7d9c90f46b	tool-call: nemo tweak (accept raw sql again)	2024-10-31 04:39:40 +00:00
ochafik	61655b9cdd	Merge remote-tracking branch 'origin/master' into tool-call	2024-10-31 01:45:07 +00:00
Olivier Chafik	e4d5449638	tool-calls: test Qwen2.5-7B-Instruct-Q4_K_M.gguf	2024-10-30 21:40:15 +00:00
Olivier Chafik	64287a328d	tool-call: test Hermes-3-Llama-3.1-8B	2024-10-29 14:52:25 +00:00
Georgi Gerganov	8d8ff71536	llama : remove Tail-Free sampling (#10071) ... ggml-ci	2024-10-29 10:42:05 +02:00
ochafik	b51c71c734	tool-call: remove duplicate script to fetch templates	2024-10-28 21:35:18 +00:00
arch-btw	61715d5cc8	llama : Add IBM granite template (#10013) ... * Add granite template to llama.cpp * Add granite template to test-chat-template.cpp * Update src/llama.cpp Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com> * Update tests/test-chat-template.cpp Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com> * Added proper template and expected output * Small change to \n Small change to \n * Add code space & Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com> * Fix spacing * Apply suggestions from code review * Update src/llama.cpp --------- Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>	2024-10-28 18:45:33 +01:00
ochafik	ec9f3b101b	nits	2024-10-27 16:44:54 +00:00
wwoodsTM	ff252ea48e	llama : add DRY sampler (#9702) ... * sampling : add DRY sampler (post-refactor) * DRY: Trying to fix coauthors, removed unneeded line * DRY: Fixed redundant code * DRY: Fixed crash issue due to DRY being in chain but uninitialized --------- Co-authored-by: l3utterfly <gc.pthzfoldr@gmail.com> Co-authored-by: pi6am <34464159+pi6am@users.noreply.github.com>	2024-10-25 19:07:34 +03:00
Johannes Gäßler	c39665f589	CUDA: fix MMQ for non-contiguous src0, add tests (#10021) ... * CUDA: fix MMQ for non-contiguous src0, add tests * revise test code	2024-10-24 11:09:36 +02:00
Johannes Gäßler	80273a306d	CUDA: fix 1D im2col, add tests (ggml/993)	2024-10-23 16:50:02 +03:00
Jun Hee Yoo	4c9388fb96	metal : add POOL2D and fix IM2COL (#9943) ... * add pool_2d Signed-off-by: Junhee Yoo <junhee.yoo@navercorp.com> * fix im2col and add unittest for N>=1024 Signed-off-by: Junhee Yoo <junhee.yoo@navercorp.com> * add tests for N % 1024 != 0 Signed-off-by: Junhee Yoo <junhee.yoo@navercorp.com> * remove trailing whitespaces Signed-off-by: Junhee Yoo <junhee.yoo@navercorp.com> * apply suggestions Signed-off-by: Junhee Yoo <junhee.yoo@navercorp.com> * apply more optimization - original IM2COL kernel + _ext with MIN() Signed-off-by: Junhee Yoo <junhee.yoo@navercorp.com> * apply review: change kernel name of pool_2d Signed-off-by: Junhee Yoo <junhee.yoo@navercorp.com> * apply review Signed-off-by: Junhee Yoo <junhee.yoo@navercorp.com> * fix more formatting and enhance readability Signed-off-by: Junhee Yoo <junhee.yoo@navercorp.com> --------- Signed-off-by: Junhee Yoo <junhee.yoo@navercorp.com>	2024-10-23 13:33:45 +03:00
Olivier Chafik	5f4aef10ba	Merge remote-tracking branch 'origin/master' into tool-call	2024-10-23 11:28:28 +01:00
ochafik	3e12b9b38e	tool-calls: basic Nemo support, default parallel to true if template mentions tool_call_id	2024-10-23 02:30:31 +01:00
ochafik	a4f12a4594	minja: fix string subscripts, add string pipe to support Mistral-Nemo template	2024-10-22 23:39:46 +01:00
Olivier Chafik	db4bf93812	Merge remote-tracking branch 'origin/master' into tool-call	2024-10-22 14:37:30 +01:00
Molly Sophia	11d47057a5	Rwkv chat template fix (#10001) ... * llama: remove useless template matching for rwkv-world Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * converter: Add comment about the hack for rwkv models Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * Update src/llama.cpp Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com> --------- Signed-off-by: Molly Sophia <mollysophia379@gmail.com> Co-authored-by: Xuan Son Nguyen <thichthat@gmail.com>	2024-10-22 15:22:26 +02:00
Molly Sophia	4ff7fe1fb3	llama : add chat template for RWKV-World + fix EOT (#9968) ... * Add chat template for RWKV-World Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * RWKV: Fix the chat template not being used Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * RWKV v6: Set EOT token to ``\n\n`` Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * readme: add rwkv into supported model list Signed-off-by: Molly Sophia <mollysophia379@gmail.com> --------- Signed-off-by: Molly Sophia <mollysophia379@gmail.com>	2024-10-22 13:33:37 +03:00
ochafik	b53362a148	Update test-tool-call.cpp	2024-10-22 10:54:48 +01:00
Georgi Gerganov	55e47786e3	llama : default sampling changes + greedy update (#9897) ... * llama : deprecate softmax sampler + fix dist sampler ggml-ci * tests : replace macros with functions ggml-ci * sampling : change temperature sampler logic For t <= 0.0f, keep the max logit intact and set the rest to -inf * cont : no need for special "greedy" logic top-k == 1 is the same * tests : init prob correctly * llama : handle temp <= 0.0 in the temp_ext sampler too ggml-ci * cont : avoid extra loop in temperature sampler for sub-zero temp ggml-ci	2024-10-21 09:46:40 +03:00
Joe Eli McIlvain	66c2c93082	grammar : fix JSON Schema for string regex with top-level alt. (#9903) ... Prior to this commit, using a JSON Schema containing a string with `pattern` regular expression that uses top-level alternation (e.g. `"pattern": "^A|B|C|D$"`) would result in invalid JSON output from the constrained sampling grammar, because it ended up creating a grammar rule like this for the string: ``` thing ::= "\"" "A" | "B" | "C" | "D" "\"" space ``` Note that this rule will only match a starting quote for the "A" case, and will only match an ending quote for the "D" case, so this rule will always produce invalid JSON when used for sampling (that is, the JSON will always be lacking the starting quote, the ending quote, or both). This was fixed in a simple way by adding parentheses to the generated rule (for all string pattern rules, to keep it simple), such that the new generated rule looks like this (correct): ``` thing ::= "\"" ("A" | "B" | "C" | "D") "\"" space ```	2024-10-16 19:03:24 +03:00
MaggotHATE	fbc98b748e	sampling : add XTC sampler (#9742) ... * Initial XTC commit Adds XTC sampler, not activated by default, but recommended settings by default. * Cleanup * Simplified chances calculation To be more inline with the original implementation, chance is calculated once at the beginning. * First fixes by comments Still need to look into sorting * Fixed trailing backspaces * Fixed RNG to be reproduceable Thanks to @slaren for directions * Fixed forgotten header * Moved `min_keep` Moved from conditions to a simple check at the end. * Fixed broken randomization Thanks to @slaren for explanation * Swapped sorting for a custom algorithm Shifts tokens to remove the penalized ones, then puts the penalized at the back. Should make `min_keep` still viable. * Algorithm rework 1. Scan token from top till the first non-penalizable 2. Remove the last captured token (the least probable above threshold) 3. Shift all tokens to override the remaining penalizable 4. Penalize and put them at the the bottom. * Added XTC to `test-sampling` * Simplified algorithm and more tests * Updated info in common and args * Merged back lost commits in common and arg * Update dump info in common * Fixed incorrect min_keep check * Added XTC to README * Renamed parameters, fixed info and defaults * probability is at 0 by default, but XTC is included in sampling queue * threshold higher than 0.5 switches XTC off * Initial server support * Added XTC to server UIs * Fixed labels in old server UI * Made algorithm safer and more readable * Removed xtc_threshold_max * Fixed arg after update * Quick fixes by comments * Simplified algorithm since threshold_max is removed * Renamed random distribution * Fixed tests and outdated README * Small fixes	2024-10-15 12:54:55 +02:00
Diego Devesa	7eee341bee	common : use common_ prefix for common library functions (#9805) ... * common : use common_ prefix for common library functions --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2024-10-10 22:57:42 +02:00
Diego Devesa	dca1d4b58a	ggml : fix BLAS with unsupported types (#9775) ... * ggml : do not use BLAS with types without to_float * ggml : return pointer from ggml_internal_get_type_traits to avoid unnecessary copies * ggml : rename ggml_internal_get_type_traits -> ggml_get_type_traits it's not really internal if everybody uses it	2024-10-08 14:21:43 +02:00
Diego Devesa	6374743747	ggml : add backend registry / device interfaces to BLAS backend (#9752) ... * ggml : add backend registry / device interfaces to BLAS backend * fix mmap usage when using host buffers	2024-10-07 21:55:08 +02:00
ochafik	332506910f	tool-call: accept {"type": "function", "name": "fn" for llama 3.x	2024-10-07 02:23:37 +01:00
Olivier Chafik	366efc8a18	tool-call: fix llama 3.x tc parsing when there are spaces before "name"	2024-10-03 21:46:41 +01:00
Olivier Chafik	ece12b074f	antiprompts: ensure partial match is at end of string (or else server stops sending replies)	2024-10-03 19:23:08 +01:00
Johannes Gäßler	fabdc3bda3	ggml/ex: calculate accuracy in graph, adapt MNIST (ggml/980)	2024-10-03 21:17:26 +03:00
Diego Devesa	c83ad6d01e	ggml-backend : add device and backend reg interfaces (#9707) ... Co-authored-by: Johannes Gäßler <johannesg@5d6.de>	2024-10-03 01:49:47 +02:00
Olivier Chafik	9e502e89a5	tool-call: promote getting chat templates w/ dedicated script rather than rely on test resources	2024-10-02 15:03:08 +01:00
Olivier Chafik	c76b14501e	tool-call: fix Makefile	2024-10-02 00:06:42 +01:00
Olivier Chafik	c36a196f53	tool-call: prepare possible externalization of minja + factor tool call style out of template	2024-10-01 23:12:24 +01:00
Johannes Gäßler	e98c1c188e	test: fix OPT_STEP_ADAMW for test-backend-ops (ggml/974)	2024-10-01 16:07:40 +03:00
Johannes Gäßler	7254cdf7e8	ggml: fix gradient allocation logic (ggml/966) ... * ggml: fix gradient allocation logic * gradient allocation in ggml_build_backward_expand * fixup * fix test-backend-ops grad * suggestions by slaren * fix test1.c * fix legacy opt API * fix test-grad0 * remove keep arg	2024-10-01 16:07:38 +03:00
ochafik	d9451fd647	antiprompts: avoid c++20 struct initializers in test	2024-09-30 04:08:55 +01:00
ochafik	0fc5ad7ae1	minja: avoid c++20 struct initializers in test	2024-09-30 03:51:48 +01:00
ochafik	9ac4b04aa2	tool-call: add fs_list_files to common, w/ win32 impl for msys2 build	2024-09-29 00:42:52 +01:00
ochafik	cb7912ee74	chat-template: add phi-3.5-vision-instruct	2024-09-29 00:33:19 +01:00
ochafik	c87c12168a	tool-call: fix memory leak in test	2024-09-28 23:44:28 +01:00
ochafik	22493c8e9e	tests: fix test-chat-template run from build	2024-09-28 23:31:23 +01:00
ochafik	ad6719e2a7	tests: fix typo	2024-09-28 23:26:19 +01:00
ochafik	a072f30a8d	tests: attempt to find assets for tests run from build subfolder	2024-09-28 23:15:36 +01:00
ochafik	bc3e0c0830	tool-call: Qwen 2.5 Instruct also requires object arguments	2024-09-28 23:05:35 +01:00
ochafik	dbda025f87	tool-call: test messages -> template -> grammar -> tool call parser	2024-09-28 22:32:47 +01:00
ochafik	9358d1f62c	minja: fix gcc8 build of test	2024-09-28 19:50:08 +01:00
ochafik	6e0053a81b	chat-template: enumerate files w/ C API rather than private using std::__fs::filesystem	2024-09-28 18:47:11 +01:00
ochafik	c657857e21	tool-call: cleanup tools.py	2024-09-28 18:33:40 +01:00