7 Commits

Author	SHA1	Message	Date
Georgi Gerganov	013457885a	grammar : remove one nested level	2023-09-04 15:48:38 +03:00
Georgi Gerganov	2d89da4f77	grammar : add comments to new grammar file	2023-09-04 15:48:38 +03:00
Georgi Gerganov	e0a8658e7c	grammars : add json_arr.gbnf	2023-09-04 15:48:38 +03:00
Ben Siraphob	d8d6977f48	examples : add C grammar (#2357)	2023-09-01 16:32:14 +03:00
Evan Jones	f5fe98d11b	docs : add grammar docs (#2701) ... * docs : add grammar docs * tweaks to grammar guide * rework GBNF example to be a commented grammar	2023-08-22 21:01:57 -04:00
Evan Jones	8183159cf3	examples : generate JSON according to schema (#1887) ... * examples : add JSON schema grammars * complete JSON grammar * ensure primitive types can be used as root of schema * support integer type and adjust usage text	2023-08-02 22:05:44 -04:00
Evan Jones	84e09a7d8b	llama : add grammar-based sampling (#1773) ... * llama, main : constrain sampling to grammar * allow loading grammar from file * fix whitespace errors * handle & print parser errors * add comments to grammar syntax and allow newlines where unambiguous * add missing include * support alternates in root rule * fix bugs with empty token and EOS * adjust JSON grammar * remove swp file * rewrite ternary expressions Co-authored-by: Henri Vasserman <henv@hot.ee> * use struct for grammar elements and add Unicode support * add unicode escapes * add inverse char ranges * only sample full tokens (no peeking or truncation) * llama : minor style changes blindly applied in online editor - hopefully I didn't break something * update help text * add warning message if EOS is disabled --------- Co-authored-by: Henri Vasserman <henv@hot.ee> Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2023-07-23 23:58:10 -04:00