417 Commits

Author	SHA1	Message	Date
piDack	f91cf62b89	Merge branch 'master' into support_glm_edge_model	2024-12-19 16:50:23 +08:00
Eric Curtin	7909e8588d	llama-run : improve progress bar (#10821) ... Set default width to whatever the terminal is. Also fixed a small bug around default n_gpu_layers value. Signed-off-by: Eric Curtin <ecurtin@redhat.com>	2024-12-19 03:58:00 +01:00
redbeard	6b064c92b4	docs: Fix HIP (née hipBLAS) in README (#10880) ... Related to #10524 / be0e350c references to hipBLAS have been removed across the repository. This fixes the link from the repositories `README.md`. Signed-off-by: Brian 'redbeard' Harrington <redbeard@dead-city.org>	2024-12-18 10:35:00 +02:00
Ruan	4f51968aca	readme : update typos (#10863)	2024-12-17 11:47:20 +02:00
Valentin Mamedov	a0974156f3	llama : add Deepseek MoE v1 & GigaChat models (#10827) ... * Add deepseek v1 arch & gigachat template * improve template code * add readme * delete comments * remove comment * fix format * lint llama.cpp * fix order of deepseek and deepseek2, move gigachat temlate to the end of func * fix order of deepseek and deepseek2 in constants; mark shared exp as deepseek arch need * remove comments * move deepseek above deepseek2 * change placement of gigachat chat template	2024-12-15 19:02:46 +02:00
HimariO	ba1cb19cdd	llama : add Qwen2VL support + multimodal RoPE (#10361) ... * Barebone Qwen2VL LLM convertor * Add Qwen2VL cli entrypoint * [WIP] add qwen2vl arch * Verify m-rope output * Add vl-rope/2d-rope support for qwen2vl ViT * update qwen2vl cli tool * update 5D tensor op workaround * [WIP] qwen2vl vision model * make batch and clip utils compatible with qwen2vl * [WIP] create inference workflow, gguf convert script but fix * correcting vision-rope behavior, add the missing last layer back to ViT * add arg parser to qwen2vl_surgery * replace variable size array with vector * cuda-gdb cmake preset * add fp32 mrope, vision rope kernel * add fp16 support for qwen2vl and m-rope * add `GGML_ROPE_TYPE_MROPE`, `GGML_ROPE_TYPE_VISION` * fix rope op mode switching, out dated func args * update `llama_hparams` * update to keep up stream changes * resolve linter, test errors * add makefile entry, update speical image padding token * add mrope unit test, fix few compiler warnings * rename `mrope` related function, params * minor updates on debug util, bug fixs * add `m-rope` testcase to `test-backend-ops` * Apply suggestions from code review Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * fix traililng whitespce * store `llama_hparams.rope_sections` with fixed size array * update position id tensor size check in GGML_OP_ROPE * minor updates * update `ggml_backend_*_supports_op` of unsupported backends * remote old `rope_section` compare operator --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>	2024-12-14 14:43:46 +02:00
Eric Curtin	c27ac678dd	Opt class for positional argument handling (#10508) ... Added support for positional arguments `model` and `prompt`. Added functionality to download via strings like: llama-run llama3 llama-run ollama://granite-code llama-run ollama://granite-code:8b llama-run hf://QuantFactory/SmolLM-135M-GGUF/SmolLM-135M.Q2_K.gguf llama-run huggingface://bartowski/SmolLM-1.7B-Instruct-v0.2-GGUF/SmolLM-1.7B-Instruct-v0.2-IQ3_M.gguf llama-run https://example.com/some-file1.gguf llama-run some-file2.gguf llama-run file://some-file3.gguf Signed-off-by: Eric Curtin <ecurtin@redhat.com>	2024-12-13 19:34:25 +01:00
piDack	82cbfda7b9	Merge branch 'master' into support_glm_edge_model	2024-12-03 13:27:23 +08:00
Georgi Gerganov	6acce39710	readme : update the usage section with examples (#10596) ... * readme : update the usage section with examples * readme : more examples	2024-12-01 11:25:17 +02:00
Georgi Gerganov	3e0ba0e604	readme : remove old badge	2024-11-30 10:09:21 +02:00
Georgi Gerganov	abadba05be	readme : refresh (#10587) ... * readme : refresh * readme : move section [no ci] * readme : clarify [no ci] * readme : fixes [no ci] * readme : more fixes [no ci] * readme : simplify [no ci] * readme : clarify GGUF	2024-11-30 09:47:07 +02:00
Diego Devesa	a3a3048e7a	cleanup UI link list (#10577) ... * cleanup UI link list * sort list alphabetically * add missing licenses	2024-11-29 17:45:08 +01:00
piDack	816d93db75	Merge branch 'master' of https://github.com/piDack/llama.cpp into support_glm_edge_model	2024-11-29 08:11:31 +00:00
Shane A	de5097351c	Add OLMo 2 model in docs (#10530) ... * Add link to OLMo 2 model in docs * Change link to landing page	2024-11-26 21:55:29 +01:00
piDack	3b27041727	vision model support	2024-11-26 12:36:48 +00:00
Johannes Gäßler	467576b6cc	CMake: default to -arch=native for CUDA build (#10320)	2024-11-17 09:06:34 +01:00
Small Grass Forest	1ee9eea094	docs : update bindings list (#10261) ... Signed-off-by: tianzixuan <tianzixuan335@hellobike.com>	2024-11-13 13:17:10 +02:00
Georgi Gerganov	ba6f62eb79	readme : update hot topics	2024-11-01 17:31:51 +02:00
Molly Sophia	4ff7fe1fb3	llama : add chat template for RWKV-World + fix EOT (#9968) ... * Add chat template for RWKV-World Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * RWKV: Fix the chat template not being used Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * RWKV v6: Set EOT token to ``\n\n`` Signed-off-by: Molly Sophia <mollysophia379@gmail.com> * readme: add rwkv into supported model list Signed-off-by: Molly Sophia <mollysophia379@gmail.com> --------- Signed-off-by: Molly Sophia <mollysophia379@gmail.com>	2024-10-22 13:33:37 +03:00
Asghar Ghorbani	994cfb1acb	readme : update UI list (#9972) ... add PocketPal AI app	2024-10-21 21:20:59 +03:00
Loïc Carrère	45f097645e	readme : update bindings list (#9951) ... Update the binding list by adding LM-Kit.NET (C# & VB.NET)	2024-10-20 19:25:41 +03:00
icppWorld	7cab2083c7	readme : update infra list (#9942) ... llama_cpp_canister allows you to run llama.cpp as a Smart Contract on the Internet Computer. The smart contract runs as WebAssembly in a so-called 'canister'.	2024-10-20 19:01:34 +03:00
Ma Mingfei	60ce97c9d8	add amx kernel for gemm (#8998) ... add intel amx isa detection add vnni kernel for gemv cases add vnni and amx kernel support for block_q8_0 code cleanup fix packing B issue enable openmp fine tune amx kernel switch to aten parallel pattern add error message for nested parallelism code cleanup add f16 support in ggml-amx add amx kernels for QK_K quant formats: Q4_K, Q5_K, Q6_K and IQ4_XS update CMakeList update README fix some compilation warning fix compiler warning when amx is not enabled minor change ggml-ci move ggml_amx_init from ggml.c to ggml-amx/mmq.cpp ggml-ci update CMakeLists with -mamx-tile, -mamx-int8 and -mamx-bf16 ggml-ci add amx as an ggml-backend update header file, the old path for immintrin.h has changed to ggml-cpu-impl.h minor change update CMakeLists.txt minor change apply weight prepacking in set_tensor method in ggml-backend fix compile error ggml-ci minor change ggml-ci update CMakeLists.txt ggml-ci add march dependency minor change ggml-ci change ggml_backend_buffer_is_host to return false for amx backend ggml-ci fix supports_op use device reg for AMX backend ggml-ci minor change ggml-ci minor change fix rebase set .buffer_from_host_ptr to be false for AMX backend	2024-10-18 13:34:36 +08:00
Tim Wang	3752217ed5	readme : update bindings list (#9918) ... Co-authored-by: Tim Wang <tim.wang@ing.com>	2024-10-17 09:57:14 +03:00
Michał Tuszyński	4c42f93b22	readme : update bindings list (#9889)	2024-10-15 11:20:34 +03:00
R0CKSTAR	943d20b411	musa : update doc (#9856) ... Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>	2024-10-12 08:09:53 +03:00
Viet-Anh NGUYEN (Andrew)	71967c2a6d	Add Llama Assistant (#9744)	2024-10-04 20:29:35 +02:00
Paweł Wodnicki	3f1ae2e32c	Update README.md (#9591) ... Add Bielik model.	2024-10-01 19:18:46 +02:00
Georgi Gerganov	589b48d41e	contrib : add Resources section (#9675)	2024-09-29 14:38:18 +03:00
Aarni Koskela	43bcdd9703	readme : add tool (#9655)	2024-09-28 15:07:14 +03:00
Georgi Gerganov	b5de3b74a5	readme : update hot topics	2024-09-27 20:57:51 +03:00
Riceball LEE	1d48e98e4f	readme : add programmable prompt engine language CLI (#9599)	2024-09-23 18:58:17 +03:00
Shane A	0aadac10c7	llama : support OLMoE (#9462)	2024-09-16 09:47:37 +03:00
OSecret	d6b37c881f	readme : update tools list (#9475) ... * Added link to proprietary wrapper for Unity3d into README.md Wrapper has prebuild library and was tested on iOS, Android, WebGL, PC, Mac platforms, has online demos like [this](https://d23myu0xfn2ttc.cloudfront.net/rich/index.html) and [that](https://d23myu0xfn2ttc.cloudfront.net/). * Update README.md Fixes upon review	2024-09-15 10:36:53 +03:00
Faisal Zaghloul	449ccfb6f5	Add Jais to list of supported models (#9439) ... Co-authored-by: fmz <quic_fzaghlou@quic.com>	2024-09-12 02:29:53 +02:00
Georgi Gerganov	38ca6f644b	readme : update hot topics	2024-09-09 15:51:37 +03:00
Antonis Makropoulos	5ed087573e	readme : add LLMUnity to UI projects (#9381) ... * add LLMUnity to UI projects * add newline to examples/rpc/README.md to fix editorconfig-checker unit test	2024-09-09 14:21:38 +03:00
Georgi Gerganov	b69a480af4	readme : refactor API section + remove old hot topics	2024-09-03 10:00:36 +03:00
Younes Belkada	b40eb84895	llama : support for falcon-mamba architecture (#9074) ... * feat: initial support for llama.cpp * fix: lint * refactor: better refactor * Update src/llama.cpp Co-authored-by: compilade <git@compilade.net> * Update src/llama.cpp Co-authored-by: compilade <git@compilade.net> * fix: address comments * Update convert_hf_to_gguf.py Co-authored-by: compilade <git@compilade.net> * fix: add more cleanup and harmonization * fix: lint * Update gguf-py/gguf/gguf_writer.py Co-authored-by: compilade <git@compilade.net> * fix: change name * Apply suggestions from code review Co-authored-by: compilade <git@compilade.net> * add in operator * fix: add `dt_b_c_rms` in `llm_load_print_meta` * fix: correct printf format for bool * fix: correct print format * Update src/llama.cpp Co-authored-by: compilade <git@compilade.net> * llama : quantize more Mamba tensors * llama : use f16 as the fallback of fallback quant types --------- Co-authored-by: compilade <git@compilade.net>	2024-08-21 11:06:36 +03:00
wangshuai09	cfac111e2b	cann: add doc for cann backend (#8867) ... Co-authored-by: xuedinge233 <damow890@gmail.com> Co-authored-by: hipudding <huafengchun@gmail.com>	2024-08-19 16:46:38 +08:00
Minsoo Cheong	c679e0cb5c	llama : add EXAONE model support (#9025) ... * add exaone model support * add chat template * fix whitespace Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * add ftype * add exaone pre-tokenizer in `llama-vocab.cpp` Co-Authored-By: compilade <113953597+compilade@users.noreply.github.com> * fix lint Co-Authored-By: compilade <113953597+compilade@users.noreply.github.com> * add `EXAONE` to supported models in `README.md` * fix space Co-authored-by: compilade <git@compilade.net> --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> Co-authored-by: compilade <113953597+compilade@users.noreply.github.com> Co-authored-by: compilade <git@compilade.net>	2024-08-16 09:35:18 +03:00
Frank Mai	84eb2f4fad	docs: introduce gpustack and gguf-parser (#8873) ... * readme: introduce gpustack GPUStack is an open-source GPU cluster manager for running large language models, which uses llama.cpp as the backend. Signed-off-by: thxCode <thxcode0824@gmail.com> * readme: introduce gguf-parser GGUF Parser is a tool to review/check the GGUF file and estimate the memory usage without downloading the whole model. Signed-off-by: thxCode <thxcode0824@gmail.com> --------- Signed-off-by: thxCode <thxcode0824@gmail.com>	2024-08-12 14:45:50 +02:00
Eric Curtin	b42978e7e4	readme : add ramalama to the availables UI (#8811) ... ramalama is a repo agnostic boring CLI tool that supports pulling from ollama, huggingface and oci registries. Signed-off-by: Eric Curtin <ecurtin@redhat.com>	2024-08-05 15:45:01 +03:00
BarfingLemurs	400ae6f65f	readme : update model list (#8851)	2024-08-05 08:54:10 +03:00
R0CKSTAR	e54c35e4fb	feat: Support Moore Threads GPU (#8383) ... * Update doc for MUSA Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com> * Add GGML_MUSA in Makefile Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com> * Add GGML_MUSA in CMake Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com> * CUDA => MUSA Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com> * MUSA adds support for __vsubss4 Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com> * Fix CI build failure Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com> --------- Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>	2024-07-28 01:41:25 +02:00
MorganRO8	68504f0970	readme : update games list (#8673) ... Added link to game I made that depends on llama	2024-07-24 19:48:00 +03:00
Thorsten Sommer	3a7ac5300a	readme : update UI list [no ci] (#8505)	2024-07-24 15:52:30 +03:00
Georgi Gerganov	be0cfb4175	readme : fix server badge	2024-07-19 14:34:55 +03:00
Andy Salerno	fd560fe680	Update README.md to fix broken link to docs (#8399) ... Update the "Performance troubleshooting" doc link to be correct - the file was moved into a dir called 'development'	2024-07-09 14:58:44 -04:00
b4b4o	c4dd11d1d3	readme : fix web link error [no ci] (#8347)	2024-07-08 17:19:24 +03:00