root/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggerganov/llama.cpp.git synced 2024-12-30 21:34:36 +00:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity
1,085 Commits 360 Branches 2,875 Tags 407 MiB
f8218477b3
Commit Graph

1085 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Henri Vasserman
7297128db8
[Zig] Rewrite build for Zig 0.11 (#2514) 
* zig build fixes

* Disable LTO on Windows.
 2023-08-07 08:35:53 +03:00
DannyDaemonic
86c3219895
console : fix issue related to Windows 11 PowerShell console mode persistence (#2521) 2023-08-06 09:49:34 +03:00
Keiichi Tabata
2e8265ae17
convert.py : add missing abstract methods for quantized data (#2491) 2023-08-06 09:34:05 +03:00
Johannes Gäßler
f514d1b306
CUDA: faster k-quant mul_mat_q kernels (#2525) 2023-08-05 18:20:44 +02:00
Jonas Wunderlich
332311234a
fix firefox autoscroll (#2519) 2023-08-04 22:16:11 +02:00
Cebtenzzre
182af739c4
server: regenerate completion.js.hpp (#2515) 2023-08-04 21:00:57 +02:00
Cebtenzzre
4329d1acb0
CUDA: use min compute capability of GPUs actually used (#2506) 2023-08-04 17:35:22 +02:00
Cebtenzzre
02f9d96a86
CUDA: check if event is NULL before cudaStreamWaitEvent (#2505) 
Fixes #2503
 2023-08-04 17:34:32 +02:00
DannyDaemonic
3498588e0f
Add --simple-io option for subprocesses and break out console.h and cpp (#1558) 2023-08-04 08:20:12 -07:00
Stephen Nichols
5f631c2679
Fixing race condition in server and partial stream handling in frontend. (#2391) 
* Fixing race condition in server.cpp and partial stream handling in completion.js

* Reverting assert edits.

* Adding newline to eof
 2023-08-04 13:37:24 +02:00
l3utterfly
415e99fec2
Stream save llama context data to file instead of allocating entire buffer upfront (#2488) 
* added stream saving context data to file to avoid allocating unnecessary amounts of memory

* generalised copying state data to file or buffer

* added comments explaining how copy_state_data works

* fixed trailing whitespaces

* fixed save load state example

* updated save load state to use public function in llama.cpp

* - restored breakage of the llama_copy_state_data API
- moved new logic for copying llama state data to internal function

* fixed function declaration order

* restored save load state example

* fixed whitepace

* removed unused llama-util.h include

* Apply suggestions from code review

Co-authored-by: slaren <slarengh@gmail.com>

* Apply code review suggestions

Co-authored-by: slaren <slarengh@gmail.com>

---------

Co-authored-by: slaren <slarengh@gmail.com>
 2023-08-04 13:29:52 +02:00
Borislav Stanimirov
ff966e7ca6
build : fix several cast and printf warnings (#2499) 2023-08-04 13:07:21 +03:00
klosax
db5618ad99
cmpnct_gpt2bpe.hpp : comments 2023-08-04 04:57:51 +02:00
klosax
278ada9572
gguf.py : bytesarray for gpt2bpe tokenizer 2023-08-04 04:07:57 +02:00
klosax
fb0b243705
Makefile : remove gptneox-common 2023-08-04 04:02:10 +02:00
klosax
5d98989cf6
gpt2 bpe tokenizer (handles merges and unicode) 2023-08-04 03:58:44 +02:00
klosax
e6f19ba240
gptneox-main.cpp : gpt2 bpe tokenizer 2023-08-04 03:56:37 +02:00
klosax
2922280a1a
convert-gptneox-h5-to-gguf.py : gpt2bpe tokenizer 2023-08-04 03:55:23 +02:00
klosax
6691aa8797
Delete gptneox-common.h 2023-08-04 03:52:01 +02:00
klosax
23abbe8e00
Delete gptneox-common.cpp 2023-08-04 03:51:43 +02:00
Evan Jones
8183159cf3
examples : generate JSON according to schema (#1887) 
* examples : add JSON schema grammars

* complete JSON grammar

* ensure primitive types can be used as root of schema

* support integer type and adjust usage text
 2023-08-02 22:05:44 -04:00
Johannes Gäßler
468ea24fb4
CUDA: faster non k-quant mul_mat_q kernels (#2483) 2023-08-02 18:04:04 +02:00
Johannes Gäßler
4f6b60c776
CUDA: Fix models with output size != 32000 (#2480) 2023-08-02 16:48:10 +02:00
klosax
c5ba5efda2
convert-llama-h5-to-gguf.py : special tokens 2023-08-02 11:26:07 +02:00
klosax
e1e9b28547
convert-llama-h5-to-gguf.py : accumulate kv / ti + special tokens 2023-08-02 11:15:33 +02:00
ldwang
220d931864
readme : add Aquila-7B model series to supported models (#2487) 
* support bpe tokenizer in convert

Signed-off-by: ldwang <ftgreat@gmail.com>

* support bpe tokenizer in convert

Signed-off-by: ldwang <ftgreat@gmail.com>

* support bpe tokenizer in convert, fix

Signed-off-by: ldwang <ftgreat@gmail.com>

* Add Aquila-7B models in README.md

Signed-off-by: ldwang <ftgreat@gmail.com>

* Up Aquila-7B models in README.md

Signed-off-by: ldwang <ftgreat@gmail.com>

---------

Signed-off-by: ldwang <ftgreat@gmail.com>
Co-authored-by: ldwang <ftgreat@gmail.com>
 2023-08-02 11:21:11 +03:00
M. Yusuf Sarıgöz
c3a65c4bbe gguf-util.h : update note 2023-08-02 11:16:23 +03:00
M. Yusuf Sarıgöz
cf365fbc20 gguf : gguf counterpart of llama-util.h 2023-08-02 11:13:56 +03:00
Eve
81844fbcfd
tests : Fix compilation warnings (Linux/GCC) (#2451) 
* fix hellaswag print format, cast away warning in test-double-float

* c++11 cannot use designated initializers

* add static to test-grad0.c internal functions

* use memcpy in test-double-float.c

* port c tests to c++

* use initializer list for ggml_init_params
 2023-08-02 11:06:19 +03:00
Yiming Cui
a312193e18
readme : Add Chinese LLaMA-2 / Alpaca-2 to supported models (#2475) 
* add support for chinese llama-2 / alpaca-2

* remove white spaces
 2023-08-02 09:18:31 +03:00
klosax
1b4f9c8eb9
convert-gptneox-h5-to-gguf.py : accumulate kv and ti + special tokens 2023-08-01 23:40:50 +02:00
klosax
49380a23a3
gguf.py : accumulate kv and tensor info data + special tokens 2023-08-01 23:37:48 +02:00
klosax
ff1cb02397
constants.py : special tokens 2023-08-01 23:17:21 +02:00
Bono Lv
c574bddb36
fix a typo in examples/server/README.md (#2478) 2023-08-01 14:54:28 +02:00
klosax
36a36c32a3
Update gptneox-main.cpp 2023-08-01 14:44:28 +02:00
klosax
c77fabb1f9
gptneox-main.cpp : special tokens 2023-08-01 14:32:53 +02:00
klosax
e7a741695c
convert-gptneox-h5-to-gguf.py : Special tokens 2023-08-01 14:30:00 +02:00
ebraminio
86aeb27734
server : Support dark mode (#2414) 
* server : Support dark mode

So it respects user system light / dark settings.

* Update index.html.hpp by running ./deps.sh
 2023-08-01 10:56:23 +02:00
Matteo Boschini
1873ff586b
metal : add gqa8 kernel to allow llama-2-70B on metal (#2459) 
* Added gqa8 kernel to allow llama-2-70B on metal

* Update ggml-metal.m

Co-authored-by: Cebtenzzre <cebtenzzre@gmail.com>

* Extend kernel_mul_mat_f16_f32 to handle gqa broadcast

* Added ne03==ne13 assertion

---------

Co-authored-by: Cebtenzzre <cebtenzzre@gmail.com>
 2023-08-01 10:43:12 +03:00
klosax
da4900e835
Update convert-llama-h5-to-gguf.py 2023-07-31 23:04:03 +02:00
M. Yusuf Sarıgöz
f3de876a12 fix : update convert-llama-h5-to-gguf.py 2023-07-31 23:58:29 +03:00
Johannes Gäßler
49e7cb5bb1
CUDA: fixed LLAMA_FAST compilation option (#2473) 2023-07-31 21:02:19 +02:00
Johannes Gäßler
b772bba42e
CUDA: fixed cmake F16 option (#2471) 2023-07-31 19:52:22 +02:00
M. Yusuf Sarıgöz
bb42aefaeb gguf : mmap tensor data example 2023-07-31 17:46:12 +03:00
Johannes Gäßler
0728c5a8b9
CUDA: mmq CLI option, fixed mmq build issues (#2453) 2023-07-31 15:44:35 +02:00
M. Yusuf Sarıgöz
b26f5b2e43 gguf : fix typo in function call 2023-07-31 16:23:54 +03:00
Johannes Gäßler
1215ed7d5c
CUDA: Implemented row flattening for non-glm RoPE (#2468) 2023-07-31 14:32:30 +02:00
Johannes Gäßler
2dbf518911
CUDA: fewer memory bank conflicts for mul_mat_q (#2458) 2023-07-31 13:18:51 +02:00
slaren
9d2382b3e4
Fix Metal backend broken from the allocator changes (#2455) 
* fix Metal backend broken from the allocator changes
 2023-07-31 11:02:53 +02:00
M. Yusuf Sarıgöz
7aa0a0e7f7 gguf : support custom alignment value 2023-07-31 09:59:36 +03:00