Zhenwei Jin
6102037bbb
vocab : refactor tokenizer to reduce init overhead ( #9449 )
...
* refactor tokenizer
* llama : make llm_tokenizer more private
ggml-ci
* refactor tokenizer
* refactor tokenizer
* llama : make llm_tokenizer more private
ggml-ci
* remove unused files
* remove unused fileds to avoid unused filed build error
* avoid symbol link error
* Update src/llama.cpp
* Update src/llama.cpp
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2024-09-28 15:10:58 +03:00
nopperl
9a913110cf
llama : add support for Chameleon ( #8543 )
...
* convert chameleon hf to gguf
* add chameleon tokenizer tests
* fix lint
* implement chameleon graph
* add swin norm param
* return qk norm weights and biases to original format
* implement swin norm
* suppress image token output
* rem tabs
* add comment to conversion
* fix ci
* check for k norm separately
* adapt to new lora implementation
* fix layer input for swin norm
* move swin_norm in gguf writer
* add comment regarding special token regex in chameleon pre-tokenizer
* Update src/llama.cpp
Co-authored-by: compilade <git@compilade.net>
* fix punctuation regex in chameleon pre-tokenizer (@compilade)
Co-authored-by: compilade <git@compilade.net>
* fix lint
* trigger ci
---------
Co-authored-by: compilade <git@compilade.net>
2024-09-28 15:08:43 +03:00
Aarni Koskela
43bcdd9703
readme : add tool ( #9655 )
2024-09-28 15:07:14 +03:00
Dan Johansson
6a0f779484
ggml : add run-time detection of neon, i8mm and sve ( #9331 )
...
* ggml: Added run-time detection of neon, i8mm and sve
Adds run-time detection of the Arm instructions set features
neon, i8mm and sve for Linux and Apple build targets.
* ggml: Extend feature detection to include non aarch64 Arm arch
* ggml: Move definition of ggml_arm_arch_features to the global data section
2024-09-28 15:06:16 +03:00
Markus Tavenrath
89f9944981
Enable use to the rebar feature to upload buffers to the device. ( #9251 )
2024-09-28 12:05:05 +02:00
ochafik
887951beb0
minja: generate chat goldens w/ fixed date to support Llama-3.2-3B-Instruct (uses strftime_now)
2024-09-27 19:52:15 +01:00
ochafik
701b664551
minja: add indent filter to support command-r-plus's chat templates
2024-09-27 19:00:14 +01:00
Georgi Gerganov
b5de3b74a5
readme : update hot topics
Nix CI / nix-eval (macos-latest) (push) Has been cancelled
Nix CI / nix-eval (ubuntu-latest) (push) Has been cancelled
Nix CI / nix-build (macos-latest) (push) Has been cancelled
Nix CI / nix-build (ubuntu-latest) (push) Has been cancelled
flake8 Lint / Lint (push) Has been cancelled
update-flake-lock / lockfile (push) Has been cancelled
2024-09-27 20:57:51 +03:00
ochafik
0093a5e527
minja: fix identifiers parsing (when start w/ not/is/etc) and lstrip_blocks corner case (needed by DeepSeek-V2.5
2024-09-27 18:30:44 +01:00
Borislav Stanimirov
44f59b4301
cmake : add option for common library ( #9661 )
Nix CI / nix-eval (macos-latest) (push) Waiting to run
Nix CI / nix-eval (ubuntu-latest) (push) Waiting to run
Nix CI / nix-build (macos-latest) (push) Waiting to run
Nix CI / nix-build (ubuntu-latest) (push) Waiting to run
flake8 Lint / Lint (push) Waiting to run
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/full-cuda.Dockerfile platforms:linux/amd64 tag:full-cuda]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/full.Dockerfile platforms:linux/amd64,linux/arm64 tag:full]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/llama-cli-cuda.Dockerfile platforms:linux/amd64 tag:light-cuda]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/llama-cli-intel.Dockerfile platforms:linux/amd64 tag:light-intel]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/llama-cli.Dockerfile platforms:linux/amd64,linux/arm64 tag:light]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/llama-server-cuda.Dockerfile platforms:linux/amd64 tag:server-cuda]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/llama-server-intel.Dockerfile platforms:linux/amd64 tag:server-intel]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/llama-server.Dockerfile platforms:linux/amd64,linux/arm64 tag:server]) (push) Has been cancelled
2024-09-27 10:42:06 +03:00
ochafik
2f25ee30ef
Update README.md
2024-09-27 07:18:07 +01:00
ochafik
86e4f99092
Update README.md
2024-09-27 07:15:25 +01:00
ochafik
e62b5de3cf
tool-call: fix functionary-small-3.2 (first tool starts w/ name\n, subsequent are >>>name\n)
2024-09-27 07:06:33 +01:00
ochafik
e33b342da7
tool-call: fix passing of tools to template + allow agent to finish
2024-09-27 06:24:22 +01:00
ochafik
f62e688387
tool-call: fix crash / test non-tool call case (added llama_sampler_is_grammar_empty)
2024-09-27 06:04:41 +01:00
ochafik
0abfa36ca7
tool-call: move usage examples to examples/agent
2024-09-27 05:10:30 +01:00
ochafik
6610ecf965
server: rm bad debug code
2024-09-27 04:07:35 +01:00
ochafik
27cd07a056
json: fix grammar conversion typo
2024-09-27 03:57:48 +01:00
ochafik
9295ca95db
tool-call: fix agent type lints
2024-09-27 03:53:56 +01:00
ochafik
1e5c0e747e
chat-template: fix jinja tests (make safe a passthrough)
2024-09-27 03:50:04 +01:00
ochafik
f9c1743bb5
minja: fix iterables
2024-09-27 03:36:49 +01:00
ochafik
8299fac07c
tool-call: adapt very simple agent + docker isolation from https://github.com/ggerganov/llama.cpp/pull/6389
2024-09-26 21:07:46 +01:00
ochafik
10f9fe8d49
tool-call: fix tool call return format
2024-09-26 21:01:04 +01:00
ochafik
c88c932d98
fix gcc error + lint
2024-09-26 19:18:40 +01:00
ochafik
2926089c5d
fix lints
2024-09-26 19:06:29 +01:00
ochafik
5840e10069
tool-call: merge & fix jinja template tests into test-chat-template
2024-09-26 19:05:00 +01:00
ochafik
50685f837f
minja: add str.title()
2024-09-26 19:03:59 +01:00
ochafik
296331bba3
minja: update chat template goldens w/ llama.3.1 arguments workaround
2024-09-26 18:10:27 +01:00
ochafik
9cfe4d7202
tool-call: refactor llama_chat_template class + use in validate_model_chat_template
2024-09-26 18:06:03 +01:00
ochafik
cf7bece6a7
tool-call: factor chat template away from legacy API
2024-09-26 17:19:29 +01:00
Neo Zhang Jianyu
95bc82fbc0
[SYCL] add missed dll file in package ( #9577 )
...
Nix CI / nix-eval (macos-latest) (push) Has been cancelled
Nix CI / nix-eval (ubuntu-latest) (push) Has been cancelled
Nix CI / nix-build (macos-latest) (push) Has been cancelled
Nix CI / nix-build (ubuntu-latest) (push) Has been cancelled
flake8 Lint / Lint (push) Has been cancelled
* update oneapi to 2024.2
* use 2024.1
---------
Co-authored-by: arthw <14088817+arthw@users.noreply.github.com>
2024-09-26 17:38:31 +08:00
ochafik
d7ec84f78c
tool-call: allow <|python_tag|> in functionary-medium-3.1
2024-09-26 06:52:34 +01:00
ochafik
3d2650ce65
fix gcc build
2024-09-26 06:52:34 +01:00
ochafik
749a21c67a
gcc appeasement
2024-09-26 06:08:18 +01:00
ochafik
0c870133d8
tool-call: test/fix functionary-medium-v3.1's template (can "look" like llama3.1 template)
2024-09-26 05:56:15 +01:00
ochafik
8e4a9bad8a
minja: allow none input to selectattr, and add safe passthrough filter
2024-09-26 05:53:12 +01:00
ochafik
5f5be9cde7
minja: gcc tweaks
2024-09-26 05:06:11 +01:00
ochafik
2eb29bf8b8
tool-call: update chat templates/goldens
2024-09-26 04:00:10 +01:00
ochafik
4cd82d61dd
tool-call: fix pyright type errors
2024-09-26 03:59:38 +01:00
ochafik
059babdd9b
minja: try to please gcc
2024-09-26 03:58:18 +01:00
ochafik
94377d743c
server: catch errors in format_final_response_oaicompat instead of taking server down
2024-09-26 03:42:36 +01:00
ochafik
595e11cb11
tool-call: fix/test functionary v3
2024-09-26 03:42:05 +01:00
ochafik
c124ab48ea
minja: add str.endswith
2024-09-26 03:21:23 +01:00
ochafik
76d2938ef8
fix flake8 lints
2024-09-26 02:30:17 +01:00
ochafik
1b6280102b
fix editorconfig lints
2024-09-26 02:27:46 +01:00
R0CKSTAR
7691654c68
mtgpu: enable VMM ( #9597 )
...
Nix CI / nix-eval (macos-latest) (push) Waiting to run
Nix CI / nix-eval (ubuntu-latest) (push) Waiting to run
Nix CI / nix-build (macos-latest) (push) Waiting to run
Nix CI / nix-build (ubuntu-latest) (push) Waiting to run
flake8 Lint / Lint (push) Waiting to run
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/full-cuda.Dockerfile platforms:linux/amd64 tag:full-cuda]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/full.Dockerfile platforms:linux/amd64,linux/arm64 tag:full]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/llama-cli-cuda.Dockerfile platforms:linux/amd64 tag:light-cuda]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/llama-cli-intel.Dockerfile platforms:linux/amd64 tag:light-intel]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/llama-cli.Dockerfile platforms:linux/amd64,linux/arm64 tag:light]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/llama-server-cuda.Dockerfile platforms:linux/amd64 tag:server-cuda]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/llama-server-intel.Dockerfile platforms:linux/amd64 tag:server-intel]) (push) Has been cancelled
Publish Docker image / Push Docker image to Docker Hub (map[dockerfile:.devops/llama-server.Dockerfile platforms:linux/amd64,linux/arm64 tag:server]) (push) Has been cancelled
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
2024-09-26 03:27:40 +02:00
ochafik
ab25e3fbf9
tool-call: allow empty message content when there's tool_calls in format_chat
2024-09-26 02:19:04 +01:00
ochafik
d928ff4dfd
server: catch errors in oaicompat_completion_params_parse instead of taking server down
2024-09-26 02:18:01 +01:00
ochafik
a774093a99
tool-call: add server tests for llama 3.1
2024-09-26 02:17:30 +01:00
ochafik
9e366b3d03
server: fix tailing comma in completions_seed
2024-09-26 02:15:48 +01:00
