root/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggerganov/llama.cpp.git synced 2025-01-14 04:24:30 +00:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

convert-falcon-hf-to-gguf.py : add tensor data layout

Browse Source
This commit is contained in:
klosax 2023-08-19 12:05:11 +02:00 committed by GitHub
parent 2c8055b65b
commit b3a7a2b486
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
convert-falcon-hf-to-gguf.py
View File

@ -95,7 +95,8 @@ print("gguf: get model metadata")
block_count = hparams["n_layer"]
gguf_writer.add_name(last_dir)
gguf_writer.add_context_length(2048)
gguf_writer.add_context_length(2048) # not in config.json
gguf_writer.add_tensor_data_layout("jploski") # qkv tensor transform
gguf_writer.add_embedding_length(hparams["hidden_size"])
gguf_writer.add_feed_forward_length(4 * hparams["hidden_size"])
gguf_writer.add_block_count(block_count)