root/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggerganov/llama.cpp.git synced 2024-12-26 19:34:35 +00:00
Code Issues Actions 4 Packages Projects Releases Wiki Activity

dev-note: more compact table

brian khuu 2024-06-12 00:28:26 +10:00
parent 315efe7c92
commit 333b2970fa
1 changed files with 26 additions and 24 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

50
dev-notes.md

@ -3,10 +3,10 @@
These are general free form note with pointers to good jumping to point to under These are general free form note with pointers to good jumping to point to under
stand the llama.cpp codebase. stand the llama.cpp codebase.
(`@<symbol>` is a vscode jump to symbol code for your convenience. [Also making a feature request to vscode to be able to jump to file and symbol](https://github.com/microsoft/vscode/issues/214870)) (`@<symbol>` is a vscode jump to symbol code for your convenience. [Also making a feature request to vscode to be able to jump to file and symbol via `<file>:@<symbol>`](https://github.com/microsoft/vscode/issues/214870))
## Where are the definitions? ## Where are the definitions for GGUF in llama.cpp?
[GGUF file structure spec (WARN: As of 2024-06-11 the llama.cpp implementation is the canonical source for now)](https://github.com/ggerganov/ggml/blob/master/docs/gguf.md#file-structure) [GGUF file structure spec (WARN: As of 2024-06-11 the llama.cpp implementation is the canonical source for now)](https://github.com/ggerganov/ggml/blob/master/docs/gguf.md#file-structure)
@ -26,28 +26,30 @@ Please use this as an index not as canonical reference.
The purpose of this table is to allow you to quickly locate major elements of The purpose of this table is to allow you to quickly locate major elements of
the gguf file standard. the gguf file standard.
| GGUF Elements Of Interest | c name | c type | gguf.c reference | vscode search line | | Header Name | GGUF Elements Of Interest | c name | c type | vscode search line |
|-------------------------------------------------------|-------------------------|---------------------------|---------------------------|---------------------| |-------------------|-----------------------------------|-------------------------|---------------------------|----------------------------|
| Magic | magic | `uint8_t[4]` | `struct gguf_header` | `@gguf_header` | | GGUF Header | Magic | magic | `uint8_t[4]` | gguf.c:`@gguf_header` |
| Version | version | `uint32_t` | `struct gguf_header` | `@gguf_header` | | GGUF Header | Version | version | `uint32_t` | gguf.c:`@gguf_header` |
| Tensor Count | n_tensors | `uint64_t` | `struct gguf_header` | `@gguf_header` | | GGUF Header | Tensor Count | n_tensors | `uint64_t` | gguf.c:`@gguf_header` |
| Key Value Count | n_kv | `uint64_t` | `struct gguf_header` | `@gguf_header` | | GGUF Header | Key Value Count | n_kv | `uint64_t` | gguf.c:`@gguf_header` |
| Key Value Linked List | kv | `gguf_kv *` | `struct gguf_context` | `@gguf_context` | | GGUF Context | Key Value Linked List | kv | `gguf_kv *` | gguf.c:`@gguf_context` |
| Tensor Info Linked List | infos | `gguf_tensor_info *` | `struct gguf_context` | `@gguf_context` | | GGUF Context | Tensor Info Linked List | infos | `gguf_tensor_info *` | gguf.c:`@gguf_context` |
| Key Value Entry - Key | gguf_kv.key | `gguf_str` | `struct gguf_kv` | `@gguf_kv` | | Key Value Entry | Key | gguf_kv.key | `gguf_str` | gguf.c:`@gguf_kv` |
| Key Value Entry - Type | gguf_kv.type | `gguf_type` | `struct gguf_kv` | `@gguf_kv` | | Key Value Entry | Type | gguf_kv.type | `gguf_type` | gguf.c:`@gguf_kv` |
| Key Value Entry - Type | gguf_kv.value | `gguf_value` | `struct gguf_kv` | `@gguf_kv` | | Key Value Entry | Type | gguf_kv.value | `gguf_value` | gguf.c:`@gguf_kv` |
| Tensor Info Entry - Name | gguf_tensor_info.name | `gguf_str` | `struct gguf_tensor_info` | `@gguf_tensor_info` | | Tensor Info Entry | Name | gguf_tensor_info.name | `gguf_str` | gguf.c:`@gguf_tensor_info` |
| Tensor Info Entry - Tensor shape dimension count | gguf_tensor_info.n_dim | `uint32_t` | `struct gguf_tensor_info` | `@gguf_tensor_info` | | Tensor Info Entry | Tensor shape dimension count | gguf_tensor_info.n_dim | `uint32_t` | gguf.c:`@gguf_tensor_info` |
| Tensor Info Entry - Tensor shape sizing array | gguf_tensor_info.ne | `uint64_t[GGML_MAX_DIMS]` | `struct gguf_tensor_info` | `@gguf_tensor_info` | | Tensor Info Entry | Tensor shape sizing array | gguf_tensor_info.ne | `uint64_t[GGML_MAX_DIMS]` | gguf.c:`@gguf_tensor_info` |
| Tensor Info Entry - Tensor Encoding Scheme / Strategy | gguf_tensor_info.type | `ggml_type` | `struct gguf_tensor_info` | `@gguf_tensor_info` | | Tensor Info Entry | Tensor Encoding Scheme / Strategy | gguf_tensor_info.type | `ggml_type` | gguf.c:`@gguf_tensor_info` |
| Tensor Info Entry - Offset from start of 'data' | gguf_tensor_info.offset | `uint64_t` | `struct gguf_tensor_info` | `@gguf_tensor_info` | | Tensor Info Entry | Offset from start of 'data' | gguf_tensor_info.offset | `uint64_t` | gguf.c:`@gguf_tensor_info` |
Also just note that these values are not actually part of gguf but is there for internal usage and is calculated during model loading. Also just note that these values are not actually part of gguf but is there for internal usage and is calculated during model loading.
Aka it's for the writing/reading api Aka it's for the writing/reading api.
| GGML Elements Of Interest | c name | c type | gguf.c reference | vscode search line | | Header Name | GGML Elements Of Interest | c name | c type | vscode search line |
|-------------------------------------------------------|-------------------------|---------------------------|---------------------------|---------------------| |-------------------|-----------------------------------------------------|-----------|----------|----------------------------|
| Alignment | alignment | `size_t` | `struct gguf_context` | `@gguf_context` | | GGUF Context | Alignment | alignment | `size_t` | gguf.c:`@gguf_context` |
| Offset Of 'Data' From Beginning Of File | offset | `size_t` | `struct gguf_context` | `@gguf_context` | | GGUF Context | Offset Of 'Data' From Beginning Of File | offset | `size_t` | gguf.c:`@gguf_context` |
| Size Of 'Data' In Bytes | size | `size_t` | `struct gguf_context` | `@gguf_context` | | GGUF Context | Size Of 'Data' In Bytes | size | `size_t` | gguf.c:`@gguf_context` |
| Tensor Info Entry | Tensor memory mapped pointer location in computer | data | `void *` | gguf.c:`@gguf_tensor_info` |
| Tensor Info Entry | Tensor memory mapped size of layer data in computer | size | `size_t` | gguf.c:`@gguf_tensor_info` |