root/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggerganov/llama.cpp.git synced 2024-11-11 13:30:35 +00:00
Code Issues Actions 11 Packages Projects Releases Wiki Activity

llama : update llm_build_copy_mask_state comment [no ci] (#9385)

Browse Source 
This commit updates the comment, which seems to contain a typo or be an
outdated comment, in the copy_mask_state function changing the variable
n_rs to n_kv.

I believe this change is correct and what the comment wants to
convey is to copy the states that are not going to be used in the
upcoming processing, which are the tokens states from n_seqs up to
the number of possible token states n_kv.
This commit is contained in:
Daniel Bevenius 2024-09-10 09:03:21 +02:00 committed by GitHub
parent 0b4ac75772
commit 83008b7cfe
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
src/llama.cpp
View File

@ -9258,7 +9258,7 @@ static struct ggml_tensor * llm_build_copy_mask_state(
// FIXME: zero-out NANs? // FIXME: zero-out NANs?
states = ggml_mul(ctx, states, state_mask); states = ggml_mul(ctx, states, state_mask);
// copy states which won't be changed further (between n_seqs and n_rs) // copy states which won't be changed further (between n_seqs and n_kv)
ggml_build_forward_expand(graph, ggml_build_forward_expand(graph,
ggml_cpy(ctx, ggml_cpy(ctx,
ggml_view_1d(ctx, states, n_state*(n_kv - n_seqs), n_seqs*n_state*ggml_element_size(states)), ggml_view_1d(ctx, states, n_state*(n_kv - n_seqs), n_seqs*n_state*ggml_element_size(states)),