moves ggml-vocab.bin into test folder where it's used.

It appears this file is only used during tests as of now.
Removing it from the model folder makes it more flexible
for how users are loading their model data into the project
(e.g. are they using a docker bind-mounts, are they using
symlinks, are they DLing models directly into this folder?)

By moving this, the instructions for getting started can be
safely simplified to:

$  rm models/.gitkeep
$  rm -r models
$  ln -s /mnt/c/ai/models/LLaMA $(pwd)/models

I think it's a good idea because the model files are quite large, and
be useful across multiple projects so symlinks shine in this use case
without creating too much confusion for the onboardee..
This commit is contained in:
TheNotary 2023-04-26 12:55:45 -05:00
parent 0b2da20538
commit 0e41441fa1
3 changed files with 2 additions and 2 deletions

2
.gitignore vendored
View File

@ -20,7 +20,7 @@ build-no-accel/
build-sanitize-addr/
build-sanitize-thread/
models/*
models
/main
/quantize

View File

@ -8,4 +8,4 @@ endfunction()
# llama_add_test(test-double-float.c) # SLOW
llama_add_test(test-quantize-fns.cpp)
llama_add_test(test-quantize-perf.cpp)
llama_add_test(test-tokenizer-0.cpp ${CMAKE_CURRENT_SOURCE_DIR}/../models/ggml-vocab.bin)
llama_add_test(test-tokenizer-0.cpp ${CMAKE_CURRENT_SOURCE_DIR}/ggml-vocab.bin)

0
models/ggml-vocab.bin → tests/ggml-vocab.bin Normal file → Executable file
View File