llama.cpp/pyrightconfig.json

{
  "extraPaths": ["gguf-py"],
  "pythonVersion": "3.9",
  "pythonPlatform": "All",
  "reportUnusedImport": "warning",
  "reportDuplicateImport": "error",
  "reportDeprecated": "warning",
  "reportUnnecessaryTypeIgnoreComment": "warning",
  "executionEnvironments": [
    {
      // TODO: make this version override work correctly
      "root": "gguf-py",
      "pythonVersion": "3.8",
    },
    {
      // uses match expressions in steps.py
      "root": "examples/server/tests",
      "pythonVersion": "3.10",
    },
  ],
 }
convert-hf : save memory with lazy evaluation (#7075) * convert-hf : begin refactoring write_tensor * convert : upgrade to sentencepiece v0.2.0 * convert-hf : remove unused n_dims in extra__tensors convert-hf : simplify MoE weights stacking * convert-hf : flake8 linter doesn't like semicolons * convert-hf : allow unusual model part names For example, loading `model-00001-of-00001.safetensors` now works. * convert-hf : fix stacking MoE expert tensors `torch.stack` and `torch.cat` don't do the same thing. * convert-hf : fix Mamba conversion Tested to work even with a SentencePiece-based tokenizer. * convert : use a string for the SentencePiece tokenizer path * convert-hf : display tensor shape * convert-hf : convert norms to f32 by default * convert-hf : sort model part names `os.listdir` is said to list files in arbitrary order. Sorting the file names should let "model-00009-of-00042.safetensors" be loaded before "model-00010-of-00042.safetensors". * convert-hf : use an ABC for Model again It seems Protocol can't be used as a statically type-checked ABC, because its subclasses also can't be instantiated. (why did it seem to work?) At least there's still a way to throw an error when forgetting to define the `model_arch` property of any registered Model subclasses. * convert-hf : use a plain class for Model, and forbid direct instantiation There are no abstract methods used anyway, so using ABC isn't really necessary. * convert-hf : more consistent formatting of cmdline args * convert-hf : align the message logged for converted tensors * convert-hf : fix Refact conversion * convert-hf : save memory with lazy evaluation * convert-hf : flake8 doesn't like lowercase L as a variable name * convert-hf : remove einops requirement for InternLM2 * convert-hf : faster model parts loading Instead of pre-loading them all into a dict, iterate on the tensors in the model parts progressively as needed in Model.write_tensors Conversion for some architectures relies on checking for the presence of specific tensor names, so for multi-part models, the weight map is read from the relevant json file to quickly get these names up-front. * convert-hf : minor changes for consistency * gguf-py : add tqdm as a dependency It's small, and used for a progress bar in GGUFWriter.write_tensors_to_file 2024-05-08 22:16:38 +00:00			`{`
			`"extraPaths": ["gguf-py"],`
py : type-check all Python scripts with Pyright (#8341) * py : type-check all Python scripts with Pyright * server-tests : use trailing slash in openai base_url * server-tests : add more type annotations * server-tests : strip "chat" from base_url in oai_chat_completions * server-tests : model metadata is a dict * ci : disable pip cache in type-check workflow The cache is not shared between branches, and it's 250MB in size, so it would become quite a big part of the 10GB cache limit of the repo. * py : fix new type errors from master branch * tests : fix test-tokenizer-random.py Apparently, gcc applies optimisations even when pre-processing, which confuses pycparser. * ci : only show warnings and errors in python type-check The "information" level otherwise has entries from 'examples/pydantic_models_to_grammar.py', which could be confusing for someone trying to figure out what failed, considering that these messages can safely be ignored even though they look like errors. 2024-07-07 19:04:39 +00:00			`"pythonVersion": "3.9",`
			`"pythonPlatform": "All",`
			`"reportUnusedImport": "warning",`
			`"reportDuplicateImport": "error",`
			`"reportDeprecated": "warning",`
			`"reportUnnecessaryTypeIgnoreComment": "warning",`
			`"executionEnvironments": [`
			`{`
			`// TODO: make this version override work correctly`
			`"root": "gguf-py",`
			`"pythonVersion": "3.8",`
			`},`
			`{`
			`// uses match expressions in steps.py`
			`"root": "examples/server/tests",`
			`"pythonVersion": "3.10",`
			`},`
			`],`
			`}`