From 73233040924ab4e14f8a2cfa5f2131c5840896cf Mon Sep 17 00:00:00 2001 From: kerthcet Date: Tue, 20 Aug 2024 10:43:41 +0800 Subject: [PATCH] Add llmaz as another platform to run llama.cpp on Kubernetes Signed-off-by: kerthcet --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 04d315db7..911d67481 100644 --- a/README.md +++ b/README.md @@ -191,6 +191,7 @@ Unless otherwise noted these projects are open-source with permissive licensing: **Infrastructure:** +- [llmaz](https://github.com/InftyAI/llmaz) - ☸️ Effortlessly serve state-of-the-art LLMs on Kubernetes, see [llama.cpp example](https://github.com/InftyAI/llmaz/tree/main/docs/examples/llamacpp) here. - [Paddler](https://github.com/distantmagic/paddler) - Stateful load balancer custom-tailored for llama.cpp - [GPUStack](https://github.com/gpustack/gpustack) - Manage GPU clusters for running LLMs