Compare commits

...

3 Commits

Author SHA1 Message Date
Kante Yin
49c1a6217e
Merge 7323304092 into 912c331d3d 2024-09-22 22:27:38 +08:00
Molly Sophia
912c331d3d
Fix merge error in #9454 (#9589)
Signed-off-by: Molly Sophia <mollysophia379@gmail.com>
2024-09-22 15:26:50 +02:00
kerthcet
7323304092 Add llmaz as another platform to run llama.cpp on Kubernetes
Signed-off-by: kerthcet <kerthcet@gmail.com>
2024-08-20 10:43:41 +08:00
2 changed files with 2 additions and 0 deletions

View File

@ -178,6 +178,7 @@ Unless otherwise noted these projects are open-source with permissive licensing:
**Infrastructure:**
- [llmaz](https://github.com/InftyAI/llmaz) - ☸️ Effortlessly serve state-of-the-art LLMs on Kubernetes, see [llama.cpp example](https://github.com/InftyAI/llmaz/tree/main/docs/examples/llamacpp) here.
- [Paddler](https://github.com/distantmagic/paddler) - Stateful load balancer custom-tailored for llama.cpp
- [GPUStack](https://github.com/gpustack/gpustack) - Manage GPU clusters for running LLMs

View File

@ -2351,6 +2351,7 @@ static bool ggml_cuda_compute_forward(ggml_backend_cuda_context & ctx, struct gg
break;
case GGML_OP_RWKV_WKV:
ggml_cuda_op_rwkv_wkv(ctx, dst);
break;
case GGML_OP_CROSS_ENTROPY_LOSS_BACK:
ggml_cuda_cross_entropy_loss_back(ctx, dst);
break;