llama.cpp/CMakeLists.txt at 0d2ec438330271d201c2e9224aca23d0d5c908bf - llama.cpp - Gitea: Git with a cup of tea

root/llama.cpp

mirror of https://github.com/ggerganov/llama.cpp.git synced 2024-11-11 13:30:35 +00:00

Radoslav Gerganov 5e31828d3e

ggml : add RPC backend (#6829 )

* ggml : add RPC backend

The RPC backend proxies all operations to a remote server which runs a
regular backend (CPU, CUDA, Metal, etc).

* set TCP_NODELAY

* add CI workflows

* Address review comments

* fix warning

* implement llama_max_devices() for RPC

* Address review comments

* Address review comments

* wrap sockfd into a struct

* implement get_alignment and get_max_size

* add get_device_memory

* fix warning

* win32 support

* add README

* readme : trim trailing whitespace

* Address review comments

* win32 fix

* Address review comments

* fix compile warnings on macos

2024-05-14 14:27:19 +03:00

3 lines

95 B

CMake

Raw Blame History

	`add_executable(rpc-server rpc-server.cpp)`
	`target_link_libraries(rpc-server PRIVATE ggml llama)`