upstream-archive/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-07-23 11:10:55 -05:00
Code Issues Packages Projects Releases Wiki Activity

CUDA: Support CUDA Virtual Devices (#25228)

Browse Source 
* support cuda virtual devices

* disable NCCL path when virtual devices are used

* label virtual devices in description; add GPUx2 server CI jobs

* code refactor
This commit is contained in:
Anav Prasad
2026-07-16 03:37:35 -07:00
committed by GitHub
parent 3f08ef2c51
commit 79bba02a67
3 changed files with 173 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
.github/workflows/server-self-hosted.yml vendored
View File

@@ -143,6 +143,24 @@ jobs:
export LLAMA_ARG_BACKEND_SAMPLING=1
pytest -v -x -m "not slow"
- name: Tests (GPUx2)
id: server_integration_tests_gpu2
if: ${{ !github.event.pull_request }}
run: |
cd tools/server/tests
source venv/bin/activate
export GGML_CUDA_DEVICES=2
pytest -v -x -m "not slow"
- name: Tests (GPUx2, backend-sampling)
id: server_integration_tests_gpu2_backend_sampling
if: ${{ !github.event.pull_request }}
run: |
cd tools/server/tests
source venv/bin/activate
export GGML_CUDA_DEVICES=2 LLAMA_ARG_BACKEND_SAMPLING=1
pytest -v -x -m "not slow"
server-kleidiai:
runs-on: ah-ubuntu_22_04-c8g_8x