mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2026-08-07 02:20:48 -05:00
llama-bench: add --allreduce flag to select AllReduce provider
Adds --allreduce <auto|nccl|internal> to llama-bench (and via the shared field pattern, consistent with other multi-value flags). Useful for isolating hangs or regressions in tensor-parallel mode: pass --allreduce nccl to force NCCL and bypass the internal provider. Also fixes ggml_cuda_select_allreduce_provider() to treat an empty GGML_CUDA_ALLREDUCE env var the same as unset (avoids spurious warning when llama-bench sets it to "" for the "auto" case). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> xt gains ar_pipeline field - Provider selection via GGML_CUDA_ALLREDUCE env var ("nccl" / "internal") - INTERNAL provider initialises the pipeline at comm_init time - Dispatch routes to ggml_cuda_ar_allreduce(); falls back to meta-backend CPU reduce for unsupported sizes or GPU counts (> 2) Current scope: 2 GPUs, FP32, tensors <= 256 KB. Notes in NOTES-allreduce.md. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -1177,7 +1177,7 @@ struct ggml_backend_cuda_comm_context {
|
||||
static ggml_cuda_allreduce_provider ggml_cuda_select_allreduce_provider(
|
||||
const std::vector<int> & device_ids) {
|
||||
const char * env = getenv("GGML_CUDA_ALLREDUCE");
|
||||
if (env != nullptr) {
|
||||
if (env != nullptr && env[0] != '\0') {
|
||||
if (strcmp(env, "internal") == 0) {
|
||||
return GGML_CUDA_ALLREDUCE_INTERNAL;
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user