llama-bench: add --allreduce flag to select AllReduce provider

Adds --allreduce <auto|nccl|internal> to llama-bench (and via the shared
field pattern, consistent with other multi-value flags).  Useful for
isolating hangs or regressions in tensor-parallel mode: pass --allreduce nccl
to force NCCL and bypass the internal provider.

Also fixes ggml_cuda_select_allreduce_provider() to treat an empty
GGML_CUDA_ALLREDUCE env var the same as unset (avoids spurious warning when
llama-bench sets it to "" for the "auto" case).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
xt gains ar_pipeline field
- Provider selection via GGML_CUDA_ALLREDUCE env var ("nccl" / "internal")
- INTERNAL provider initialises the pipeline at comm_init time
- Dispatch routes to ggml_cuda_ar_allreduce(); falls back to meta-backend
  CPU reduce for unsupported sizes or GPU counts (> 2)

Current scope: 2 GPUs, FP32, tensors <= 256 KB. Notes in NOTES-allreduce.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Scott Cutler
2026-04-21 17:56:20 -07:00
parent 4c1178686b
commit 52af99d337
2 changed files with 2472 additions and 2432 deletions

View File

@@ -1177,7 +1177,7 @@ struct ggml_backend_cuda_comm_context {
static ggml_cuda_allreduce_provider ggml_cuda_select_allreduce_provider(
const std::vector<int> & device_ids) {
const char * env = getenv("GGML_CUDA_ALLREDUCE");
if (env != nullptr) {
if (env != nullptr && env[0] != '\0') {
if (strcmp(env, "internal") == 0) {
return GGML_CUDA_ALLREDUCE_INTERNAL;
}

File diff suppressed because it is too large Load Diff