upstream-archive/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-09-28 00:47:25 -05:00
Code Issues Packages Projects Releases Wiki Activity
Files
b11207
llama.cpp/CODEOWNERS
T
Line: 20: incorrect codeowner user: ngxson Line: 20: no users/groups matched Line: 21: incorrect codeowner organization: ggml-org/ci Line: 21: no users/groups matched Line: 22: incorrect codeowner organization: ggml-org/ci Line: 22: no users/groups matched Line: 23: incorrect codeowner user: ggerganov Line: 23: no users/groups matched Line: 24: incorrect codeowner user: ggerganov Line: 24: no users/groups matched Line: 25: incorrect codeowner organization: ggml-org/llama-common Line: 25: no users/groups matched Line: 26: incorrect codeowner user: JohannesGaessler Line: 26: no users/groups matched Line: 27: incorrect codeowner user: CISC Line: 27: no users/groups matched Line: 28: incorrect codeowner user: srogmann Line: 28: no users/groups matched Line: 29: incorrect codeowner user: CISC Line: 29: no users/groups matched Line: 30: incorrect codeowner user: CISC Line: 30: no users/groups matched Line: 31: incorrect codeowner organization: ggml-org/ggml-hexagon Line: 31: no users/groups matched Line: 32: incorrect codeowner user: ggerganov Line: 32: no users/groups matched Line: 33: incorrect codeowner user: ggerganov Line: 33: no users/groups matched Line: 34: incorrect codeowner user: ggerganov Line: 34: no users/groups matched Line: 35: incorrect codeowner user: danbev Line: 35: incorrect codeowner user: pwilkin Line: 35: no users/groups matched Line: 36: incorrect codeowner user: ggerganov Line: 36: no users/groups matched Line: 37: incorrect codeowner user: am17an Line: 37: no users/groups matched Line: 38: incorrect codeowner user: ggerganov Line: 38: no users/groups matched Line: 39: incorrect codeowner user: ggerganov Line: 39: no users/groups matched Line: 40: incorrect codeowner user: ggerganov Line: 40: no users/groups matched Line: 41: incorrect codeowner user: ggerganov Line: 41: no users/groups matched Line: 42: incorrect codeowner user: ggerganov Line: 42: no users/groups matched Line: 43: incorrect codeowner user: ggerganov Line: 43: incorrect codeowner user: hanyin-arm Line: 43: incorrect codeowner user: naco-siren Line: 43: no users/groups matched Line: 44: incorrect codeowner user: ggerganov Line: 44: no users/groups matched Line: 45: incorrect codeowner user: ggerganov Line: 45: no users/groups matched Line: 46: incorrect codeowner user: ggerganov Line: 46: no users/groups matched Line: 47: incorrect codeowner user: JohannesGaessler Line: 47: no users/groups matched Line: 48: incorrect codeowner user: danbev Line: 48: no users/groups matched Line: 49: incorrect codeowner user: ggerganov Line: 49: no users/groups matched Line: 50: incorrect codeowner user: ggerganov Line: 50: no users/groups matched Line: 51: incorrect codeowner user: ggerganov Line: 51: no users/groups matched Line: 52: incorrect codeowner user: ggerganov Line: 52: no users/groups matched Line: 53: incorrect codeowner user: ggerganov Line: 53: no users/groups matched Line: 54: incorrect codeowner user: ggerganov Line: 54: no users/groups matched Line: 55: incorrect codeowner user: ggerganov Line: 55: no users/groups matched Line: 56: incorrect codeowner user: JohannesGaessler Line: 56: no users/groups matched Line: 57: incorrect codeowner organization: ggml-org/ggml-cann Line: 57: no users/groups matched Line: 58: incorrect codeowner user: ggerganov Line: 58: no users/groups matched Line: 59: incorrect codeowner user: ggerganov Line: 59: no users/groups matched Line: 60: incorrect codeowner user: jbooth Line: 60: incorrect codeowner user: bartowski1182 Line: 60: no users/groups matched Line: 61: incorrect codeowner user: alex-spacemit Line: 61: no users/groups matched Line: 62: incorrect codeowner organization: ggml-org/ggml-cuda Line: 62: no users/groups matched Line: 63: incorrect codeowner user: IMbackK Line: 63: no users/groups matched Line: 64: incorrect codeowner organization: ggml-org/ggml-hexagon Line: 64: no users/groups matched Line: 65: incorrect codeowner user: IMbackK Line: 65: no users/groups matched Line: 66: incorrect codeowner user: marty1885 Line: 66: no users/groups matched Line: 67: incorrect codeowner user: ggerganov Line: 67: no users/groups matched Line: 68: incorrect codeowner organization: ggml-org/ggml-metal Line: 68: no users/groups matched Line: 69: incorrect codeowner organization: ggml-org/ggml-opencl Line: 69: no users/groups matched Line: 70: incorrect codeowner user: cavusmustafa Line: 70: incorrect codeowner user: wine99 Line: 70: no users/groups matched Line: 71: incorrect codeowner user: JohannesGaessler Line: 71: no users/groups matched Line: 72: incorrect codeowner user: ggerganov Line: 72: no users/groups matched Line: 73: incorrect codeowner organization: ggml-org/ggml-rpc Line: 73: no users/groups matched Line: 74: incorrect codeowner organization: ggml-org/ggml-sycl Line: 74: no users/groups matched Line: 75: incorrect codeowner user: ggerganov Line: 75: no users/groups matched Line: 76: incorrect codeowner user: kpouget Line: 76: no users/groups matched Line: 77: incorrect codeowner organization: ggml-org/ggml-vulkan Line: 77: no users/groups matched Line: 78: incorrect codeowner organization: ggml-org/ggml-webgpu Line: 78: no users/groups matched Line: 79: incorrect codeowner organization: ggml-org/ggml-zdnn Line: 79: incorrect codeowner user: Andreas-Krebbel Line: 79: incorrect codeowner user: AlekseiNikiforovIBM Line: 79: no users/groups matched Line: 80: incorrect codeowner user: avinashcpandey Line: 80: incorrect codeowner user: Jiten1parmar Line: 80: incorrect codeowner user: z-vishal Line: 80: no users/groups matched Line: 81: incorrect codeowner user: ggerganov Line: 81: no users/groups matched Line: 82: incorrect codeowner user: ggerganov Line: 82: no users/groups matched Line: 83: incorrect codeowner user: JohannesGaessler Line: 83: incorrect codeowner user: Green-Sky Line: 83: no users/groups matched Line: 84: incorrect codeowner user: CISC Line: 84: no users/groups matched Line: 85: incorrect codeowner user: ggerganov Line: 85: no users/groups matched Line: 86: incorrect codeowner user: ggerganov Line: 86: no users/groups matched Line: 87: incorrect codeowner user: ggerganov Line: 87: no users/groups matched Line: 88: incorrect codeowner user: ggerganov Line: 88: no users/groups matched Line: 89: incorrect codeowner organization: ggml-org/ggml-hexagon Line: 89: no users/groups matched Line: 90: incorrect codeowner user: ggerganov Line: 90: no users/groups matched Line: 91: incorrect codeowner user: CISC Line: 91: no users/groups matched Line: 92: incorrect codeowner user: CISC Line: 92: no users/groups matched Line: 93: incorrect codeowner user: ngxson Line: 93: no users/groups matched Line: 94: incorrect codeowner user: CISC Line: 94: no users/groups matched Line: 95: incorrect codeowner user: CISC Line: 95: no users/groups matched Line: 96: incorrect codeowner user: CISC Line: 96: no users/groups matched Line: 97: incorrect codeowner user: CISC Line: 97: no users/groups matched Line: 98: incorrect codeowner user: ggerganov Line: 98: no users/groups matched Line: 99: incorrect codeowner user: pwilkin Line: 99: no users/groups matched Line: 100: incorrect codeowner user: ggerganov Line: 100: no users/groups matched Line: 101: incorrect codeowner user: ngxson Line: 101: no users/groups matched Line: 102: incorrect codeowner user: ggerganov Line: 102: no users/groups matched Line: 103: incorrect codeowner organization: ggml-org/llama-mtmd Line: 103: no users/groups matched Line: 104: incorrect codeowner user: ggerganov Line: 104: no users/groups matched Line: 105: incorrect codeowner user: pwilkin Line: 105: no users/groups matched Line: 106: incorrect codeowner user: ggerganov Line: 106: no users/groups matched Line: 107: incorrect codeowner organization: ggml-org/ggml-rpc Line: 107: no users/groups matched Line: 108: incorrect codeowner organization: ggml-org/llama-server Line: 108: no users/groups matched Line: 109: incorrect codeowner organization: ggml-org/llama-server Line: 109: no users/groups matched Line: 110: incorrect codeowner organization: ggml-org/llama-ui Line: 110: no users/groups matched Line: 111: incorrect codeowner user: ggerganov Line: 111: no users/groups matched Line: 112: incorrect codeowner user: ggerganov Line: 112: no users/groups matched Line: 113: incorrect codeowner user: ggerganov Line: 113: no users/groups matched Line: 114: incorrect codeowner user: ggerganov Line: 114: no users/groups matched Line: 115: incorrect codeowner user: ggerganov Line: 115: no users/groups matched Line: 116: incorrect codeowner user: ggerganov Line: 116: no users/groups matched Line: 117: incorrect codeowner user: ggerganov Line: 117: no users/groups matched Line: 118: incorrect codeowner user: ggerganov Line: 118: no users/groups matched Line: 119: incorrect codeowner user: ggerganov Line: 119: no users/groups matched Line: 120: incorrect codeowner user: danbev Line: 120: no users/groups matched Line: 121: incorrect codeowner user: CISC Line: 121: no users/groups matched Line: 122: incorrect codeowner user: ngxson Line: 122: no users/groups matched
jboothandGeorgi Gerganov d834d44e64 ggml-cpu: tiled mul_mat for k-quants (#27851) 
* Added tiled mul_mat.

For each mul_mat_one_chunk, quants are unpacked into (max) 256x256 tiles of int8,
one routine per quent.  Then microkernel computes 16x16 tiles before writing out
256x256 float reults to main memory.

Tests/benches in tests/test-tiled-mulmat.cpp.  3-6x speed improvement
for large matmul, break even at 4096x64 * 64x4096, 80% performance (net
loss) for GEMV.  Error rates trivial (order of 1-e04 max, 1-e05 rmse).

* Fixes for ARM/windows builds

* more windows fixes, ggml-cpu.h isn't visible in MSVC for some reason

* unified iqp + tiled on the Q5_K, IQ4_XS set for benchmarking, updated benchmark

* Fixed accidental removal of llama_build_and_test(test-backend-ops.cpp)

* First integration of iqp code

Co-authored-by Bartowski <3266127+bartowski1182@users.noreply.github.com>

* Cleaning up declaration of iq unpacking helpers to align with the bit unpackers

* Removed iqp path

* Fix cross-platform warnings

* Disabling benchmarks unless explicitly enabled

* Fix backend_init for DLL-based builds, add self and bartowski to CODEOWNERS for tiled

* Put benchmarks behind a flag

* kernel fix for AVX2, iq quants

* Fix for asan, leaking memory in test-tiled-mulmat and avoid stack use after return

* guarding env flags with std::call_once

* Simplified repacking for VNNI to a single call per macrotile

* No threadlocals anymore, aligned wdata access

* Doing aligned reads since we ensure alignment with padding in wdata

* Eliminated per-thread gather of Q8_K rows in mul_mat_id, we now gather/repack in a single pass.  Repack method now takes pointer array to support both dense/normal and mmid paths.  Interface with ggml-cpu.c simplified as a result

* Unified/simplified dispatch and support checks.  Put details on wdata needed inside the kernel.h body, simplified interactions with ggml-cpu.c.

* Cleanup includes and whitespace, update src1_repack to return false if we don't need a special repack, so the common case is handled by driver

* Better detection of win32 and additional whitespace fixes

* Gating fuzz tests behind a parameter and some extra prints to try and fix slow CI hosts

* Optimized AVX2 kernel

* Changed interleave format and added ability to interleave in-place after dequant

* Repacks now happen in-place, 16x64 microtiles are independent of each other

* Only repack rows in groups of 16 as they're needed.  Save work in low n_rows cases and optimize L1 usage in other cases

* Use long panels for memory-bound regime (M <= 16), reintroduce IQP path for benchmarks

* Fix unused warnings and cleanup.  Improved IQ dequantization speed.

* Removed separate process benchmarks

* Revert "Removed separate process benchmarks"

This reverts commit 0688cf43d5.

* AVX2 optimizations and guards for tests on windows

* Removed temp perf harness

* Remove perf-mulmat from build

* Removed IQP path, simplified tests to not use sub processes

* Cleaning up alignment of wdata

* Whitespace fixes and aligning L2 workspace to clean 512kb boundaries

* Update ggml/src/ggml-cpu/tiled/tiled-kernel.cpp

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* Cleanup merge-duplicated declaration of test-backend-ops target

* Undo accidental line deletion in ggml.c

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
2026-09-26 08:38:57 +03:00

123 lines
6.3 KiB
Plaintext
Raw Permalink Blame History

# collaborators can optionally add themselves here to indicate their availability for reviewing related PRs
# multiple collaborators per item can be specified
#
# ggml-org/ci : CISC, danbev, ggerganov, netrunnereve, ngxson, taronaeo
# ggml-org/ggml-cann : hipudding
# ggml-org/ggml-cuda : JohannesGaessler, am17an, IMbackK, ORippler
# ggml-org/ggml-hexagon : lhez, max-krasnyansky
# ggml-org/ggml-metal : ggerganov
# ggml-org/ggml-opencl : lhez, max-krasnyansky
# ggml-org/ggml-rpc : rgerganov
# ggml-org/ggml-sycl : arthw
# ggml-org/ggml-vulkan : 0cc4m, jeffbolznv
# ggml-org/ggml-webgpu : reeselevine, yomaytk
# ggml-org/ggml-zdnn : taronaeo
# ggml-org/llama-common : ggerganov, aldehir, angt, danbev, ngxson, pwilkin
# ggml-org/llama-mtmd : ngxson
# ggml-org/llama-server : ggerganov, ngxson, allozaur, angt, ServeurpersoCom
# ggml-org/llama-ui : allozaur
/.devops/*.Dockerfile @ngxson
/.github/actions/ @ggml-org/ci
/.github/workflows/ @ggml-org/ci
/ci/ @ggerganov
/cmake/ @ggerganov
/common/ @ggml-org/llama-common
/common/fit.* @JohannesGaessler
/common/jinja/ @CISC
/common/ngram-map.* @srogmann
/conversion/ @CISC
/convert_*.py @CISC
/docs/backend/snapdragon/ @ggml-org/ggml-hexagon
/examples/batched.swift/ @ggerganov
/examples/batched/ @ggerganov
/examples/convert-llama2c-to-ggml/ @ggerganov
/examples/debug/ @danbev @pwilkin
/examples/deprecation-warning/ @ggerganov
/examples/diffusion/ @am17an
/examples/embedding/ @ggerganov
/examples/eval-callback/ @ggerganov
/examples/export-docs/ @ggerganov
/examples/gen-docs/ @ggerganov
/examples/gguf/ @ggerganov
/examples/llama.android/ @ggerganov @hanyin-arm @naco-siren
/examples/llama.swiftui/ @ggerganov
/examples/llama.vim @ggerganov
/examples/lookahead/ @ggerganov
/examples/lookup/ @JohannesGaessler
/examples/model-conversion/ @danbev
/examples/parallel/ @ggerganov
/examples/passkey/ @ggerganov
/examples/retrieval/ @ggerganov
/examples/speculative-simple/ @ggerganov
/examples/speculative/ @ggerganov
/ggml/cmake/ @ggerganov
/ggml/include/ @ggerganov
/ggml/src/ggml-backend-meta.cpp @JohannesGaessler
/ggml/src/ggml-cann/ @ggml-org/ggml-cann
/ggml/src/ggml-common.h @ggerganov
/ggml/src/ggml-cpu/ @ggerganov
/ggml/src/ggml-cpu/tiled/ @jbooth @bartowski1182
/ggml/src/ggml-cpu/spacemit/ @alex-spacemit
/ggml/src/ggml-cuda/ @ggml-org/ggml-cuda
/ggml/src/ggml-cuda/vendors/hip.h @IMbackK
/ggml/src/ggml-hexagon/ @ggml-org/ggml-hexagon
/ggml/src/ggml-hip/ @IMbackK
/ggml/src/ggml-et/ @marty1885
/ggml/src/ggml-impl.h @ggerganov
/ggml/src/ggml-metal/ @ggml-org/ggml-metal
/ggml/src/ggml-opencl/ @ggml-org/ggml-opencl
/ggml/src/ggml-openvino/ @cavusmustafa @wine99
/ggml/src/ggml-opt.cpp @JohannesGaessler
/ggml/src/ggml-quants.* @ggerganov
/ggml/src/ggml-rpc/ @ggml-org/ggml-rpc
/ggml/src/ggml-sycl/ @ggml-org/ggml-sycl
/ggml/src/ggml-threading.* @ggerganov
/ggml/src/ggml-virtgpu/ @kpouget
/ggml/src/ggml-vulkan/ @ggml-org/ggml-vulkan
/ggml/src/ggml-webgpu/ @ggml-org/ggml-webgpu
/ggml/src/ggml-zdnn/ @ggml-org/ggml-zdnn @Andreas-Krebbel @AlekseiNikiforovIBM
/ggml/src/ggml-zendnn/ @avinashcpandey @Jiten1parmar @z-vishal
/ggml/src/ggml.c @ggerganov
/ggml/src/ggml.cpp @ggerganov
/ggml/src/gguf.cpp @JohannesGaessler @Green-Sky
/gguf-py/ @CISC
/media/ @ggerganov
/scripts/gen* @ggerganov
/scripts/get* @ggerganov
/scripts/sync* @ggerganov
/scripts/snapdragon/ @ggml-org/ggml-hexagon
/src/ @ggerganov
/src/llama-adapter.* @CISC
/src/llama-arch.* @CISC
/src/llama-chat.* @ngxson
/src/llama-graph.* @CISC
/src/llama-model.* @CISC
/src/llama-vocab.* @CISC
/src/models/ @CISC
/tests/ @ggerganov
/tests/test-chat.* @pwilkin
/tools/batched-bench/ @ggerganov
/tools/cli/ @ngxson
/tools/completion/ @ggerganov
/tools/mtmd/ @ggml-org/llama-mtmd
/tools/perplexity/ @ggerganov
/tools/parser/ @pwilkin
/tools/quantize/ @ggerganov
/tools/rpc/ @ggml-org/ggml-rpc
/tools/server/* @ggml-org/llama-server # no subdir
/tools/server/tests/ @ggml-org/llama-server
/tools/ui/ @ggml-org/llama-ui
/tools/tokenize/ @ggerganov
/tools/tts/ @ggerganov
/vendor/ @ggerganov
/AUTHORS @ggerganov
/CMakeLists.txt @ggerganov
/CONTRIBUTING.md @ggerganov
/LICENSE @ggerganov
/README.md @ggerganov
/SECURITY.md @ggerganov
/build-xcframework.sh @danbev
requirements*.txt @CISC
/skills @ngxson
Reference in New Issue View Git Blame Copy Permalink