upstream-archive/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-07-28 21:50:56 -05:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • 732b5fbf5e convert : avoid calls to tokenizer.added_tokens_decoder (#12473) Bartowski 2025-03-20 02:36:37 -04:00
  • 568013d0cd context : clear sets containing encoder output sequence ids before storing new values (#12470) b4927 fairydreaming 2025-03-19 21:01:57 +01:00
  • 517b5ddbf0 CUDA: Improve flash decoding kernel GPU occupancy for BS=1 case (#12183) b4926 Gaurav Garg 2025-03-20 01:22:06 +05:30
  • a9b59288e2 vulkan: optimize iq1 coopmat2 dequant functions (#12427) b4925 Jeff Bolz 2025-03-19 13:56:23 -05:00
  • 8b80d68338 embedding : avoid common_batch Georgi Gerganov 2025-03-19 14:29:04 +02:00
  • 6f54ee660c retrieval : avoid common_batch Georgi Gerganov 2025-03-19 13:50:15 +02:00
  • 0fd8487b14 Fix visionOS build and add CI (#12415) b4924 Guus Waals 2025-03-19 10:15:23 +00:00
  • 32c2c41d5e android : fix permission Xuan Son Nguyen 2025-03-19 10:49:30 +01:00
  • 96ca6e8d23 swift : adapt to new API Georgi Gerganov 2025-03-19 10:48:42 +02:00
  • b0db7fc2c6 android : adapt to new API Georgi Gerganov 2025-03-19 10:16:55 +02:00
  • 23d7407314 Merge pull request #15 from ggml-org/xsn/private_batch_api Xuan-Son Nguyen 2025-03-19 09:15:09 +01:00
  • 108e53c2f1 llama : add support for GPT2, Bloom and CodeShell tied word embeddings (#12456) b4923 Sigbjørn Skjæret 2025-03-19 09:08:49 +01:00
  • a686171ea7 convert : Support chat_template.json (#12460) Sigbjørn Skjæret 2025-03-19 08:58:13 +01:00
  • c446b2edd2 vulkan: Submit once enough matmul work has been recorded (#12406) b4921 Jeff Bolz 2025-03-19 02:26:26 -05:00
  • 7a3c178d78 speculative : adapt to new llama API xsn/private_batch_api Georgi Gerganov 2025-03-18 16:10:26 +02:00
  • d84635b1b0 opencl: improve profiling (#12442) b4920 lhez 2025-03-18 12:54:55 -07:00
  • 75422e8bc4 graph : normalize Q, K, V shapes + sync cross attention (#12449) b4919 Georgi Gerganov 2025-03-18 21:35:19 +02:00
  • bb115d2bf7 musa: override warp_size of musa device to 32 (#12445) R0CKSTAR 2025-03-19 02:28:26 +08:00
  • 29fff308c7 llama : support converting Mistral Small text-only (#12450) Xuan-Son Nguyen 2025-03-18 19:16:19 +01:00
  • c6af2161b2 speculative : fix seg fault in certain cases (#12454) b4916 Georgi Gerganov 2025-03-18 19:35:11 +02:00
  • 99aa304fb9 llama : add support for EXAONE tied word embeddings (#12451) b4915 Xuan-Son Nguyen 2025-03-18 17:24:33 +01:00
  • dc4bb64290 Merge branch 'master' into xsn/private_batch_api Xuan Son Nguyen 2025-03-18 15:45:22 +01:00
  • 8551c44d84 context : always use non-causal attention for encoder graphs (#12447) b4914 Georgi Gerganov 2025-03-18 13:05:49 +02:00
  • 35cae5ba05 SYCL: using graphs is configurable by environment variable and compile option (#12371) b4913 Łukasz Ślusarczyk 2025-03-18 11:16:31 +01:00
  • 810e0af3f5 server : fix warmup draft cache type (#12446) b4912 Georgi Gerganov 2025-03-18 12:05:42 +02:00
  • 29acf2cf05 context : move the change to llama_context::encode() gg/context-fix-enc-attn-type Georgi Gerganov 2025-03-18 11:55:19 +02:00
  • eba92d64c3 cmake : fix PowerPC build (#12241) b4911 Prajwal B Mehendarkar 2025-03-18 15:07:33 +05:30
  • a0554c3cdc context : always use non-causal attention for encoder graphs Georgi Gerganov 2025-03-18 11:14:48 +02:00
  • d9a14523bb ggml : add SVE support for q6_K_q8_K (#12361) b4910 fj-y-saito 2025-03-18 17:14:39 +09:00
  • fd123cfead Vulkan: Default to 1GB allocations instead of 4GB to avoid fragmentation and driver issues (#12434) b4909 0cc4m 2025-03-18 07:21:40 +01:00
  • a53f7f7b88 fixed compilation warnings in ggml-sycl (#12424) b4908 Łukasz Ślusarczyk 2025-03-18 01:51:25 +01:00
  • 7dfad387e3 llama: Add support for RWKV v7 architecture (#12412) b4907 Molly Sophia 2025-03-18 07:27:50 +08:00
  • 60c902926c docs : bring llama-cli conversation/template docs up-to-date (#12426) Sigbjørn Skjæret 2025-03-17 21:14:32 +01:00
  • b1b132efcb cuda : enable CUDA Graph on CUDA Toolkit < 12.x (#12394) b4905 Gaurav Garg 2025-03-17 23:55:13 +05:30
  • 01e8f2138b ggml-vulkan: remove unused find_program(glslc) (#12416) Guus Waals 2025-03-18 00:35:43 +08:00
  • 484a8ab513 vulkan: Add N/2 and N/4 optimized paths in coopmat2 shader (#12312) b4903 Jeff Bolz 2025-03-17 09:26:18 -05:00
  • cf2270e4d3 vulkan: subgroup size tuning (#12087) b4902 Daniele 2025-03-17 12:42:33 +01:00
  • eab5606d7b Apply suggestions from code review Xuan-Son Nguyen 2025-03-17 12:17:14 +01:00
  • de788e071b Update examples/tts/tts.cpp Xuan-Son Nguyen 2025-03-17 12:05:23 +01:00
  • f07690c930 vulkan: use fp32 in coopmat2 q4_k dequant function (#12309) b4901 Jeff Bolz 2025-03-17 04:43:35 -05:00
  • 891c63956d vulkan: Pad N dimension of B matrix for coopmat2 perf, to avoid bounds checking (#12273) b4900 Jeff Bolz 2025-03-17 04:41:59 -05:00
  • 2f21123c1d vulkan: Adjust coopmat2 tile sizes and selection heuristic (#12258) b4899 Jeff Bolz 2025-03-17 04:35:00 -05:00
  • 374101fd74 cmake : enable building llama.cpp using system libggml (#12321) b4898 Christian Kastner 2025-03-17 10:05:23 +01:00
  • b3c9a65673 SYCL: set extras only on GGML_TYPE_Q4_0 (#12366) b4897 Akarshan Biswas 2025-03-17 07:15:12 +05:30
  • 8ba95dca20 llama : fix OLMo-2-0325-32B-Instruct K-norm size (#12400) b4896 Sigbjørn Skjæret 2025-03-16 18:46:36 +01:00
  • dc079cfdff context : fix init of n_outputs (#12397) b4895 Georgi Gerganov 2025-03-16 19:29:36 +02:00
  • 7b61bcc87c ci : add --symlinks to xcframework zip command (#12409) Daniel Bevenius 2025-03-16 18:22:05 +01:00
  • f6711cef44 CUDA: determine FA parallel blocks at runtime jg/cuda-fa-np-runtime Johannes Gäßler 2025-03-06 16:47:33 +01:00
  • 30ad9c2873 ggml-quants : faster exhaustive IQ4_NL rounding with k_heap Francis Couture-Harpin 2025-03-15 12:55:22 -04:00
  • f4c3dd5daa llama-tts : add '-o' option (#12398) b4893 marcoStocchi 2025-03-15 17:23:11 +01:00
  • 3d35d87b41 SYCL: Delete redundant plus sign and space (#12391) b4892 aubreyli 2025-03-15 22:49:03 +08:00
  • 0c9e442489 ggml-quants : remove some commented code Francis Couture-Harpin 2025-03-15 10:29:47 -04:00
  • b19bd064c0 SYCL : support non-contiguous tensors in binary ops (add, sub, etc) (#12399) b4891 fairydreaming 2025-03-15 15:19:30 +01:00
  • 92a391327e [CANN]MUL_MAT optimization (#12382) Chenguang Li 2025-03-15 09:31:08 +08:00
  • 624a683c6f fix compile Xuan Son Nguyen 2025-03-14 22:30:29 +01:00
  • 116b9a1662 rename to init_from_text Xuan Son Nguyen 2025-03-14 22:17:07 +01:00
  • 9f2250ba72 Add CLI arg to llama-run to adjust the number of threads used (#12370) b4889 Eric Curtin 2025-03-14 16:41:20 +00:00
  • eaffba0f2e llama_batch_ext_ptr::from_text/embd Xuan Son Nguyen 2025-03-14 17:12:03 +01:00
  • 774973b8f3 main : add -sysf / --system-prompt-file (#12249) (#12250) b4888 Sigbjørn Skjæret 2025-03-14 16:57:05 +01:00
  • 8fcb563613 Load all MoE experts during warmup (#11571) fairydreaming 2025-03-14 13:47:05 +01:00
  • 8e7714fa77 fix compile Xuan Son Nguyen 2025-03-14 11:28:15 +01:00
  • a363251fac qwen2vl: use llama_batch_ext_set_pos Xuan Son Nguyen 2025-03-14 11:25:36 +01:00
  • add2a3aa5a server: fix "--grammar-file" parameter (#12285) b4886 Victor 2025-03-14 11:21:17 +01:00
  • ba79369615 fix llama_batch_ext_init_from_embd Xuan Son Nguyen 2025-03-14 11:17:22 +01:00
  • 07d84fa3c2 fix missing n_past in various places Xuan Son Nguyen 2025-03-14 10:47:08 +01:00
  • 32940369d3 fix gemma3-cli Xuan Son Nguyen 2025-03-14 10:33:28 +01:00
  • 5e6a6d4e1c fix llama-run n_past Xuan Son Nguyen 2025-03-14 10:32:43 +01:00
  • c522ce4143 graph : simplify attn input build for unified KV cache (#12381) b4885 Georgi Gerganov 2025-03-14 10:47:44 +02:00
  • 081bee8c64 hparams : add SWA rope parameters (#12374) b4884 Georgi Gerganov 2025-03-14 09:03:24 +02:00
  • bfdddbc150 bring back mistakenly deleted llama_batch_init/free Xuan Son Nguyen 2025-03-14 00:22:28 +01:00
  • 54566ad95d correct comment Xuan Son Nguyen 2025-03-14 00:21:06 +01:00
  • 04f8641815 rm redundant llama_batch_ext_set_output_last Xuan Son Nguyen 2025-03-13 23:14:16 +01:00
  • c3dd79007b fix llama_batch_ext_init_from_text Xuan Son Nguyen 2025-03-13 23:09:27 +01:00
  • 65f0184517 compile ok Xuan Son Nguyen 2025-03-13 22:56:35 +01:00
  • 9fb2d81eab fix common_batch missing seq_id Xuan Son Nguyen 2025-03-13 22:38:04 +01:00
  • 47086fa82d apply to the rest Xuan Son Nguyen 2025-03-13 22:36:27 +01:00
  • c4aca65582 hparams : add SWA rope parameters gg/hparams-swa-rope Georgi Gerganov 2025-03-13 19:26:09 +02:00
  • 84d5475541 llama : fix Gemma3 SWA KV cache shift (#12373) Georgi Gerganov 2025-03-13 19:08:07 +02:00
  • 4aabf4e8f4 return output ID from llama_batch_ext_add/set Xuan Son Nguyen 2025-03-13 17:47:07 +01:00
  • 86973cb14a fix merge errors Xuan Son Nguyen 2025-03-13 17:32:36 +01:00
  • 21fe0ce4eb hparams : add comment [no ci] gg/swa-fix-kv-shift Georgi Gerganov 2025-03-13 17:56:38 +02:00
  • de9d18fa9c llama : fix Gemma3 SWA KV cache shift Georgi Gerganov 2025-03-13 17:16:30 +02:00
  • 17f954c8e2 Merge branch 'master' into xsn/private_batch_api Xuan Son Nguyen 2025-03-13 15:55:18 +01:00
  • be7c303410 arg : no n_predict = -2 for examples except for main and infill (#12364) b4882 Xuan-Son Nguyen 2025-03-13 12:34:54 +01:00
  • e0dbec0bc6 llama : refactor llama_context, llama_kv_cache, llm_build_context (#12181) Georgi Gerganov 2025-03-13 12:35:44 +02:00
  • 2048b5913d server : fix crash when using verbose output with input tokens that are not in printable range (#12178) (#12338) b4880 Ishaan Gandhi 2025-03-13 06:10:05 -04:00
  • f08f4b3187 Update build.yml for Windows Vulkan builder to use Vulkan 1.4.304 SDK for VK_NV_cooperative_matrix2 support (#12301) b4879 Oscar Barenys 2025-03-12 20:06:58 +01:00
  • ed58975f51 server : improve infill stop criteria gg/infill-better-stop Georgi Gerganov 2025-03-11 15:43:37 +02:00
  • 80a02aa858 llama.swiftui : fix xcframework dir in README [no ci] (#12353) Daniel Bevenius 2025-03-12 13:45:32 +01:00
  • 363f8c5d67 sycl : variable sg_size support for mmvq kernels (#12336) b4877 Alberto Cabrera Pérez 2025-03-12 09:57:32 +00:00
  • 34c961b181 CUDA/HIP: Fix fattn-vec-* when device warp size is not 32 (#12315) b4876 uvos 2025-03-12 10:14:11 +01:00
  • 7841fc723e llama : Add Gemma 3 support (+ experimental vision capability) (#12343) b4875 Xuan-Son Nguyen 2025-03-12 09:30:24 +01:00
  • bf69cfe62f vulkan: fix bug in coopmat1 mul_mat_id (#12316) b4874 Jeff Bolz 2025-03-12 00:59:19 -05:00
  • 10f2e81809 CUDA/HIP: refractor mmqv to unify the calculation of nwarps and rows per block between host and device code. (#12177) b4873 uvos 2025-03-11 20:16:03 +01:00
  • ba7654380a ggml-backend : fix backend search path (#12330) b4872 jklincn 2025-03-11 21:25:17 +08:00
  • 6ab2e4765a metal : Cache the Metal library at the device context level (#12265) b4871 BB-fat 2025-03-11 19:45:02 +08:00
  • 96e1280839 clip : bring back GPU support (#12322) b4870 Xuan-Son Nguyen 2025-03-11 09:20:16 +01:00
  • 2c9f833d17 mat vec double buffer (#12188) b4869 Eve 2025-03-10 19:28:11 +00:00
  • 251364549f musa: support new arch mp_31 and update doc (#12296) b4868 R0CKSTAR 2025-03-11 01:18:25 +08:00
  • 8acdacb3ea opencl: use OpenCL C standard supported by the device (#12221) b4867 Henry Linjamäki 2025-03-10 18:57:00 +02:00