upstream-archive/llama.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-08-01 23:50:47 -05:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • cb82adadb8 metal : first working version of the inference without prompt processing Georgi Gerganov 2023-07-20 14:56:29 +03:00
  • 290cb700bf metal : map the CPU buffers to Metal buffers (WIP) Georgi Gerganov 2023-07-20 14:30:34 +03:00
  • fff0e0eafe llama : fix regression from #2000 - could not load no-mmap models master-fff0e0e Georgi Gerganov 2023-07-20 13:47:26 +03:00
  • 417a85a001 metal: minor q4 optimization and reduce code size (#2248) Shouzheng Liu 2023-07-20 06:32:22 -04:00
  • 082dd81286 [wip] chat improvements Henri Vasserman 2023-07-20 03:48:48 +03:00
  • cb205c0d13 automatically calculate compute buffer sizes (without graph allocator) slaren 2023-07-20 02:22:54 +02:00
  • 77ac8deaf1 llama.cpp: remove backend-specific code where possible slaren 2023-07-20 00:59:26 +02:00
  • 43694ca867 consistent semicolons Henri Vasserman 2023-07-20 00:58:16 +03:00
  • 890d1b8446 Merge master into server-cfg Henri Vasserman 2023-07-20 00:48:03 +03:00
  • dd3cf5760a last n tokens done Henri Vasserman 2023-07-20 00:36:36 +03:00
  • 42591a0acd remove "smooth factor" Henri Vasserman 2023-07-20 00:02:13 +03:00
  • 2cb8469e7f refactor evaluation logic Henri Vasserman 2023-07-19 23:45:40 +03:00
  • f38433ef5d Merge remote-tracking branch 'origin/ggml-backends' into ggml-backends-metal Georgi Gerganov 2023-07-19 17:45:45 +03:00
  • 70c55c17c7 metal : create backend, mostly reuse CPU backend interface Georgi Gerganov 2023-07-19 16:47:43 +03:00
  • 294f424554 llama : extend API to get max devices at runtime (#2253) master-294f424 Rinne 2023-07-19 15:06:40 +08:00
  • 45a1b07e9b flake : update flake.nix (#2270) master-45a1b07 wzy 2023-07-19 15:01:55 +08:00
  • b1f4290953 cmake : install targets (#2256) master-b1f4290 wzy 2023-07-19 15:01:11 +08:00
  • 295f85654a allocators wip renamed ggml_backend functions changed ggml_buffer and ggml_backend to always be used as pointers rename ggml_tensor::params -> op_params slaren 2023-07-17 19:03:51 +02:00
  • ed960fa1ab llama : separate compute buffer for metal Georgi Gerganov 2023-07-18 19:19:59 +03:00
  • 652c849643 ggml : add is_ram_shared to ggml_backend Georgi Gerganov 2023-07-18 18:51:02 +03:00
  • 90503f150d llama : init metal backend as CPU backend for now Georgi Gerganov 2023-07-18 17:52:13 +03:00
  • 0a3861c47b metal : adapting to ggml_backend (WIP) Georgi Gerganov 2023-07-18 16:54:41 +03:00
  • d01bccde9f ci : integrate with ggml-org/ci (#2250) master-d01bccd Georgi Gerganov 2023-07-18 14:24:43 +03:00
  • 6cbf9dfb32 llama : shorten quantization descriptions master-6cbf9df Georgi Gerganov 2023-07-18 11:50:49 +03:00
  • 7568d1a2b2 Support dup & cont ops on CUDA (#2242) master-7568d1a Jiahao Li 2023-07-18 01:39:29 +08:00
  • 1102ff56db fix double-free with --no-mmap slaren 2023-07-17 12:00:17 +02:00
  • 4e94af3060 improve layer backend printing with ranges slaren 2023-07-17 11:53:01 +02:00
  • c2beeb8e3a only allocate as much memory as is required in each backend for the model slaren 2023-07-17 11:18:19 +02:00
  • b7647436cc llama : fix t_start_sample_us initialization warning (#2238) master-b764743 Alex Klinkhamer 2023-07-16 14:01:45 -07:00
  • 672dda10e4 ggml : fixed runtime bugs and compile errors related to GGML_PERF and GGML_DEBUG (#2219) master-672dda1 Qingyou Meng 2023-07-17 03:57:28 +08:00
  • 27ab66e437 py : turn verify-checksum-models.py into executable (#2245) Jiří Podivín 2023-07-16 21:54:47 +02:00
  • 9c72e7e916 rebase to master (except ggml-cuda) slaren 2023-07-16 14:36:32 +02:00
  • 33ab185dd1 fix NVCC version on Makefile, __halves2half2 -> make_half2 slaren 2023-07-16 00:20:43 +02:00
  • 24cc6f008f minor fixes slaren 2023-07-15 19:04:37 +02:00
  • 5765d7a587 restore simple.cpp for now slaren 2023-07-15 12:44:47 +02:00
  • 0d2b66c638 ggml backend interface wip slaren 2023-07-10 17:32:06 +02:00
  • 6e7cca4047 llama : add custom RoPE (#2054) master-6e7cca4 Xiao-Yong Jin 2023-07-15 06:34:16 -04:00
  • a6803cab94 flake : add runHook preInstall/postInstall to installPhase so hooks function (#2224) Dave Della Costa 2023-07-14 15:13:38 -04:00
  • 7dabc66f3c make : use pkg-config for OpenBLAS (#2222) master-7dabc66 wzy 2023-07-15 03:05:08 +08:00
  • 7cdd30bf1f cuda : allocate all temporary ggml_tensor_extra_gpu from a fixed-size buffer (#2220) master-7cdd30b Bach Le 2023-07-15 03:00:58 +08:00
  • e8035f141e ggml : fix static_assert with older compilers #2024 (#2218) master-e8035f1 Evan Miller 2023-07-14 14:55:56 -04:00
  • 7513b7b0a1 llama : add functions that work directly on model (#2197) master-7513b7b Bach Le 2023-07-15 02:55:24 +08:00
  • de8342423d build.zig : install config header (#2216) Ali Chraghi 2023-07-14 11:50:58 -07:00
  • c48c525f87 examples : fixed path typos in embd-input (#2214) Shangning Xu 2023-07-15 02:40:05 +08:00
  • 206e01de11 cuda : support broadcast add & mul (#2192) master-206e01d Jiahao Li 2023-07-15 02:38:24 +08:00
  • 4304bd3cde CUDA: mul_mat_vec_q kernels for k-quants (#2203) master-4304bd3 Johannes Gäßler 2023-07-14 19:44:08 +02:00
  • 229aab351c make : fix combination of LLAMA_METAL and LLAMA_MPI (#2208) master-229aab3 James Reynolds 2023-07-14 11:34:40 -06:00
  • 697966680b ggml : sync (ggml_conv_2d, fix mul_mat bug, CUDA GLM rope) master-6979666 Georgi Gerganov 2023-07-14 16:36:41 +03:00
  • 27ad57a69b Metal: faster Q4_0 and Q4_1 matrix x vector kernels (#2212) Kawrakow 2023-07-14 12:46:21 +03:00
  • 4cae9f5673 Port CFG to server. Henri Vasserman 2023-07-13 22:37:57 +03:00
  • 3a13d1e829 Apply formatting. Henri Vasserman 2023-07-13 20:24:31 +03:00
  • 32c5411631 Revert "Support using mmap when applying LoRA (#2095)" (#2206) master-32c5411 Howard Su 2023-07-13 21:58:25 +08:00
  • ff5d58faec Fix compile error on Windows CUDA (#2207) master-ff5d58f Howard Su 2023-07-13 21:58:09 +08:00
  • b782422a3e devops : add missing quotes to bash script (#2193) Bodo Graumann 2023-07-13 15:49:14 +02:00
  • 1cbf561466 metal : new q4_0 matrix-vector kernel (#2188) Shouzheng Liu 2023-07-12 16:10:55 -04:00
  • 975221e954 ggml : broadcast mul_mat + conv batch support (#2199) Georgi Gerganov 2023-07-12 20:51:29 +03:00
  • 4523d10d0c ggml : add ggml_pool_1d and ggml_pool_2d Georgi Gerganov 2023-07-12 20:27:03 +03:00
  • 680e6f9177 cuda : add gelu support Georgi Gerganov 2023-07-12 20:26:18 +03:00
  • 4e7464ef88 FP16 is supported in CM=6.0 (#2177) master-4e7464e Howard Su 2023-07-12 20:18:40 +08:00
  • 2b5eb72e10 Fixed __dp4a compute capability: 6.0 -> 6.1 (#2189) master-2b5eb72 Johannes Gäßler 2023-07-12 10:38:52 +02:00
  • f7d278faf3 ggml : revert CUDA broadcast changes from #2183 (#2191) master-f7d278f Georgi Gerganov 2023-07-12 10:54:19 +03:00
  • 20d7740a9b ggml : sync (abort callback, mul / add broadcast, fix alibi) (#2183) master-20d7740 Georgi Gerganov 2023-07-11 22:53:34 +03:00
  • 5bf2a27718 ggml : remove src0 and src1 from ggml_tensor and rename opt to src (#2178) master-5bf2a27 Spencer Sutton 2023-07-11 12:31:10 -04:00
  • c9c74b4e3f llama : add classifier-free guidance (#2135) master-c9c74b4 Bach Le 2023-07-12 00:18:43 +08:00
  • 3ec7e596b2 docker : add '--server' option (#2174) Jinwoo Jeong 2023-07-12 01:12:35 +09:00
  • 917831c63a readme : fix zig build instructions (#2171) Chad Brewbaker 2023-07-11 11:03:06 -05:00
  • 2347463201 Support using mmap when applying LoRA (#2095) master-2347463 Howard Su 2023-07-11 22:37:01 +08:00
  • bbef28218f Possible solution to allow K-quants on models with n_vocab!=32000 (#2148) master-bbef282 LostRuins 2023-07-11 22:01:08 +08:00
  • 5656d10599 mpi : add support for distributed inference via MPI (#2099) master-5656d10 Evan Miller 2023-07-10 11:49:56 -04:00
  • 0492363137 mpi : fix after master merge refactor-mpi Georgi Gerganov 2023-07-09 22:23:04 +03:00
  • 81c5ddd532 Merge branch 'mpi' into refactor-mpi Georgi Gerganov 2023-07-09 22:20:14 +03:00
  • 03cc12be0d [mpi] continue-on-error: true Evan Miller 2023-07-09 15:10:43 -04:00
  • 4a9a4748e9 Add OpenMPI to GH action Evan Miller 2023-07-09 15:05:58 -04:00
  • 0f557c2ac4 Merge branch 'master' into mpi Evan Miller 2023-07-09 15:02:19 -04:00
  • 9da9d26c70 mpi : minor Georgi Gerganov 2023-07-09 18:38:32 +03:00
  • beadbf3380 mpi : fix inference Georgi Gerganov 2023-07-09 18:26:20 +03:00
  • ef37dd14e7 mpi : fix output tensor after MPI compute (still not working) Georgi Gerganov 2023-07-09 17:01:08 +03:00
  • c717c5185f mpi : various fixes - communication now works but results are wrong Georgi Gerganov 2023-07-09 16:40:16 +03:00
  • 01abb3b3b9 mpi : move all MPI logic into ggml-mpi Georgi Gerganov 2023-07-09 16:04:27 +03:00
  • e339d35579 mpi : add names for layer inputs + prep ggml_mpi_graph_compute() Georgi Gerganov 2023-07-09 14:42:36 +03:00
  • 3232db628c mpi : trying to move more MPI stuff into ggml-mpi (WIP) (#2099) Georgi Gerganov 2023-07-09 14:08:53 +03:00
  • 1d16309969 llama : remove "first token must be BOS" restriction (#2153) master-1d16309 oobabooga 2023-07-09 05:59:53 -03:00
  • db4047ad5c main : escape prompt prefix/suffix (#2151) master-db4047a Nigel Bosch 2023-07-09 03:56:18 -05:00
  • 18780e0a5e readme : update Termux instructions (#2147) JackJollimore 2023-07-09 05:20:43 -03:00
  • 3bbc1a11f0 ggml : fix buidling with Intel MKL but ask for "cblas.h" issue (#2104) (#2115) master-3bbc1a1 clyang 2023-07-09 16:12:20 +08:00
  • 2492a53fd0 readme : add more docs indexes (#2127) rankaiyx 2023-07-09 15:38:42 +08:00
  • 64639555ff Fixed OpenLLaMA 3b CUDA mul_mat_vec_q (#2144) master-6463955 Johannes Gäßler 2023-07-08 20:01:44 +02:00
  • e344540620 ci: add linux binaries to release build ci_cublas_linux-e344540 Green Sky 2023-05-05 00:01:30 +02:00
  • 061f5f8d21 CUDA: add __restrict__ to mul mat vec kernels (#2140) master-061f5f8 Johannes Gäßler 2023-07-08 00:25:15 +02:00
  • 84525e7962 docker : add support for CUDA in docker (#1461) master-84525e7 dylan 2023-07-07 11:25:25 -07:00
  • a7e20edf22 ci : switch threads to 1 (#2138) master-a7e20ed Georgi Gerganov 2023-07-07 21:23:57 +03:00
  • 1d656d6360 ggml : change ggml_graph_compute() API to not require context (#1999) Qingyou Meng 2023-07-08 00:24:01 +08:00
  • 7242140283 ggml : remove sched_yield() call in ggml_graph_compute_thread() (#2134) master-7242140 Georgi Gerganov 2023-07-07 18:36:37 +03:00
  • 3e08ae99ce convert.py: add mapping for safetensors bf16 (#1598) Aarni Koskela 2023-07-07 16:12:49 +03:00
  • ef61acfbf5 Add info to README Evan Miller 2023-07-07 09:02:23 -04:00
  • 481f793acc Fix opencl by wrap #if-else-endif with \n (#2086) master-481f793 Howard Su 2023-07-07 11:34:18 +08:00
  • 55207ba2b8 Add GH workflow, fix test Evan Miller 2023-07-06 21:40:18 -04:00
  • 1f0a2cfeda Update CMakeLists.txt Evan Miller 2023-07-06 21:25:34 -04:00
  • 06a239343c PR comments Evan Miller 2023-07-06 20:18:41 -04:00
  • 32deabfdc8 Merge branch 'master' into mpi Evan Miller 2023-07-06 19:04:50 -04:00