upstream-archive/whisper.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/whisper.cpp.git synced 2026-07-31 15:11:29 -05:00
Code Issues Packages Projects Releases Wiki Activity
548 Commits 65 Branches 87 Tags
v1.3.0
Commit Graph

135 Commits

Author SHA1 Message Date
Georgi Gerganov
5108b30e6d whisper : pad audio instead of spectrogram (#579) 
Also, fallback only if more temperatures are available and if we are
at least 3 seconds before the end of the audio
 2023-04-15 17:19:19 +03:00
Georgi Gerganov
f19e23fbd1 whisper : restore decoder temperature fallbacks 
I disabled this because there were many complaints about slow decoding.
The current implementation does not allow batching the decoders when
using the "best of" or "beam size" parameters, so the decoding time is
proportional to the number of decoders, which is obviously not great.

However, now there are even more complaints about wrong decodings and
repetition.

So, making a compromise by re-enabling the fallbacks, but defaulting to
just 2 "best of" / "beam size" decoders. Also, the temperature step is
increased from 0.2 to 0.4 - i.e. from maximum of 5 fallbacks to maximum
of 2.

Also, the stream example now has fallbacks enabled by default.

close #471 #477 #508 #612 #719 #731
 2023-04-15 16:12:55 +03:00
Georgi Gerganov
3dead611bb whisper : slightly faster Log Mel computation + n-1 FFT threads (#568) 2023-04-15 14:18:46 +03:00
Georgi Gerganov
5e47e223bd whisper : add Core ML support (#566) 
* coreml : use Core ML encoder inference

* coreml : simlpify whisper_encode + log messages

* whisper : resolve rebase conflicts

* coreml : add scripts for CoreML model generation

* bench-all : recognize COREML flag
 2023-04-15 13:21:27 +03:00
Maximiliano Levi
794ff3074a whisper : do not launch log_mel threads when n_thread is 1 (#763) 2023-04-14 22:35:34 +03:00
AfryMask
7e2afa4384 whisper : fix the bug related to word splitting errors in the "tokenize" function. (#760) 
Co-authored-by: AfryMask <afrymask@gmail.com>
 2023-04-14 20:35:03 +03:00
Bader-eddine Ouaich
2c856fb9e5 whisper : fix potential memory leaks (#740) 
* fix potential memory leak if whisper_init_state failed

* fix potential memory leak if gpt2_init failed
 2023-04-14 20:05:56 +03:00
Georgi Gerganov
514cd04452 whisper : fix bug in prompt processing (close #705) 
Was dereferencing a dangling pointer
 2023-04-14 19:17:07 +03:00
Georgi Gerganov
69b8503935 ggml : backport llama.cpp updates (close #709) 
- About x2 overall performance improvement on Apple Silicon
- Results should now be the same for different number of threads (not
  tested)
 2023-04-10 22:28:54 +03:00
pajowu
0a2d1210bc whisper : add progress callback (#600) 2023-03-30 20:29:29 +03:00
Jhen-Jie Hong
eefed45e37 whisper : add initial_prompt param (#645) 2023-03-29 23:23:23 +03:00
Georgi Gerganov
42c6855103 whisper : bump "large" scratch buffer even mode (close #671) 2023-03-28 10:50:49 +03:00
Georgi Gerganov
0be9cd3497 whisper : increase scratch buffers after recent change (#671) 
Should fix the error:

ggml_new_tensor_impl: not enough space in the scratch memory
 2023-03-28 10:36:16 +03:00
Georgi Gerganov
4a0deb8b1e talk-llama : add new example + sync ggml from llama.cpp (#664) 
* talk-llama : talk with LLaMA AI

* talk.llama : disable EOS token

* talk-llama : add README instructions

* ggml : fix build in debug
 2023-03-27 21:00:32 +03:00
Georgi Gerganov
8e361d90d7 whisper : disable fallbacks until the performance is improved (#588) 2023-03-22 22:34:39 +02:00
sandrohanea
d4fa0d92ad fixed language auto-detection for state provided processing (#627) 
Co-authored-by: Sandro Hanea <sandrohanea@microsoft.com>
 2023-03-22 21:47:09 +02:00
Leo Moll
8fcd1a3b32 main : provide option for creating JSON output (#615) 
* examples : provide option for exporting also as JSON file (ggerganov/whisper.cpp#614)

* main : remove leftovers

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-03-22 21:37:36 +02:00
Georgi Gerganov
1beff6f66d models : change HF hosting from dataset to model 2023-03-22 20:44:56 +02:00
Takeshi Inoue
09e9068007 whisper.android : support benchmark for Android example. (#542) 
* whisper.android: Support benchmark for Android example.

* whisper.android: update screenshot in README.

* update: Make text selectable for copy & paste.

* Update whisper.h to restore API name

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* whisper.android: Restore original API names.

---------

Co-authored-by: tinoue <tinoue@xevo.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-03-07 21:36:30 +02:00
sandrohanea
59fdcd19c8 whisper : add whisper_state + default state on the whisper_context (#523) 
* Added whisper state + default state on the whisper_context

* Fixed some examples and bindings

* Fixed whisper_n_len (which was used in some binding) and added whisper_n_len_from_state

* Fixed comments

* whisper : reuse kv_cache_free() and fix compiler warnings

* whisper : clean-up the API comments

---------

Co-authored-by: Sandro Hanea <sandrohanea@microsoft.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-03-05 21:42:19 +02:00
Georgi Gerganov
478289a4b3 whisper : set no_context == true by default (#537) 2023-03-05 20:53:43 +02:00
Georgi Gerganov
373043cabe whisper : zero-initialize some more context variables 
Just in case
 2023-02-21 19:00:42 +02:00
Finn Voorhees
fb4d0d470f whisper : fix uninitialized exp_n_audio_ctx 2023-02-21 18:58:08 +02:00
Georgi Gerganov
0d229163bb whisper : add API for applying custom logits filters during decoding 2023-02-19 18:35:01 +02:00
Georgi Gerganov
a94897bcde whisper : by default disable non-speech tokens suppression (#473) 
This seems to be causing hallucinations in the end of the audio, e.g.:

"Thank you for listening"
"Amen"
..
 2023-02-15 21:48:49 +02:00
shikokuchuo
0336161b7d whisper : fix signedness compiler warning (#506) 2023-02-15 19:08:25 +02:00
shibukazu
cfc06bf8df whisper : suppress non-speech-related token outputs (#473) 
* add non-speech-token suppression

* add suppress non-speech_tokens param
 2023-02-08 09:05:34 +02:00
sandrohanea
2bfe0ebc0f whisper : fixed Beam Search Strategy and exposed whisper_pcm_to_mel_phase_vocoder (#474) 
Co-authored-by: Sandro Hanea <sandrohanea@microsoft.com>
 2023-02-08 09:01:47 +02:00
boolemancer
4dd7119deb whisper : only trim if split_on_word is true (#476) 2023-02-08 08:43:23 +02:00
kamranjon
a1c1583cc7 whisper : add whisper_full_lang_id() for getting the context lang (#461) 2023-02-05 14:46:26 +02:00
Matija Pevec
d012b5c7e4 whisper : add "split_on_word" flag when using using "max_len" option (#455) 
* Update whisper.cpp

* fix: trim function

* feat: added flag to split on word

* fix: arguments for main
 2023-02-05 14:44:23 +02:00
Georgi Gerganov
f3ee4a9673 whisper : reduce memory usage during inference (#431) 
* ggml : add "scratch" buffer support

* ggml : support for scratch ring-buffer

* ggml : bug fix in ggml_repeat()

* ggml : error on scratch buffer overflow

* whisper : use scratch buffers during inference (base model only)

* whisper : update memory usage for all models

* whisper : fix encoder memory usage

* whisper : use whisper_context functions instead of macros

* whisper : fix FF + remove it from README

* ggml : reuse ggml_new_i32

* ggml : refactor the scratch buffer storage

* whisper : reorder scratch buffers in the decoder

* main : add option to disable temp fallback

* Update README.md
 2023-02-04 09:45:52 +02:00
Georgi Gerganov
291980369c whisper : suppress task tokens (#442) 2023-02-04 09:03:14 +02:00
Georgi Gerganov
b992f3709e whisper : do not provide past prompt when n_max_text_ctx == 0 2023-01-25 20:01:00 +02:00
Georgi Gerganov
b5ddb16ec7 whisper : condition timestamps to be monotonically increasing (#425) 2023-01-23 20:48:26 +02:00
fitzsim
ae16c21e9c whisper : PPC64 big-endian support (#398) 
* ggml : set cache line size to 128 on POWER9

* whisper : add PPC64 big endian support
 2023-01-23 20:48:10 +02:00
Georgi Gerganov
78f166174f whisper : fix condition for providing past prompt (critical) 
This bug has been present since v1.1.0.

Effectively, the past transcribed text wasn't being used for following
transcriptions, which likely significantly reduces the transcription
quality.

Likely related to #419
 2023-01-22 10:47:01 +02:00
Georgi Gerganov
21c569ba4a whisper : extend information in whisper_print_timings() 2023-01-19 18:50:33 +02:00
Georgi Gerganov
1a91c19af9 whisper : perform entropy check only when we have at least 32 tokens (#412) 2023-01-18 22:52:18 +02:00
Georgi Gerganov
a6cf6f4c4a bench : minor fixes 2023-01-18 21:40:10 +02:00
Georgi Gerganov
1ccb8a46a5 bench : fix Windows linkage by moving ggml benches in whisper lib .. 2023-01-18 21:19:50 +02:00
Georgi Gerganov
8088a977af whisper : fix possible uninitialized variables (#291) 2023-01-16 21:44:40 +02:00
Georgi Gerganov
00ea21668b whisper : account speed_up flag for short audio (close #405) 2023-01-15 12:42:15 +02:00
Georgi Gerganov
8de452c18b Improve decoding (#291) 
* whisper : prepare infra for new decoding strategies

* whisper : apply logit filters and compute logprobs

* whisper : add whisper_get_logits()

* whisper : separate self and cross attention memory

Initial step needed for supporting parallel decoders

* whisper : move probs_id buffer to whisper_context

* whisper : refactor kv cache into separate struct

* whisper : move self-attention kv cache to whisper_decoder

* whisper : wip decoding parameters + strategies

* whisper : wip decoding parameters + strategies (part 2)

* whisper : wip decoding parameters + strategies (part 3)

* whisper : wip decoding parameters + strategies (part 4)

* whisper : fix prompt_past update to not include prompt_init

* whisper : temperature + best_of support

* whisper : support for compression_ration_threshold

We actually use entropy, but it is similar

* command : fix example to use logits instead of obsolete probs

* whisper : handle empty sequence ranking

* whisper : add WHISPER_DEBUG + diagnostic prints + new main args

* whisper : minor fixes

* whisper : add beam-search support

* whisper : bug fix when there no previous context

* whisper : add comments

* stream : disable temperature fallback

For real-time processing, we always want a single decoder running at T=0

* whisper.swiftui : update example - fix paths + add empty folders
 2023-01-15 11:29:57 +02:00
Georgi Gerganov
4ef3398e8f ggml : remove obsolete zeroing + comment fixes (#390) 2023-01-08 20:21:03 +02:00
boolemancer
08dc705a69 whisper : fix sample_to_timestamp calculation with 64 bit precision to avoid overflow (#388) 
* Do calculation with 64 bit precision to avoid overflow

* Update whisper.cpp

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-01-08 15:08:45 +02:00
Syahmi Azhar
1512545149 whisper : add loader class to allow loading from buffer and others (#353) 
* whisper : add loader to allow loading from other than file

* whisper : rename whisper_init to whisper_init_from_file

* whisper : add whisper_init_from_buffer

* android : Delete local.properties

* android : load models directly from assets

* whisper : adding <stddef.h> needed for size_t + code style

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-01-08 13:03:33 +02:00
Georgi Gerganov
65fdcbbbbb whisper : revert accidental MB change 2023-01-07 16:18:21 +02:00
Georgi Gerganov
d61d55cd4b ggml : speed-up soft max via Accelerate + unroll 2023-01-07 16:16:42 +02:00
Abitofevrything
a62170c656 ggml : add SSE3 and fp16 conversion lookup table (#368) 
* Improves WASM performance:
  On MacBook M1 Pro, I observe 25% faster using Firefox and 35% faster using Chrome

* Add support for SSE3 SIMD

* Add SSE3 to system information

* Add Imath support for fp16-fp32 conversions

* Add Imath to system information

* Wrap Imath calls to avoid static function warnings

* Drop Imath; Add lookup table for f16 -> f32 conversions

* Remove TODO comments

* Update SSE3 to new macro arguments

* Correct updated macro definitions

* Prefer static inline where possible

* ggml : static inlines + add public f16 <-> f32 conversions

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-01-06 18:45:59 +02:00