With GGML_BACKEND_DL=ON no backend is registered until something calls
ggml_backend_load_all(). Every example does; the tests never did, so
whisper_init_* and parakeet_init_* ran with devices = 0 and aborted on
GGML_ASSERT(device) in ggml_backend_dev_backend_reg.
Three tests aborted on such a build - test-whisper-zero-samples,
test-vad and test-parakeet - taking ctest -L gh to 2 of 4.
test-vad-full has the same fault and reaches it only once a real
base.en model is present.
No workflow caught this because none pairs the two: build-clang,
build-gcc and build-sanitize run ctest -L gh but never set
GGML_BACKEND_DL, while release.yml sets it and runs no tests.
ggml_backend_load_all() is already reachable through whisper.h and
parakeet.h via ggml-cpu.h, so no new include is needed.
Resolves: https://github.com/ggml-org/whisper.cpp/issues/4030
* added check for nullptr for wctx
Signed-off-by: GodRishUniverse <risagarw@amd.com>
* Apply the update for failed message in tests/test-vad-full.cpp
Co-authored-by: Daniel Bevenius <daniel.bevenius@gmail.com>
---------
Signed-off-by: GodRishUniverse <risagarw@amd.com>
Co-authored-by: Daniel Bevenius <daniel.bevenius@gmail.com>
When transcribing with params.vad = true, whisper already computes the speech
segments and keeps them in the state. Expose them so callers can reuse those
boundaries (for example to align or clip subtitles to speech) instead of running
a second, separate VAD pass.
Times are on the original audio timeline in centiseconds; the count is 0 when VAD
was not used. test-vad-full.cpp checks the segments are ordered and non-empty.
whisper_full_get_token_data().t0/t1 are in the VAD-processed timeline (silences
removed), so they don't line up with the original audio. Add
whisper_full_get_token_t0/t1 that map them back.
A token inside a speech segment is interpolated within it; a token that falls in a
removed inter-segment silence snaps to the nearest boundary, so it never lands in
the middle of a cut-out gap. Without VAD the raw times are returned unchanged.
* tests : update VAD tests to use Silero V6.2.0
This commit updates the VAD tests to use the Silero V6.2.0 instead of
V5.1.2. I'm was not sure if we needed to keep testing for both versions,
but opted to just update to the latest version for simplicity.
* wasm : use C++17 for emscripten builds
This commit updates the CMakeLists.txt file to explicitly set the C++
standard to C++17 when building with Emscripten.
The motivation for this change is that building with Emscripten
will currently fail locally and on CI with the following error:
```console
[ 75%] Building CXX object examples/CMakeFiles/common.dir/common-ggml.cpp.o
In file included from /home/danbev/work/ai/whisper.cpp/examples/stream.wasm/emscripten.cpp:5:
/home/danbev/work/utils/emsdk/upstream/emscripten/cache/sysroot/include/emscripten/bind.h:11:2: error:
"embind requires -std=c++17 or newer"
11 | #error "embind requires -std=c++17 or newer"
| ^
In file included from /home/danbev/work/ai/whisper.cpp/examples/whisper.wasm/emscripten.cpp:4:
/home/danbev/work/utils/emsdk/upstream/emscripten/cache/sysroot/include/emscripten/bind.h:11:2: error:
"embind requires -std=c++17 or newer"
11 | #error "embind requires -std=c++17 or newer"
| ^
```
This commit modifies the test-vad and test-vad-full tests to use CMake
definitions for the model and sample paths.
The motivation for this is that currently the tests use relative paths
which might not always be correct depending on the working directory.
With the changes in this commit the tests can be run usins ctest:
```console
$ ctest -R ^test-vad$ --test-dir build
```
Or directly (which is not currently possible without this fix):
```
./build/bin/test-vad
```
Resolves: https://github.com/ggml-org/whisper.cpp/issues/3404
* vad : add initial Voice Activity Detection (VAD) support
This commit add support for Voice Activity Detection (VAD). When enabled
this feature will process the audio input and detect speech segments.
This information is then used to reduce the number of samples that need
to be processed by whisper_full.
Resolves: https://github.com/ggml-org/whisper.cpp/issues/3003
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Abir Deol