* whisper : optional ANEForge encoder backend (Apple Neural Engine)
Runs the Whisper encoder directly on the Apple Neural Engine via ANEForge, an
alternative to the CoreML encoder backend. About 2x faster than the CoreML encoder
and faster than the Metal GPU encoder on M-series, at ~5x lower energy, with the
same transcripts (cosine 0.999 vs the reference encoder).
Mirrors the existing CoreML/OpenVINO seam: fills embd_enc from the external aneforge
package via the same whisper_encode_external path, gated at runtime by the
ANEFORGE_ENCODER env var. With the variable unset the build behaves exactly like
stock whisper.cpp (the backend dlopens its dispatch dylib only when enabled, no new
link-time dependency). Encoder only; the decoder is untouched.
Adds src/aneforge/whisper-aneforge.{h,cpp}, one line to src/CMakeLists.txt, and ~30
lines to src/whisper.cpp (include, state field, encode branch, init, free).
* aneforge: one statement per line, align dlsym block (review)
* aneforge: guard the backend to Apple Silicon, stub elsewhere (fix non-Apple CI)
whisper-aneforge.cpp uses <dlfcn.h> and __fp16, which are unavailable on Windows
(MSVC) and non-ARM Linux, so the unconditional build broke those CI jobs. Wrap the
real implementation in #if defined(__APPLE__) && defined(__aarch64__) and provide
no-op stubs elsewhere; the encoder is Apple-Neural-Engine-only anyway, and the
library now links on every platform.
* examples : include whisper-aneforge.cpp in android examples
Refs: https://github.com/ggml-org/whisper.cpp/actions/runs/32758607709/job/97689705921?pr=3905#step:5:200
* ci : specify explicit packages (platform-tools)
This commit updates the android jobs to specify platform-tools as an
explicit package.
The motivation for this to try to fix the Setup Android SDK step which
is also trying to install an obolete tools package.
Refs: https://github.com/ggml-org/whisper.cpp/actions/runs/35215042978/job/105181619536?pr=3905
---------
Co-authored-by: Daniel Bevenius <daniel.bevenius@gmail.com>
This commit renames the cublas jobs and release artifacts to just use
cuda instead of cublas.
The motivation for this is that I think that the cublas naming came for
a time when the ggml-cuda backend mostly or perhaps completely delegated
to cublas. But this is not the case any more and naming this cuda would
make sense now. This is also consistent with how llama.cpp names their
cuda binary build artifacts.
* ci : add Windows On ARM support to release job (wip)
This commit adds Windows On Arm (WoA) support to whisper.cpp and the
release process.
The underlying support was already in place as this had been synced
with ggml, but the missing part was producesing release artifacts which
is what this commit does.
* ci : add note about CUDA WoA is a preview edition [no ci]
This commit updates the close-issue CI job to not automatically close
issues. It also adds a stale lable message that explains why the issue
was marked as stale.
The motivation for this is that we enabled this job recently and there
have been a few comments from users/reporters that they did not get any
notification or motivation for closing. Hopefully marking the issues as
stale will give a notification the the reporters and we can manually
look through stale issues.
Refs: https://github.com/ggml-org/whisper.cpp/issues/586#issuecomment-5579674875
This commit moves the close-issue.yml file to the .github/workflows/
directory.
The motivation for this is that it is currently in the root of the repo
and is not currently active. Moving it to the workflows directory will
make it active and allow it to run.
* cmake : update semver handling to be consistent with ggml/llama.cpp
This commit modifies the semantic version handling to be consistent with
how llama.cpp and ggml handle semver.
This commit introdues a new example named test-cmake which is intended
to be used to test the cmake configuration and installation.
* ci : update release workflow to be consistent with llama.cpp
work in progress...
* ci : fix if statement in release.yml
* ci : comment out all but one build in release.yml
This is just for testing and this commit should not be included in the
main PR later.
* ci : use DEPLOY_KEY_RELEASE
This commit updates the release and make-release workflows to use the
DEPLOY_KEY_RELEASE secret. Two github ruleset have been imported.
* ci : add github rulesets for releases
These were retrived from llama.cpp and then imported into my fork for
testing. If all works well they will be imported into whisper.cpp
upstream as well.
* fix move artifacts step
* examples : use FetchContent for llama.cpp in talk-llama
This commit updated the example talk-llama to remove the vendored
llama.cpp and instead use FetchContent to pull it in from the
upstream repo.
* ci: add GGML_NATIVE=OFF to build-clang.yml
This commit disables native CPU instructions from the ubuntu-22-clang
job.
The motivation for this is that currently it is possible that the
running compiling llama.cpp (via ccache) might have support for cpu
instructions that are not available on the target runner.
Refs: https://github.com/ggml-org/whisper.cpp/actions/runs/32224048267/job/95980031403?pr=3996
* ci : add missing GGML_NATIVE=OFF to jobs
* ci : add attestation for signed release artifacts
This commit add attenstions of artifacts to the release workflow.
After building the artifacts can be verified with the following command:
```console
$ curl -sSL -o whisper-bin-ubuntu-x64.tar.gz \
https://github.com/danbev/whisper.cpp/releases/download/b4947/whisper-bin-ubuntu-x64.tar.gz
$ gh attestation verify --repo danbev/whisper.cpp whisper-bin-ubuntu-x64.tar.gz
Loaded digest sha256:722a6812263195d7ee2192b57fc64a6d6b09a6cdf2f55a152f793db27a651e31 for file://whisper-bin-ubuntu-x64.tar.gz
Loaded 1 attestation from GitHub API
The following policy criteria will be enforced:
- Predicate type must match:................ https://slsa.dev/provenance/v1
- Source Repository Owner URI must match:... https://github.com/danbev
- Source Repository URI must match:......... https://github.com/danbev/whisper.cpp
- Subject Alternative Name must match regex: (?i)^https://github\.com/danbev/whisper\.cpp/
- OIDC Issuer must match:................... https://token.actions.githubusercontent.com
✓ Verification succeeded!
The following 1 attestation matched the policy criteria
- Attestation #1
- Build repo:..... danbev/whisper.cpp
- Build workflow:. .github/workflows/release.yml@refs/heads/master
- Signer repo:.... danbev/whisper.cpp
- Signer workflow: .github/workflows/release.yml@refs/heads/master
```
* cmake : add WHISPER_USE_SYSTEM_LLAMA option [no ci]
This commit adds a new CMake option WHISPER_USE_SYSTEM_LLAMA that allows
the talk-llama example to use a system-installed llama.cpp library.
Setting this will automatically also set WHISPER_USE_SYSTEM_GGML to ON
and the system ggml library will be used in addition to the system
llama.cpp.
* ci : remove unused ccache step
* Revert "ci : comment out all but one build in release.yml"
This reverts commit 24b56776e1.
* ci : set WHISPER_BUILD_IS_DEV=OFF in release.yml
This commit adds the cmake option GGML_NATIVE=OFF to the build-vad
workflow to avoid the situation where the ccache is populated with
object files containing instructions may not be supported by all
runners.
The motivation for this is that the test-vad is currently failing
(depending on the runner assigned) with the linked error below.
Refs: https://github.com/ggml-org/whisper.cpp/actions/runs/30625192129/job/91600926109?pr=3963
* ci, examples : add package-lock.json to examples addon node
This commit commits the package-lock.json file for examples/addon.node.
This is needed to make the CI workflow work properly and after we
started pinning the action versions in the workflow, the CI started
failing because the package-lock.json was not committed. At least that
is what I think is happening.
* ci: use node 20.x and 22.x instead of 16.x and 18.x
* fix CMakeLists.txt target properties for addon.node
* ci : add GGML_NATIVE=OFF and build all cpu-variants
This commit adds -DGGML_BACKEND_DL=ON, -DGGML_NATIVE=OFF, and
-DGGML_CPU_ALL_VARIANTS=ON to the releases.
The motivation for this is that currently the Windows BLAS build
uses the native CPU instructions and if target systems do not support
these instructions, the build will fail like the linked issue reports.
Resolves: https://github.com/ggml-org/whisper.cpp/issues/3889
* ci : update ubuntu-cpu release job for all variants [no ci]
This commit enables the ubuntu-cpu job to include all cpu variants and
ensures that the ggml backend libraries are built into the bin directory
similar to how llama.cpp does it.
The following is a build on my fork with this change:
https://github.com/danbev/whisper.cpp/releases/tag/untagged-fc3c71f0bf0f7bf19d19
* Add Whisper::Parakeet::Params
* Add tests for Parakeet::Params
* Remove unused variabel
* Add callbacks to Parakeet::Params
* Group callback and user_data params
* Undefine local macros
* Define GetParakeetParams
* Remove unused variable
* Use ITERATE_CALLBACK_PARAMS
* Use ITERATE_CALLBACK_PARAMS instead of ITERATE_USER_DATA_PARAMS
* Fix memsize
* Remove unnecessary macros
* Simplify params registration
* Define Parakeet
* Add hook methods to Parakeet::Params
* Fix typo
* Check callback container in GetParakeetParams
* Reduce if
* Free parakeet_full_params
* Implement Parakeet::Context#initialize
* Add TestParakeetContext
* Add Parakeet::Segment
* Prevent double-free
* Add Parakeet::Context#transcribe
* Add Parakeet::Context#each_segment
* Define Parakeet::Segment attributes
* Define Parakeet::Segment#deconstruct_keys
* Add tests for Parakeet::Segment#deconstruct_keys
* Run Parakeet::Context#transcribe without GVL
* Make it to abort for Parakeet
* Add Parakeet.log_set
* Define Parakeet::Token
* Define Parakeet::Segment#each_token
* Implement some hooks of Parakeet::Params
* Convert int to VALUE
* Implement hooks for Parakeet
* Implement Parakeet::Context#full
* Add tests for Parakeet::Context#full
* Add Parakeet to RBS
* Fix ruby_whisper_parakeet_params_free
* Free ruby_whisper_parakeet_context
* Add tests for hooks
* Add Parakeet section to README
* Add more attributes of Parakeet::Context
* Add tests for Parakeet::Context's attributes
* Update RBS
* Register parakeet-tdt-0.6b-v3
* Narrow scope of log constants
* Extract activate and deactivate of log_queue
* Make start_log_callback_thread private
* Don't call start_log_callback_thread unncecessarilly
* Early return from log_queue_enqueue when not active
* Gropu log_queue members
* is_active -> is_open
* Fix English
* Share parakeet full body function
* ruby_whisper_parakeet_abort_callback_user_data -> ruby_whisper_abort_callback_user_data
* NULL check for callback containers
* Fix Parakeet.log_set
* Omit Parakeet tests on CI
* Extract Whisper::LogSettable
* Join log callback thread in a log queue function
* Revert Join log callback thread in a log queue function
* Extract output methods to modules
* Move Parakeet init functions into init_parakeet()
* Add output methods to Parakeet classes
* Add Parakeet's output methods to RBS
* Use Whisper::Output in RBS
* Add LogSettable to RBS
* Fix module Token -> class Token
* Add Parakeet::Model
* Add test for Parakeet::Model
* Add Parakeet::Model to RBS
* Move position of Parakeet::Model in RBS
* Parakeet -> TestBase::Parakeet
* Add Parakeet::Context#model in RBS
* Add Whisper::Output
* Fix nil check
* Define ruby_whisper_parakeet_model_memsize
* Fix order of declaration in ruby_whisper_parakeet_model_get_xxx
* Define Parakeet.system_info_str
* Add test for Parakeet.system_info_str
* Add signature of Parakeet.system_info_str
* Define Parakeet::VERSION
* Add test for Parakeet::VERSION
* Add signature of Parakeet::VERSION
* Add Parakeet::Context::Params
* Make Parakeet::Context.new accept Context::Params
* Add test for Parakeet::Context.new with Context::Params
* Update RBS
* Remove params from Parakeet::Params which are moved from whisper_parakeet_full_params
* Remove tests for removed params
* Make Parakeet tests follow original behavior changes
* Add Parakeet model shortcuts
* Alloc token data in factory instead of alloc func
* Fix variable name
* Update RBS
* Refactor log settable module
* Use log settable for Whisper
* Address deadlock
* Make test follow change of log queue implementation
* Refactor to make abort callback use the same way to parakeet's way
* Remove redundant structs
* Fix test name
* Fix README
* Add missing parallel transcription
* Fix test for parakeet info
* Remove removed params
* Wait for logs dequeued
* Fix instance variable name
* Load etc feature
* Remove unnecessary comment
* Remove unnecessary thread safety check
* Remove outdated comment
* Skip downloading model if cache exists
* Change Hugging Face URI for Parakeet models
* Bump required Ruby version to 3.3
* Fix English
* ci : only trigger release jobs for tags
This commit removes the building of the release jobs on pushed to
master.
The motivation for this is that it can be confusing at the momement when
releasing that the push to master also triggers the release jobs but
the actual release will be skipped. With this change the release job is
only run when a tag is pushed which should result in a single Release
github actions job and make it easier to follow.
* ci : add GGML_NATIVE=OFF for ubuntu-22-gcc
* ci : remove base-devel and git from msys2 job
This commit removes the above packages as they might not be required and
could help reduce the github cache size.
* ci : try reducing the installs to only the compilers
This commit updates the setup emscripten sdk jobs to use emscripten-core
instead of mymindstorm and also pins the commit sha for the version
instead of using a version tag.
This commit updates the Install cache step to use ggml-org/ccache-action
and switched to use ccache instead of sccache.
The motivation for switching to ccache is that this is what llama.cpp
does and also there is an issue with later version of sscache:
```console
sccache C:\PROGRA~1\NVIDIA~1\CUDA\v\bin\nvcc.exe -forward-unknown-to-host-compiler -DGGML_BACKEND_BUILD -DGGML_BACKEND_SHARED -DGGML_CUDA_PEER_MAX_BATCH_SIZE=128 -DGGML_SCHED_MAX_COPIES=4 -DGGML_SHARED -D_CRT_SECURE_NO_WARNINGS -D_XOPEN_SOURCE=600 -Dggml_cuda_EXPORTS -DCMAKE_INTDIR=\"Release\" -ID:\a\whisper.cpp\whisper.cpp\ggml\src\ggml-cuda\.. -ID:\a\whisper.cpp\whisper.cpp\ggml\src\..\include -isystem "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v\include" -Xcompiler="-MD -O2 -Ob2" -DNDEBUG -std=c++17 -arch=native -use_fast_math -extended-lambda -Xcompiler /Zc:preprocessor -MD -MT ggml\src\ggml-cuda\CMakeFiles\ggml-cuda.dir\Release\allreduce.cu.obj -MF ggml\src\ggml-cuda\CMakeFiles\ggml-cuda.dir\Release\allreduce.cu.obj.d -x cu -c D:\a\whisper.cpp\whisper.cpp\ggml\src\ggml-cuda\allreduce.cu -o ggml\src\ggml-cuda\CMakeFiles\ggml-cuda.dir\Release\allreduce.cu.obj -Xcompiler=-Fdggml\src\ggml-cuda\CMakeFiles\ggml-cuda.dir\Release\,-FS
sccache: encountered fatal error
sccache: error: Could not parse shell line
sccache: caused by: Could not parse shell line
```
```
* ci : add ccache clear action
* ci : split self-hosted GPU jobs into build-self-hosted.yml
Extract self-hosted runner jobs from build.yml into a dedicated
build-self-hosted.yml following the llama.cpp pattern:
- gpu-cuda (NVIDIA Linux)
- gpu-vulkan-nvidia-cm (NVIDIA Linux)
- gpu-vulkan-nvidia-cm2 (NVIDIA Linux + COOPMAT2)
- gpu-metal (macOS ARM64)
- gpu-vulkan (macOS ARM64)
GitHub-hosted CPU jobs remain in build.yml.
Assisted-by: llama.cpp:local pi
* ci : split release jobs into release.yml
Extract release-related jobs from build.yml into a dedicated
release.yml following the llama.cpp pattern:
- determine-tag
- windows (Win32/x64, SDL2)
- windows-blas (Win32/x64, OpenBLAS)
- windows-cublas (x64, CUDA 11.8/12.4)
- ios-xcode-build
- bindings-java (depends on windows)
- release (artifact aggregation + GitHub release)
CoreML job stays in build.yml with its own local tag calculation.
Assisted-by: llama.cpp:local pi
* ci : remove bindings-java job from release.yml
Assisted-by: llama.cpp:local pi
* cont : add manual trigger for build.yml
* cont : remove obsolete ifs
* ci : extract sanitizer job to bild-sanitize.yml
* ci : extract linux jobs into build-linux.yml
* ci : extract macos jobs to build-macos.yml
* ci : extract gcc jobs to build-gcc.yml
* ci : extract clang jobs to build-clang.yml
* ci : extract sycl jobs to build-sycl.yml
* ci : extract windows jobs to build-windows.yml
* ci : extract emscripten job to build-wasm.yml
* ci : extract android jobs into build-android.yml
* ci : extract quantize job to quantize.yml
* ci : extract coreml job into coreml.yml
* ci : extract vad job to vad.yml
* ci : extract cpu jobs to build-cpu.yml
* ci : make naming of yml files consistent
* ci : add --fail to curl download and propagate
This commit adds the --fail option to the model download scripts so that
if the model download returns a server error this is picked up. This is
then detected in run.sh and a error message is displayed and the script
stops and returns an error.
The motivation for this is that currently it is possible for the model
download to fail but this script proceeds and instead of a model file
the contents will be an html page probably with the error. This will
then cause the model to not be able to load due to a missing magic
number. I'm not sure we can do much about the downloading failing,
perhaps a retry but at least this will give a clearer error message.
Refs: https://github.com/danbev/whisper.cpp/actions/runs/26866349389/job/79230794512
* ci : enable command traces to see download command in use
* ci : add retry functionality to download model script
This commit adds curl retry options to the model download script.
The motivation is that currently when CI jobs run huggingface rate limit
the requests and return:
```console
curl: (22) The requested URL returned error: 429
```
This is an attempt to work around this and if it does not work then we
can an authorization token.
* ci : extract freebsd job to build-freebsd.yml
This job has been commented out as it has been flaky in the past. I'll
monitor this and if it continues to be unreliable we can disable it in
the github actions GUI instead of commenting it out like we did before.
* ci : add ccache to jobs (non-docker builds)
The ccache will only be saved on pushed to master.
* ci : bump ccache-action version to v1.2.21
The motivation for this is that the save parameter does not seem to work
with the current version.
* ci : add ccache to docker jobs in build-linux.yml
* ci : add debug statements to linux docker build
* ci : set CCACHE_DIR for build-linux.yml
* ci : add ccache to the remaining docker jobs
* ci : remove build-linux.yml
This commit remove build-linux.yml as the same jobs are also run by
build-gcc.yml, with the exception that build-gcc.yml also run ctest).
So keeping build-gcc.yml and removing the redundant build-linux.yml.
* ci : add linux build artifacts to release
* ci : revert to hendrikmuhs/ccache-action for win job
This is currently causing the following failure:
```console
sccache C:\PROGRA~1\NVIDIA~1\CUDA\v\bin\nvcc.exe -forward-unknown-to-host-compiler -DGGML_BACKEND_BUILD -DGGML_BACKEND_SHARED -DGGML_CUDA_PEER_MAX_BATCH_SIZE=128 -DGGML_SCHED_MAX_COPIES=4 -DGGML_SHARED -D_CRT_SECURE_NO_WARNINGS -D_XOPEN_SOURCE=600 -Dggml_cuda_EXPORTS -DCMAKE_INTDIR=\"Release\" -ID:\a\whisper.cpp\whisper.cpp\ggml\src\ggml-cuda\.. -ID:\a\whisper.cpp\whisper.cpp\ggml\src\..\include -isystem "C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v\include" -Xcompiler="-MD -O2 -Ob2" -DNDEBUG -std=c++17 -arch=native -use_fast_math -extended-lambda -Xcompiler /Zc:preprocessor -MD -MT ggml\src\ggml-cuda\CMakeFiles\ggml-cuda.dir\Release\allreduce.cu.obj -MF ggml\src\ggml-cuda\CMakeFiles\ggml-cuda.dir\Release\allreduce.cu.obj.d -x cu -c D:\a\whisper.cpp\whisper.cpp\ggml\src\ggml-cuda\allreduce.cu -o ggml\src\ggml-cuda\CMakeFiles\ggml-cuda.dir\Release\allreduce.cu.obj -Xcompiler=-Fdggml\src\ggml-cuda\CMakeFiles\ggml-cuda.dir\Release\,-FS
sccache: encountered fatal error
sccache: error: Could not parse shell line
sccache: caused by: Could not parse shell line
```
Refs: https://github.com/danbev/whisper.cpp/actions/runs/26883673904/job/79290017353
* ci : make static linux artifacts
* ci : make linux release artifact names consistent
This commit removes the tag form the linux release artifacts to be
consistent with the existing artifacts.
If we want to include the tag then we can do that in a follow-up PR.
* ci : fix linux zip files to have a directory
* ci : add HF_TOKEN secret for HF download authorization
This is to avoid the HR rate limiting when downloading model.
---------
Co-authored-by: Daniel Bevenius <daniel.bevenius@gmail.com>
This commit adds an ignore for bindings-ruby and bindings-go in
build.yml as these are handled by separate .yml file (separate jobs)
and don't need to trigger a full CI build.
* ci : add on push/pull_request paths ruby job
This commit adds paths to bindings-ruby to only build if changes where
made to bindings/ruby or to include/whisper.h.
* ci : add additional paths [no ci]
This commit re-enables the arm64 docker images builds which were removed
in Commit 9366544991
("ci : fix arm builds"). It also uses ubuntu-24.04-arm as the runner
which enables us to avoid QEMU.
Resolves: https://github.com/ggml-org/whisper.cpp/issues/2859
* ci : set GGML_NATIVE=OFF for bindings-java
This commit attempts to address an issue with the bindings-java job
which is currently failing.
I've not been able to reproduce this locally my windows machine and I
suspect that what might be happning is that windows job compiles on a
runner where it has different CPU features, for example AVX512 and when
this dll is used on a different runner that does not have that feature
it will crash.
Refs: https://github.com/ggml-org/whisper.cpp/actions/runs/26496174929/job/78059073255?pr=3829
* ci : also disable BMI2
* ci : use github ubuntu-22.04-arm runner instead of qemu
This commit updates the ubuntu-22-gcc-arm64 job to use a arm github
runner instead of QEMU.
The motivation for this is that we get intermittent failure specifically
related to QEMU. For example:
```console
Segmentation fault (core dumped)
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault (core dumped)
dpkg: error processing package libc-bin (--configure):
installed libc-bin package post-installation script subprocess returned error exit status 139
Processing triggers for ca-certificates (20240203~22.04.1) ...
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Errors were encountered while processing:
libc-bin
E: Sub-process /usr/bin/dpkg returned an error code (1)
```
This is an attempt to try to avoid QEMU and hence avoid this issue.
* ci : remove QEMU where possible
* cmake:
- added `whisper-` prefix to unprefixed targets: `quantize`, `lsp`,
`vad-speech-segments`
- added `install(TARGETS ${TARGET} RUNTIME)` where it was missing
Signed-off-by: Peter A. <ink.splatters@pm.me>
* .github/workflows/build.yml: quantize -> whisper-quantize
Signed-off-by: Peter A. <ink.splatters@pm.me>
---------
Signed-off-by: Peter A. <ink.splatters@pm.me>
Daniel Bevenius