readme : document the ANEForge encoder backend (#4073)

* readme : document the ANEForge encoder backend

* Improve clarity in ANEForge support section of README

Reworded sentences for clarity and improved readability in the ANEForge section.

* Improve formatting of ANEForge support section

Reformat ANEForge section for better readability.
This commit is contained in:
Spencer Bryngelson
2026-09-18 07:06:54 +02:00
committed by GitHub
parent 4afa009405
commit 7a2ceef966
+25
View File
@@ -232,6 +232,31 @@ speed-up - more than x3 faster compared with CPU-only execution. Here are the in
For more information about the Core ML implementation please refer to PR [#566](https://github.com/ggml-org/whisper.cpp/pull/566).
## ANEForge support
On Apple Silicon, the Encoder can also run on the Apple Neural Engine via [ANEForge](https://github.com/sbryngelson/ANEForge), which dispatches to the ANE directly instead of through Core ML.
It is about 2x faster than the Core ML encoder from `tiny` to `medium` ([benchmarks](https://github.com/sbryngelson/ANEForge/tree/main/bench/whisper_encoder_ane)).
The Decoder is unchanged, and no build flag is needed.
Compile the encoder into a bundle (tied to the machine and OS build that produced it):
```bash
git clone https://github.com/sbryngelson/ANEForge && cd ANEForge
pip install -e ".[models]"
PYTHONPATH=. python3 bench/whisper_encoder_ane/export_bundle.py \
--model openai/whisper-base --out /tmp/whisper-base-encoder
```
Then point `whisper.cpp` at it:
```bash
export ANEFORGE_ENCODER=/tmp/whisper-base-encoder
export ANEFORGE_DYLIB=$PWD/aneforge/_lib/libane_e5rt_dispatch.dylib
./build/bin/whisper-cli -m models/ggml-base.bin -f samples/jfk.wav
```
For more information about the ANEForge implementation, see PR [#3905](https://github.com/ggml-org/whisper.cpp/pull/3905).
## OpenVINO support
On platforms that support [OpenVINO](https://github.com/openvinotoolkit/openvino), the Encoder inference can be executed