diff --git a/README.md b/README.md index 6bf407df9..a037e9545 100644 --- a/README.md +++ b/README.md @@ -232,6 +232,31 @@ speed-up - more than x3 faster compared with CPU-only execution. Here are the in For more information about the Core ML implementation please refer to PR [#566](https://github.com/ggml-org/whisper.cpp/pull/566). +## ANEForge support + +On Apple Silicon, the Encoder can also run on the Apple Neural Engine via [ANEForge](https://github.com/sbryngelson/ANEForge), which dispatches to the ANE directly instead of through Core ML. +It is about 2x faster than the Core ML encoder from `tiny` to `medium` ([benchmarks](https://github.com/sbryngelson/ANEForge/tree/main/bench/whisper_encoder_ane)). +The Decoder is unchanged, and no build flag is needed. + +Compile the encoder into a bundle (tied to the machine and OS build that produced it): + +```bash +git clone https://github.com/sbryngelson/ANEForge && cd ANEForge +pip install -e ".[models]" +PYTHONPATH=. python3 bench/whisper_encoder_ane/export_bundle.py \ + --model openai/whisper-base --out /tmp/whisper-base-encoder +``` + +Then point `whisper.cpp` at it: + +```bash +export ANEFORGE_ENCODER=/tmp/whisper-base-encoder +export ANEFORGE_DYLIB=$PWD/aneforge/_lib/libane_e5rt_dispatch.dylib +./build/bin/whisper-cli -m models/ggml-base.bin -f samples/jfk.wav +``` + +For more information about the ANEForge implementation, see PR [#3905](https://github.com/ggml-org/whisper.cpp/pull/3905). + ## OpenVINO support On platforms that support [OpenVINO](https://github.com/openvinotoolkit/openvino), the Encoder inference can be executed