mirror of
https://github.com/ggml-org/whisper.cpp.git
synced 2026-09-25 07:27:34 -05:00
readme : document the ANEForge encoder backend (#4073)
* readme : document the ANEForge encoder backend * Improve clarity in ANEForge support section of README Reworded sentences for clarity and improved readability in the ANEForge section. * Improve formatting of ANEForge support section Reformat ANEForge section for better readability.
This commit is contained in:
@@ -232,6 +232,31 @@ speed-up - more than x3 faster compared with CPU-only execution. Here are the in
|
||||
|
||||
For more information about the Core ML implementation please refer to PR [#566](https://github.com/ggml-org/whisper.cpp/pull/566).
|
||||
|
||||
## ANEForge support
|
||||
|
||||
On Apple Silicon, the Encoder can also run on the Apple Neural Engine via [ANEForge](https://github.com/sbryngelson/ANEForge), which dispatches to the ANE directly instead of through Core ML.
|
||||
It is about 2x faster than the Core ML encoder from `tiny` to `medium` ([benchmarks](https://github.com/sbryngelson/ANEForge/tree/main/bench/whisper_encoder_ane)).
|
||||
The Decoder is unchanged, and no build flag is needed.
|
||||
|
||||
Compile the encoder into a bundle (tied to the machine and OS build that produced it):
|
||||
|
||||
```bash
|
||||
git clone https://github.com/sbryngelson/ANEForge && cd ANEForge
|
||||
pip install -e ".[models]"
|
||||
PYTHONPATH=. python3 bench/whisper_encoder_ane/export_bundle.py \
|
||||
--model openai/whisper-base --out /tmp/whisper-base-encoder
|
||||
```
|
||||
|
||||
Then point `whisper.cpp` at it:
|
||||
|
||||
```bash
|
||||
export ANEFORGE_ENCODER=/tmp/whisper-base-encoder
|
||||
export ANEFORGE_DYLIB=$PWD/aneforge/_lib/libane_e5rt_dispatch.dylib
|
||||
./build/bin/whisper-cli -m models/ggml-base.bin -f samples/jfk.wav
|
||||
```
|
||||
|
||||
For more information about the ANEForge implementation, see PR [#3905](https://github.com/ggml-org/whisper.cpp/pull/3905).
|
||||
|
||||
## OpenVINO support
|
||||
|
||||
On platforms that support [OpenVINO](https://github.com/openvinotoolkit/openvino), the Encoder inference can be executed
|
||||
|
||||
Reference in New Issue
Block a user