upstream-archive/whisper.cpp
1
0
Fork 0
mirror of https://github.com/ggml-org/whisper.cpp.git synced 2026-07-23 11:10:57 -05:00
Code Issues Packages Projects Releases Wiki Activity

whisper : map token timestamps to original time when VAD is enabled (#3910)

Browse Source 
whisper_full_get_token_data().t0/t1 are in the VAD-processed timeline (silences
removed), so they don't line up with the original audio. Add
whisper_full_get_token_t0/t1 that map them back.

A token inside a speech segment is interpolated within it; a token that falls in a
removed inter-segment silence snaps to the nearest boundary, so it never lands in
the middle of a cut-out gap. Without VAD the raw times are returned unchanged.
This commit is contained in:
Lin Xiaodong
2026-07-01 14:23:50 +08:00
committed by GitHub
parent 909307c8bd
commit 5cad7abb1e
3 changed files with 86 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
include/whisper.h
View File

@@ -667,6 +667,16 @@ extern "C" {
WHISPER_API whisper_token_data whisper_full_get_token_data (struct whisper_context * ctx, int i_segment, int i_token);
WHISPER_API whisper_token_data whisper_full_get_token_data_from_state(struct whisper_state * state, int i_segment, int i_token);
// Get the start/end time of the specified token, in centiseconds. When VAD is enabled
// these are mapped back to the original audio timeline (a token landing in a removed
// inter-segment silence snaps to the nearest speech boundary), unlike
// whisper_full_get_token_data().t0/t1 which stay in VAD-processed time. Without VAD the
// raw token times are returned unchanged.
WHISPER_API int64_t whisper_full_get_token_t0 (struct whisper_context * ctx, int i_segment, int i_token);
WHISPER_API int64_t whisper_full_get_token_t0_from_state(struct whisper_state * state, int i_segment, int i_token);
WHISPER_API int64_t whisper_full_get_token_t1 (struct whisper_context * ctx, int i_segment, int i_token);
WHISPER_API int64_t whisper_full_get_token_t1_from_state(struct whisper_state * state, int i_segment, int i_token);
// Get the probability of the specified token in the specified segment
WHISPER_API float whisper_full_get_token_p (struct whisper_context * ctx, int i_segment, int i_token);
WHISPER_API float whisper_full_get_token_p_from_state(struct whisper_state * state, int i_segment, int i_token);