canary: add long-form chunking#112
Draft
brauliobo wants to merge 2 commits into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Dependency
This is a stacked PR on top of #111 (
canary: add v2 word timestamps). The branch intentionally includes that commit; please review this PR after #111, then rebase it ontomainonce #111 lands.Summary
Motivation
Canary 1B v2 has a 40 second decoder window. Running longer audio as one window can silently omit substantial interior speech even when inference reaches the final phrase. On the 84.381 second
samples/whole-earth.wav, the pre-long-form path returned 141 normalized words with a 35.040 second internal transcript gap. The chunked path returned 217 normalized words with a maximum 1.840 second inter-word gap and continuous coverage through the final phrase.The first fixed-duration prototype also lost repeated words at chunk seams. Selecting low-energy boundaries and restricting text deduplication to a true suffix fixed that failure rather than masking it with a broader overlap heuristic.
Correctness
Adversarial repeated-speech tests passed without missing or duplicated repetitions:
The 39 s, 40 s, 40.001 s, and 41 s threshold cases also passed. Every benchmark returned monotonic positive word intervals within the audio duration, with normalized full text matching the word sequence.
Performance
Warm Q8_0 measurements on a shared RTX 4060 Ti:
F32 timestamp validation at 55 seconds passed on an RTX 3090 in 899.36 ms (61.2x realtime). The shared RTX 4060 Ti did not have enough free memory for the F32 model's 6.1 GiB CUDA allocation, so that cell was moved to the 3090 rather than changing running services.
Verification
git diff --checkpassed