Fix psd_array_welch for good data spans shorter than n_per_seg (#13039)#14003
Fix psd_array_welch for good data spans shorter than n_per_seg (#13039)#14003CedricConday wants to merge 4 commits into
Conversation
…13039) When good data spans (between bad annotations) are shorter than n_per_seg, SciPy reduces nperseg to the span length but leaves noverlap unchanged, so a span shorter than n_overlap raised 'noverlap must be less than nperseg'. Reduce noverlap per-span to stay < nperseg (nfft unchanged so frequency bins match across spans). Adds a regression test.
|
Hello! 👋 Thanks for opening your first pull request here! ❤️ We will try to get back to you soon. 🚴 |
|
Hi Cedric, thank you for your contribution. Could you disclose your AI usage according to our adopted policy. I ran into this same issue in my analysis recently and ended up deciding that dropping short spans was the safer approach, so I would be rather cautious about shrinking noverlap, particularly without warning the user about this behaviour. |
Per maintainer feedback (CarinaFo): rather than shrinking n_overlap to fit good-data spans shorter than n_per_seg, drop them from the estimate and warn, since a single Welch window does not fit them and shrinking the window per-span mixes incompatible estimates. Raise a clear ValueError if every good span is too short. Replaces the earlier noverlap-clamp approach.
for more information, see https://pre-commit.ci
|
Hi Carina, thanks for the careful review. On the AI disclosure — I've added it to the PR description per the policy: I'm an AI engineer and use Claude Code in my workflow; I find, fix, and test, then review and verify everything before opening it under my name. Happy to walk through any part of the reasoning. On the approach — I agree, silently shrinking Happy to sync with @drammock before this is finalized. On your open question — I don't want to guess at why Thanks! |
|
Quick follow-up on the history question — I traced it. In 2015 ( |
Reference issue
Fixes #13039.
What does this implement/fix?
When
bad_*annotations split the data into good spans, a span shorter thann_per_segpreviously raised from SciPy:Following @CarinaFo's review, such spans are now dropped from the PSD estimate with a warning rather than analyzed with a shrunken window — a single Welch window does not fit them, and shrinking the window per-span would mix incompatible estimates. If every good span is shorter than
n_per_seg, a clearValueErroris raised pointing the user to lowern_per_seg/n_fft.Test
test_psd_welch_short_span_droppedcovers both the warning-and-drop path (one short span among long ones) and the all-spans-too-shortValueError. Fulltest_psd.pyis green under-W error.Disclosure: I'm an AI engineer and use Claude Code in my workflow. All code here is hand-reviewed and the reasoning is mine; I'm happy to walk through any of it.