Skip to content

Fix psd_array_welch for good data spans shorter than n_per_seg (#13039)#14003

Open
CedricConday wants to merge 4 commits into
mne-tools:mainfrom
CedricConday:fix/welch-short-span-overlap
Open

Fix psd_array_welch for good data spans shorter than n_per_seg (#13039)#14003
CedricConday wants to merge 4 commits into
mne-tools:mainfrom
CedricConday:fix/welch-short-span-overlap

Conversation

@CedricConday

@CedricConday CedricConday commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Reference issue

Fixes #13039.

What does this implement/fix?

When bad_* annotations split the data into good spans, a span shorter than n_per_seg previously raised from SciPy:

ValueError: noverlap must be less than nperseg.

Following @CarinaFo's review, such spans are now dropped from the PSD estimate with a warning rather than analyzed with a shrunken window — a single Welch window does not fit them, and shrinking the window per-span would mix incompatible estimates. If every good span is shorter than n_per_seg, a clear ValueError is raised pointing the user to lower n_per_seg / n_fft.

Test

test_psd_welch_short_span_dropped covers both the warning-and-drop path (one short span among long ones) and the all-spans-too-short ValueError. Full test_psd.py is green under -W error.


Disclosure: I'm an AI engineer and use Claude Code in my workflow. All code here is hand-reviewed and the reasoning is mine; I'm happy to walk through any of it.

…13039)

When good data spans (between bad annotations) are shorter than n_per_seg,
SciPy reduces nperseg to the span length but leaves noverlap unchanged, so a
span shorter than n_overlap raised 'noverlap must be less than nperseg'.
Reduce noverlap per-span to stay < nperseg (nfft unchanged so frequency bins
match across spans). Adds a regression test.
@welcome

welcome Bot commented Jun 30, 2026

Copy link
Copy Markdown

Hello! 👋 Thanks for opening your first pull request here! ❤️ We will try to get back to you soon. 🚴

@CarinaFo

CarinaFo commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Hi Cedric, thank you for your contribution.

Could you disclose your AI usage according to our adopted policy.

I ran into this same issue in my analysis recently and ended up deciding that dropping short spans was the safer approach, so I would be rather cautious about shrinking noverlap, particularly without warning the user about this behaviour.
I suggest we sync with @drammock on the right approach here (I haven't looked at the "old" behaviour but I think it would be important to know why noverlap < nperseg did not throw a ValueError in older MNE versions.)

Per maintainer feedback (CarinaFo): rather than shrinking n_overlap to fit
good-data spans shorter than n_per_seg, drop them from the estimate and warn,
since a single Welch window does not fit them and shrinking the window
per-span mixes incompatible estimates. Raise a clear ValueError if every good
span is too short. Replaces the earlier noverlap-clamp approach.
@CedricConday CedricConday changed the title Fix psd_array_welch for good data spans shorter than n_overlap (#13039) Fix psd_array_welch for good data spans shorter than n_per_seg (#13039) Jun 30, 2026
@CedricConday

Copy link
Copy Markdown
Contributor Author

Hi Carina, thanks for the careful review.

On the AI disclosure — I've added it to the PR description per the policy: I'm an AI engineer and use Claude Code in my workflow; I find, fix, and test, then review and verify everything before opening it under my name. Happy to walk through any part of the reasoning.

On the approach — I agree, silently shrinking noverlap didn't sit right with me either. I've pushed an update that instead drops good-data spans shorter than n_per_seg with a warning, and raises a clear ValueError if every span is too short. Glad that matches where you landed.

Happy to sync with @drammock before this is finalized. On your open question — I don't want to guess at why noverlap < n_per_seg didn't raise in older MNE, so I'll dig into the history and report back here, so we're deciding on the real prior behavior rather than assumptions.

Thanks!

@CedricConday

Copy link
Copy Markdown
Contributor Author

Quick follow-up on the history question — I traced it. In 2015 (ca23ab6) _check_nfft silently clamped both: oversized n_fft down to the data length, and n_overlap >= n_fft down to n_fft - 1 — nothing raised. The ValueError for n_overlap >= n_per_seg was added in 2017 with #4003 (the PR that introduced n_per_seg). That same PR kept the segment-length clamp silent — n_per_seg = n if n_per_seg > n — and that line is still on main. So the short-span case never raised: the window just gets quietly shrunk to the span length, which is exactly the behavior here. Only the overlap check ever became strict; the segment-length clamp stayed silent throughout.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Using n_overlap in raw.compute_psd() fails if good data segments are shorter than n_overlap

2 participants