Cherry-pick Beatport matching + ID3 hygiene + OAuth safety fixes from PR #526 (credit: @rosgr100)#5
Merged
Merged
Conversation
id3.rs: Resolved the duplicate TXXX frame bug (e.g., UNIQUEFILEID, WWWAUDIOFILE) by explicitly clearing existing extended text frames before writing new ones. lib.rs (tag) & lib.rs (autotagger) : Fixed ID3 date mapping to ensure the standard YEAR frame strictly outputs a 4-digit year (YYYY). lib.rs (tag) & lib.rs (autotagger): Added logic to automatically inject the full YYYY-MM-DD date strings into custom RELEASETIME and PUBLISHTIME tags.
**The Problem:** Perfectly matched tracks were receiving ~56% accuracy scores because OneTagger's standard fuzzy matching compared long local titles like `Title (Extended Mix)` against Beatport's shorter base `Title` field. **The Solution:** * **Smart Fallback:** Added a secondary matching pass that only triggers if the initial score is < 80%. * **Regex Extraction:** Safely splits the local title and mix name for independent grading. * **Weighted Scoring:** Calculates a new accuracy score heavily weighted toward the base title (70%) but rewarding accurate mix names (30%). * **False Positive Prevention:** Added a strict boolean check to immediately reject API tracks if their mix name directly contradicts the local file's mix name. * **Deps:** Added `strsim` to `onetagger-platforms`.
**The Problem:** Perfectly matched tracks were receiving ~56% accuracy scores because OneTagger's standard fuzzy matching compared long local titles like `Title (Extended Mix)` against Beatport's shorter base `Title` field. **The Solution:** * **Smart Fallback:** Added a secondary matching pass that only triggers if the initial score is < 80%. * **Regex Extraction:** Safely splits the local title and mix name for independent grading. * **Weighted Scoring:** Calculates a new accuracy score heavily weighted toward the base title (70%) but rewarding accurate mix names (30%). * **False Positive Prevention:** Added a strict boolean check to immediately reject API tracks if their mix name directly contradicts the local file's mix name. * **Deps:** Added `strsim` to `onetagger-platforms`.
…y and remix matching
This PR introduces a highly optimized, secondary fallback matching engine exclusively within beatport.rs. It is designed to resolve widespread false negatives caused by Beatport's inconsistent metadata formatting (e.g., artist reordering, alias variations, and arbitrary (Extended Mix) suffixes) without altering the primary MatchingUtils core logic. The fallback only triggers if the primary matcher returns a confidence score below 0.80, acting as a localized rescue mission for difficult DJ metadata.
Key Features & Improvements
Categorical Version Taxonomy (MixType): Replaces brittle string equality with a strict enum matrix. This guarantees that functionally different DJ mixes (e.g., Club Mix vs Extended Mix) are strictly isolated and cannot falsely match, while safely bridging tracks missing explicit tags (Unknown ↔ Original). Jaccard (Token-Based) Similarity: Transitions from Levenshtein distance to Jaccard set intersection for Artist arrays and Remix titles. This completely resolves the mathematical penalties previously caused by word reordering (e.g., "Guetta Remix" vs "Remix Guetta") and punctuation differences ("&" vs "and"). Remix Stopword Filtering: Violently strips noise words (remix, rmx, mix, edit, vip, dub, etc.) prior to Jaccard comparison. This prevents false positives where two completely different remixers achieve a high similarity score simply because both strings contain the word "remix". Deterministic Confidence Ceiling: Enforces a strict scoring hierarchy where fuzzy Jaccard artist matches are mathematically capped (0.9) to ensure they can never outrank a verified exact match (1.0).
Performance Optimizations
O(N²) Prevention: Eliminates nested vector iteration by utilizing an internal HashMap lookup during fallback score updates. Regex Caching: Migrates the primary Mix Regex to a static OnceLock so it compiles exactly once per application lifetime. Closure Hoisting: Moves allocation-heavy normalizer closures (normalize_punctuation, normalize_artists) outside the main iteration loop.
Architectural Safety
Zero Core Impact: These changes are 100% fenced behind the < 0.80 fallback gate and contained entirely inside beatport.rs. The primary engine remains completely untouched, ensuring stability across other platform matchers.
…ag pagination The Beatport module struggled to match tracks containing featured artists in the local title (e.g., "Forever Ft. Sabrina Johnston"). Beatport's search API frequently chokes and returns 0 results when featured artists are included in the search string. Even when found, OneTagger's strict accuracy thresholds would fail the match because Beatport's base title didn't contain the featured artist. Furthermore, Beatport's Auto Tag fall back search was failing on valid tracks because the engine exited pagination prematurely (only checking Page 1) and didn't sort the fallback array by accuracy. The Solution This PR completely overhauls the Beatport matching logic and search sanitization to be bulletproof out-of-the-box. Key Changes: Hardcoded Search Sanitization: Baked a feature-stripping Regex ((?i)\s+(?:ft|feat|featuring)\.?\s+[^()]+) directly into clear_search_query. The Beatport API now always receives a clean base title, preventing 0-result API crashes. Universal Math Fallback: Added a smart fallback engine that dynamically strips features from both the local title and the Beatport API title during the Levenshtein calculation. This guarantees a 1.0 accuracy score for valid tracks even if the user has the OneTagger title cleanup regex empty. Autotag Pagination Fix: Modified the search loop to respect the user's max_pages config during Auto Tag runs, rather than prematurely returning after Page 1 if a match wasn't instantly found. Array Sorting: Forced matched_tracks to sort by accuracy descending so Auto Tag's automated selection reliably grabs the 100% match instead of a scrambled lower-tier match.
…ashes Two edge-case bugs were causing the Beatport module to freeze or aggressively fail out of track matching: Token Deadlock: If the Beatport API token expired mid-session, the update_token function attempted a recursive call while still holding the Mutex guard, causing the application thread to permanently freeze. ISRC Hard Crash: The ISRC matching path used the ? operator on the track detail API fetch. If the API rate-limited or dropped the connection on that specific call, it hard-failed the entire matching process instead of safely falling back to the text-search engine. The Solution Mutex Deadlock Fix: Explicitly added drop(token); to release the Mutex guard before the recursive update_token call, ensuring thread safety upon token expiry. Graceful ISRC Fallback: Replaced the ? operator in the ISRC search block with a match statement. If a track detail fetch fails, it now logs a warning and gracefully falls through to the standard text search, rather than crashing the tagger.
Implemented a bounded retry loop in the update_token method to handle expired tokens more effectively by preventing infinite recursion. Added error handling for cases where the Beatport API continuously provides expired tokens.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-picks the safe, high-value commits from @rosgr100's upstream PR Marekkon5#526 (backported as our own PR #4). Each commit is preserved with the original author, so attribution is intact.
What's included
Beatport matching quality (@spirosg):
(?i)\s+(?:ft|feat|featuring)\.?\s+[^()]+regex intoclear_search_querybecause the v4 catalog API returns 0 results when features are in the query. Also strips brackets and commas.max_pagesinstead of exiting after page 1 without a match. Sorts fallback matches by accuracy descending.ID3 hygiene (@spirosg):
+ 'UNIQUEFILEID' +,+ 'WWWAUDIOFILE' +) were accumulating on overwrite. Now the writer explicitly clears the existing extended text frames before appending new ones.+ 'YEAR' +strictly 4 digits (YYYY), full YYYY-MM-DD in custom+ 'RELEASETIME' +/+ 'PUBLISHTIME' +tags for DJ apps that expect them.OAuth safety (@spirosg):
+ 'update_token' +was recursing while holding the Mutex guard. Now explicitly drops the guard before the recursive call.+ '?' +on a track-detail fetch, so an API dropout would abort the whole match. Replaced with a match that falls through to text search on error.+ 'update_token' +so an endlessly-expired API can't infinite-loop.What's explicitly NOT included
Skipped from PR #4 (draft) because these are either not needed for our fork or need more testing than a cherry-pick warrants:
Conflicts I resolved
One conflict in
+ 'clear_search_query' +between the earlier featured-artist commit and the later deadlock fix — both wanted to reintroduce the feature-stripping form. Took the final version (regex + bracket + comma stripping) since that's the intent both commits are converging on.Follow-up if we like these
Cutting v1.7.2-beta.2 on tag push after merge. If beta.2 tags a healthy match rate on real files, we can consider taking the audio pipeline migration in a separate PR — but that one really does want playback validation.