[DRAFT] Beatport/Beatsource + ID3 + deps modernization (credit: @rosgr100, upstream PR #526)#4
Draft
TX-RX wants to merge 37 commits into
Draft
[DRAFT] Beatport/Beatsource + ID3 + deps modernization (credit: @rosgr100, upstream PR #526)#4TX-RX wants to merge 37 commits into
TX-RX wants to merge 37 commits into
Conversation
Beatport no longer exposes __NEXT_DATA__ on search pages, causing search failures. Updated Beatport search to use the v4 catalog API endpoint instead of scraping the website. Adjusted deserialization to support the new API response format (tracks array).
Added a note about a temporary Beatport fix for May 2026.
Updated user agent string and fixed a typo in the search query parameter. Changed the token fetching method to use OAuth API and updated the token expiration logic.
Updated GitHub Actions to use the latest versions of actions for checkout, cache, setup-node, and pnpm.
… and update platform statics
Summary of Dependency Modernization (Non-API Changes)
Audio Engine Upgrades (rodio 0.22): Migrated the entire playback module (onetagger-player) from i16 integers to f32 floats. Updated all local decoders (aiff, alac, flac, mp3, mp4, ogg, wav) to yield floating-point data streams, and wrapped sample rates/channels in strict NonZeroU32/NonZeroU16 structural safety types.
Shazam Fingerprinting Fixes (onetagger-autotag): Adapted shazam.rs to map floating-point streams from UniformSourceIterator. Implemented a down-sampling translation block to scale and convert the raw float sample buffer back to standard i16 PCM vectors to satisfy the underlying SongRec signature generator requirements.
Window & UI Layer Adjustments (wry 0.55): Refactored the window lifecycle application loop in main.rs. Split the legacy webview navigation handler into separate modern asynchronous functions, updated the closures to support dual-argument signatures (handling url and NewWindowFeatures), and migrated target window states to use the modern NewWindowResponse::Allow and Deny enums.
Crate Configuration Updates (Cargo.toml): Swapped deprecated reqwest flags from "rustls-tls" to "rustls". Restored explicit "query" and "form" dependency compilation flags inside the platforms module to handle isolated network request actions. Replaced the legacy audio "cpal" identifier with the modernized "playback" flag to maintain access to local host speakers.
Wrapped the cover art file-writing loop in an explicit `config.album_art_file` check to prevent loose 'cover.jpg' files from being saved when the feature is disabled in settings.
id3.rs: Resolved the duplicate TXXX frame bug (e.g., UNIQUEFILEID, WWWAUDIOFILE) by explicitly clearing existing extended text frames before writing new ones. lib.rs (tag) & lib.rs (autotagger) : Fixed ID3 date mapping to ensure the standard YEAR frame strictly outputs a 4-digit year (YYYY). lib.rs (tag) & lib.rs (autotagger): Added logic to automatically inject the full YYYY-MM-DD date strings into custom RELEASETIME and PUBLISHTIME tags.
beatsource.rs: Added clear_search_query helper to strip parentheses from search strings before hitting the v4/catalog/search endpoint. This mirrors the recent Beatport fix and prevents 400/403 API crashes when scraping tracks with bracketed metadata (e.g., mix names or featured artists) in the title
Removed installation of nodejs and pnpm from dependencies. Added separate steps for installing NodeJS and pnpm.
… GitHub Actions workflow for build process
Updated macOS build job to create a universal binary for both Intel and Apple Silicon architectures. Adjusted steps for caching, installation, and artifact uploads.
**The Problem:** Perfectly matched tracks were receiving ~56% accuracy scores because OneTagger's standard fuzzy matching compared long local titles like `Title (Extended Mix)` against Beatport's shorter base `Title` field. **The Solution:** * **Smart Fallback:** Added a secondary matching pass that only triggers if the initial score is < 80%. * **Regex Extraction:** Safely splits the local title and mix name for independent grading. * **Weighted Scoring:** Calculates a new accuracy score heavily weighted toward the base title (70%) but rewarding accurate mix names (30%). * **False Positive Prevention:** Added a strict boolean check to immediately reject API tracks if their mix name directly contradicts the local file's mix name. * **Deps:** Added `strsim` to `onetagger-platforms`.
**The Problem:** Perfectly matched tracks were receiving ~56% accuracy scores because OneTagger's standard fuzzy matching compared long local titles like `Title (Extended Mix)` against Beatport's shorter base `Title` field. **The Solution:** * **Smart Fallback:** Added a secondary matching pass that only triggers if the initial score is < 80%. * **Regex Extraction:** Safely splits the local title and mix name for independent grading. * **Weighted Scoring:** Calculates a new accuracy score heavily weighted toward the base title (70%) but rewarding accurate mix names (30%). * **False Positive Prevention:** Added a strict boolean check to immediately reject API tracks if their mix name directly contradicts the local file's mix name. * **Deps:** Added `strsim` to `onetagger-platforms`.
…y and remix matching
This PR introduces a highly optimized, secondary fallback matching engine exclusively within beatport.rs. It is designed to resolve widespread false negatives caused by Beatport's inconsistent metadata formatting (e.g., artist reordering, alias variations, and arbitrary (Extended Mix) suffixes) without altering the primary MatchingUtils core logic. The fallback only triggers if the primary matcher returns a confidence score below 0.80, acting as a localized rescue mission for difficult DJ metadata.
Key Features & Improvements
Categorical Version Taxonomy (MixType): Replaces brittle string equality with a strict enum matrix. This guarantees that functionally different DJ mixes (e.g., Club Mix vs Extended Mix) are strictly isolated and cannot falsely match, while safely bridging tracks missing explicit tags (Unknown ↔ Original). Jaccard (Token-Based) Similarity: Transitions from Levenshtein distance to Jaccard set intersection for Artist arrays and Remix titles. This completely resolves the mathematical penalties previously caused by word reordering (e.g., "Guetta Remix" vs "Remix Guetta") and punctuation differences ("&" vs "and"). Remix Stopword Filtering: Violently strips noise words (remix, rmx, mix, edit, vip, dub, etc.) prior to Jaccard comparison. This prevents false positives where two completely different remixers achieve a high similarity score simply because both strings contain the word "remix". Deterministic Confidence Ceiling: Enforces a strict scoring hierarchy where fuzzy Jaccard artist matches are mathematically capped (0.9) to ensure they can never outrank a verified exact match (1.0).
Performance Optimizations
O(N²) Prevention: Eliminates nested vector iteration by utilizing an internal HashMap lookup during fallback score updates. Regex Caching: Migrates the primary Mix Regex to a static OnceLock so it compiles exactly once per application lifetime. Closure Hoisting: Moves allocation-heavy normalizer closures (normalize_punctuation, normalize_artists) outside the main iteration loop.
Architectural Safety
Zero Core Impact: These changes are 100% fenced behind the < 0.80 fallback gate and contained entirely inside beatport.rs. The primary engine remains completely untouched, ensuring stability across other platform matchers.
…ag pagination The Beatport module struggled to match tracks containing featured artists in the local title (e.g., "Forever Ft. Sabrina Johnston"). Beatport's search API frequently chokes and returns 0 results when featured artists are included in the search string. Even when found, OneTagger's strict accuracy thresholds would fail the match because Beatport's base title didn't contain the featured artist. Furthermore, Beatport's Auto Tag fall back search was failing on valid tracks because the engine exited pagination prematurely (only checking Page 1) and didn't sort the fallback array by accuracy. The Solution This PR completely overhauls the Beatport matching logic and search sanitization to be bulletproof out-of-the-box. Key Changes: Hardcoded Search Sanitization: Baked a feature-stripping Regex ((?i)\s+(?:ft|feat|featuring)\.?\s+[^()]+) directly into clear_search_query. The Beatport API now always receives a clean base title, preventing 0-result API crashes. Universal Math Fallback: Added a smart fallback engine that dynamically strips features from both the local title and the Beatport API title during the Levenshtein calculation. This guarantees a 1.0 accuracy score for valid tracks even if the user has the OneTagger title cleanup regex empty. Autotag Pagination Fix: Modified the search loop to respect the user's max_pages config during Auto Tag runs, rather than prematurely returning after Page 1 if a match wasn't instantly found. Array Sorting: Forced matched_tracks to sort by accuracy descending so Auto Tag's automated selection reliably grabs the 100% match instead of a scrambled lower-tier match.
…ashes Two edge-case bugs were causing the Beatport module to freeze or aggressively fail out of track matching: Token Deadlock: If the Beatport API token expired mid-session, the update_token function attempted a recursive call while still holding the Mutex guard, causing the application thread to permanently freeze. ISRC Hard Crash: The ISRC matching path used the ? operator on the track detail API fetch. If the API rate-limited or dropped the connection on that specific call, it hard-failed the entire matching process instead of safely falling back to the text-search engine. The Solution Mutex Deadlock Fix: Explicitly added drop(token); to release the Mutex guard before the recursive update_token call, ensuring thread safety upon token expiry. Graceful ISRC Fallback: Replaced the ? operator in the ISRC search block with a match statement. If a track detail fetch fails, it now logs a warning and gracefully falls through to the standard text search, rather than crashing the tagger.
Implemented a bounded retry loop in the update_token method to handle expired tokens more effectively by preventing infinite recursion. Added error handling for cases where the Beatport API continuously provides expired tokens.
Add pnpm-workspace.yaml and remove the npm lockfile while updating client configuration and source files. Updates include package.json and tsconfig.json changes plus code adjustments in src/main.ts and src/views/TagEditor.vue to align with the workspace/dependency setup and recent refactors.
Remove the JunoDownload platform (source, asset and module) and related references across the codebase. Update onetagger-autotag to stop registering JunoDownload and remove the junodownload module from onetagger-platforms. UI changes in AutotaggerPlatforms.vue: switch to a localPlatforms reactive array, add sorting, a watch to keep UI/config in sync, protect against re-sorts during drag operations, update draggable props and ensure settings are saved when order or enabled state changes. Minor UX/content tweaks: remove Juno mentions from README and HelpButton tooltip and from AutotaggerStatus display mapping. Misc: adjust rodio import in onetagger-player (remove unused Player) and make webview drag-drop handling ignore empty drops (fixes internal drag operations such as vuedraggable).
updates for client + some smaller rust fixes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pulls in @rosgr100's upstream PR Marekkon5#526 unchanged so the original commits (and attribution to @Simon-Zwa, @rosgr100, and @benjaminlewandowski / @jiji-lewandowski) are preserved.
Opened as draft because this is a superset of PR #1 (Simon-Zwa's clean v4-search fix) plus a very large amount of additional work that needs review before merge. Do not merge until we've decided which pieces we want.
What's included
Grouped by author:
@Simon-Zwa (3 commits, rebased): Same content as PR #1 — Beatport search moved to v4 catalog API.
@rosgr100 / @spirosg (~25 commits):
+ 'UNIQUEFILEID' +,+ 'WWWAUDIOFILE' +) on overwrite; standardizes YEAR to 4 digits and puts full YYYY-MM-DD into+ 'RELEASETIME' +/+ 'PUBLISHTIME' +.+ 'MixType' +enum), Jaccard artist comparison, remix stopword filtering, deterministic confidence ceiling — all fenced behind a < 0.80 fallback so the primary matcher stays untouched.+ 'feat' +/+ 'ft' +/+ 'featuring' +in+ 'clear_search_query' +(the v4 API returns 0 results when features are in the query).+ 'drop(token)' +before recursive refresh).+ 'update_token' +.+ 'shazam.rs' +to feed the SongRec fingerprinter its expected i16 stream.+ 'cargo-bundle' +-style cover-art-file guard (respects+ 'config.album_art_file' +before writing loose+ 'cover.jpg' +s).@benjaminlewandowski / @jiji-lewandowski (2 commits, merged in via SpirosG's PR #2):
Concerns for review
+ 'beatport.rs' +changes here will merge cleanly on top; the audio-pipeline changes are unrelated.Recommendation
Cherry-pick, don't wholesale merge. Suggested slices:
+ 'feat' +stripping, mix taxonomy, artist Jaccard).