Port CorbeauSplat to Windows / NVIDIA CUDA#8
Draft
marmotte5 wants to merge 46 commits into
Draft
Conversation
Convert the macOS/Apple Silicon Gaussian Splatting tool into a Windows-only fork focused on the video -> frames -> COLMAP -> Brush splat training -> view pipeline, accelerated by NVIDIA CUDA. Platform core - system.py: replace Apple Silicon/MPS detection with CUDA detection via nvidia-smi (get_device -> cuda/cpu); Windows memory via GlobalMemoryStatusEx; resolve_binary handles .exe/COLMAP.bat and auto-detects C:\COLMAP. - FFmpeg uses -hwaccel cuda (NVDEC); COLMAP SIFT extraction/matching pass --SiftExtraction.use_gpu / --SiftMatching.use_gpu when CUDA is present; Brush runs on wgpu with WGPU_BACKEND=dx12. Launcher & installers - New run.bat replaces run.command. - tools.py: drop Homebrew/Xcode; FFmpeg via winget, COLMAP CUDA detected/guided, Node/CMake/Ninja via winget, Rust via rustup-init.exe. - mapping.py: ColmapEngineDep (PATH/known-dir detection) replaces brew-based dep; Windows-aware Glomap build. - upscayl + brush installers: Windows asset selection and .exe binaries; checksums.json gains windows_* keys. - Drop the macOS-only pyobjc dependency. Removed Apple ML Sharp - MLX/Apple-only image/video->3D feature removed: engine, installer, GUI tab, CLI sharp subcommand, workers and related tests. Docs & tests - README.md / manifest.md / CHANGELOG.md rewritten for Windows/CUDA. - Test suite updated for the new platform; 189/189 pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
Remove the manual COLMAP/FFmpeg setup step. On first launch the dependency manager now downloads them automatically: - ColmapEngineDep fetches the latest COLMAP release and extracts the colmap-x64-windows-cuda.zip asset into engines/colmap (non-CUDA build when no NVIDIA GPU is detected). - FfmpegEngineDep installs FFmpeg via winget when available, else extracts a static build into engines/ffmpeg. - resolve_binary() searches engines/colmap and engines/ffmpeg subtrees and prefers .exe over .bat so binaries are launchable via subprocess. - ColmapEngine.run_command adds COLMAP's bin/ and lib/ dirs to PATH so the bundled DLLs load without COLMAP.bat. - Shared download_and_extract_zip helper with path-traversal-safe extraction. Docs updated; added unit tests for the COLMAP asset selector. 193 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
Real-machine testing surfaced several Windows-specific bugs: - download_and_extract_zip leaked the mkstemp file descriptor, so Windows held an exclusive lock and ZipFile failed with WinError 32 — this blocked both COLMAP and FFmpeg extraction. Now closes the fd (also fixed the same pattern in install_rust_toolchain). - winget installs (cmake/ninja/node) didn't update the current process PATH, so the next cmake/npm call failed with WinError 2. Added refresh_windows_path() (reads PATH from the registry) and call it after every winget install. - Glomap (MSVC+CUDA source build) and the 360 Extractor no longer auto-build at startup — added install_on_startup=False so heavy/optional engines only install when their feature is enabled. - FFmpeg install simplified to download a static build into engines/ffmpeg (no winget round-trip / PATH-refresh dependency); a system ffmpeg is still detected and reused. - Brush is now saved as brush.exe on Windows (was 'brush', not launchable). - npm/npx are .cmd shims that CreateProcess can't launch directly; route them through cmd.exe for SuperSplat install and the viewer launch. - validate_path: relaxed home/project containment that rejected legitimate output folders on other drives (e.g. I:\projects). User-selected absolute paths are now allowed; '..' parent-traversal is still rejected. Dropped the matching GUI drop-target restriction. 193 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
- COLMAP 4.x rejects --SiftExtraction.use_gpu / --SiftMatching.use_gpu with "unrecognised option", aborting feature extraction. Removed the explicit flags: a CUDA-enabled COLMAP uses the GPU for SIFT by default, so GPU acceleration is preserved while staying compatible across COLMAP versions. - When the selected input contains no images (images mode), the pipeline used to return success and run COLMAP on an empty folder. It now stops with a clear message (unless the working folder already has images from a re-run). 193 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
Add a fast-forward `git pull` step to run.bat so launching the app always picks up the latest fixes. Skips cleanly when git is unavailable, the repo has local changes, or the machine is offline. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
The upscale step checked only for the upscayl binary, not for a downloaded model. With no model, it moved the originals to images_src and then crashed upscayl (missing .param), leaving the working folder broken and aborting the whole dataset creation. Now: - verify a downloaded model exists before touching the images; if none, log a clear message and skip upscale (originals untouched); - on any upscale failure/exception, restore the originals from images_src and continue the pipeline instead of aborting (upscale is an optional step). 193 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
Discards out-of-focus / motion-blurred frames (common with video-extracted
images) before reconstruction, which improves splat sharpness and speeds up
COLMAP by dropping low-information frames.
- Sharpness = variance of the Laplacian (OpenCV). A frame is rejected when its
score is below blur_factor x the median sharpness; rejected frames are moved
to a sibling 'images_blurry' folder (not deleted) so they can be inspected.
- Relative/median-based threshold with a 50% safety cap so it adapts to content
and never guts the dataset. Runs on the raw frames, before upscaling. Never
fatal — skips cleanly if OpenCV is missing or there are too few images.
- GUI: "Filtrer les images floues" checkbox + Léger/Moyen/Fort severity in the
Config tab Options group, persisted in config.json.
- CLI: --filter_blur / --blur_strength {light,medium,strong} on colmap+pipeline.
- Added opencv-python-headless to requirements (also enables the previously
skipped resolution-normalization step).
- Pure selection logic factored into select_blurry_files() with unit tests.
197 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
New "Nettoyage / Cleaner" tab to clean a trained Gaussian Splat .ply without touching the original: - app/core/ply_cleaner.py: removes near-transparent splats (low opacity), oversized gaussians (sky shells / big floaters) and spatial outliers (floaters far from the cloud), via opacity + scale-percentile + distance- percentile filtering. Survivors are kept byte-for-byte (colour/SH/rotation preserved). Pure compute_keep_mask() is unit-tested; clean_ply() does the I/O. - GUI tab: load a .ply, choose severity (Léger/Moyen/Fort) or fine-tune the three thresholds, run cleanup in a worker thread, see before/after stats and per-filter breakdown, preview the result in SuperSplat, then "Save as". - CLI: python main.py clean -i in.ply -o out.ply --strength medium. - CleanerWorker added; tab wired into the main window + session persistence. Note: cleanup removes noise (sky/floaters), it does not fix reconstruction drift — that needs loop closure at the COLMAP stage. 204 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
…filter Large reconstructions (thousands of images) can crash COLMAP's global bundle adjustment with "Linear solver failure. Failed to compute a finite step.". The new robust mode stabilises it using only known COLMAP options: - camera model PINHOLE (no distortion params to diverge), - ba_refine_extra_params = off, ba_refine_principal_point = off, - multiple_models = on (a degenerate region spawns a sub-model instead of killing the whole BA), - forces blur filtering on so blurry frames aren't registered. Exposed as a "Mode robuste (grandes scènes)" checkbox in the Config tab (enabling it also ticks the blur filter), and as --robust on the colmap and pipeline CLI commands. Persisted in config.json. 206 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
The blur filter analysed every frame at full resolution with no feedback, so on large sets (6000+ images) it looked frozen for minutes. Now it: - logs progress (status every 100 images, log line every 500), - downscales images to ≤640px before the Laplacian — the relative sharpness ranking is preserved and it runs ~10x faster. 206 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
cv2.imread/resize/Laplacian release the GIL, so scoring the frames in a ThreadPoolExecutor (num_threads workers) gives a large speedup on big sets (e.g. 16x on a 16-thread machine). Progress is reported as futures complete; cancellation shuts the pool down promptly. 206 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
Owner
|
It's great idea. I can't merge to my project, but we can do another repo ? |
The resolution check full-decoded every image via cv2.imread just to read width/height — minutes of work on large sets (10k+ frames), and pointless for video frames that all share one resolution. Now it reads dimensions from the file header via PIL (no pixel decode), parallelized across CPU threads with progress, and exits instantly when the resolution is uniform. cv2 is only used for the actual resize, which only runs when resolutions genuinely differ. 206 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
…ath log The biggest hidden cost: "Domain Size Pooling" was ON by default, which disables COLMAP's GPU SIFT and silently falls back to slow CPU extraction — defeating the CUDA port. Flipped it OFF by default across the dataclass, the Params tab and the CLI (now an opt-in --domain_size_pooling). Affine Shape stays off, Cross Check / Single Camera stay on — the settings we recommended. Also: feature_extraction now logs which SIFT path runs (GPU vs CPU) and warns when Affine/Domain Pooling is forcing CPU, so a misconfiguration that loses the GPU is obvious. 206 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
On shaky video, the median-relative sharpness threshold could flag a large fraction of frames, and the 50% safety cap then moved up to half the dataset into images_blurry — users reported "my whole dataset is in blurry". Lower the default cap to 20% so at least 80% of frames are always kept, and the log now explains how to restore (move files back from images_blurry to images). 206 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
Robust mode set params.filter_blurry = True in get_current_params, overriding the user's choice — so unchecking "Filtrer les images floues" did nothing while Mode robuste was on. Robust mode now only pre-ticks the blur checkbox (via config_tab._on_robust_toggled); the actual checkbox state is what's used, so unchecking it is respected. 206 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
…ion) When the input is an image folder, the pipeline copied every image into project/images, doubling disk usage. It now hardlinks them (same data on disk, zero extra space), falling back to a copy only when the source is on a different volume. The one in-place writer (resolution resize) unlinks first so it writes a fresh file and never modifies the user's originals; blur-filter and upscale only move/replace project entries, so originals are always safe. 206 tests pass; verified hardlink (same inode) + resize-safety locally. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
…oads
- ply_utils.parse_ply_manual: numpy.void rows have no .get(); replaced with a
field-name guarded _field() helper (was AttributeError on every binary PLY
with scale/rot/opacity).
- upscale_engine.load_model: scale heuristic produced "realesrgan-2plus"
instead of "realesrgan-x2plus" (replace "x4" with f"x{scale}").
- upscayl_manager: guard os.path.relpath ValueError when binary and models
live on different Windows drives; guard tarfile.extractfile() returning None;
stream the release archive to disk instead of buffering in RAM.
- installers/tools.py and installers/brush.py: stream large COLMAP/ffmpeg/Brush
downloads with shutil.copyfileobj instead of resp.read().
- ruff --fix: style cleanup across the project.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
Two features from COLMAP 4.1.0:
GPU bundle adjustment ("Caspar"):
- New ba_use_gpu / ba_gpu_index params + "Bundle adjustment GPU" checkbox in
the COLMAP tab. Runs BA on the GPU, which fixes the "Linear solver failure"
CPU crashes on large scenes.
- engine.mapper passes --Mapper.ba_use_gpu only after probing `colmap mapper -h`
(cached) for the option, so an older COLMAP never aborts on an unknown flag —
it logs a clear fallback message instead.
Native 360 reconstruction:
- EQUIRECTANGULAR added to the camera-model dropdown plus a convenience
"360 natif" checkbox that drives it (single source of truth = camera_model,
synced both ways). feature_extraction logs the 4.1.0 requirement.
Auto-upgrade:
- ColmapEngineDep now defines REQUIRED_MIN=4.1.0 and re-downloads when the
local build is older, so existing users get the GPU-BA/360 build on next
run.bat without manually deleting engines/colmap.
- New i18n keys (check_native_360, check_ba_use_gpu) in all 9 locales.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
…dencies The ruff --fix pass deleted `from app.core.system import resolve_project_root` from setup_dependencies.py, but the module still calls resolve_project_root() in get_venv_360_python() AND re-exports it for app.core.extractor_360_engine. That re-export is imported at GUI startup, so the missing name raised ImportError before the window could open — the launcher terminal closed after a few seconds with no visible error. - Restored the import (now genuinely used at module level, so ruff's unused-import autofix will not strip it again) with a comment explaining why. - Removed the stale "# resolve_project_root is imported from app.core.system" placeholder comment left behind by the autofix. Verified by importing all 34 app.* modules and constructing/showing ColmapGUI headlessly — startup now completes and the event loop runs cleanly. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
Defaults tuned for the video→frames→COLMAP→splat workflow (fast, no quality
loss, blur filter off):
- matcher_type default exhaustive → sequential: O(n) instead of O(n²) and the
correct matcher for ordered video frames. Big speedup on large datasets.
- ba_use_gpu default off → on: GPU bundle adjustment ("Caspar"). Only applied
when COLMAP ≥ 4.1.0 supports it (probed + cached), and COLMAP auto-falls back
to CPU for small scenes, so it's safe as a default.
- Blur filter already off by default (unchanged) — left off per request.
Persistence:
- The COLMAP params were only saved on a clean window close. Now switching
tabs triggers a debounced session save, so a choice like the matcher is
persisted as soon as the user navigates away — connected after load() so it
never overwrites the just-loaded session at startup.
Verified headlessly: fresh GUI shows sequential + GPU BA on + blur off; a
matcher change followed by a tab switch is written to config.json.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
On a re-run, the working images/ folder already holds (hardlinked or copied) images from the previous run. The prepare loop saw target.exists() and made a RENAMED duplicate (images_1_frame_0001.png), so each run grew the dataset. Now each source image is compared against what's already in images/ via a new _same_image() check — same inode (hardlink from a prior run) or byte-identical content (size first, then filecmp). Already-present images are skipped; only genuinely different images that share a filename are disambiguated. The final log reports how many were skipped. Verified: _same_image handles hardlink / identical-copy / different / same-size-different-content; three consecutive _prepare_images runs keep the image count flat instead of multiplying it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
Two related "settings change by themselves" bugs:
1. Wheel-over-widget: the params live in a QScrollArea, and Qt combo/spin
widgets change value when the wheel passes over them. Scrolling the COLMAP
params page silently flipped the matcher (down to the last item =
vocab_tree), and auto-save then persisted the accident. Added a reusable
install_wheel_guard() (app/gui/widgets/wheel_guard.py) that makes combos,
spin boxes and sliders ignore the wheel unless focused, so the page scrolls
instead. Applied to params, brush, upscale and config tabs.
2. Blur re-checked on startup: ConfigTab.set_state restored blur_filter and
THEN robust_mode, but set_robust_mode(True) pre-ticks blur via
_on_robust_toggled — overriding the saved value. Reordered so robust is
applied before blur, letting the saved blur value win.
Verified headlessly: a wheel scroll over the unfocused matcher no longer
changes it; restoring {robust:true, blur:false} keeps blur off.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
Author
|
it s not working right now ^^i ll tell you when it is functionnal
Le ven. 26 juin 2026 à 21:58, freddewitt ***@***.***> a
écrit :
… *freddewitt* left a comment (freddewitt/CorbeauSplat#8)
<#8 (comment)>
It's great idea. I can't merge to my project, but we can do another repo ?
—
Reply to this email directly, view it on GitHub
<#8?email_source=notifications&email_token=B7DNCLGEJK3AAFRO7XVKRXD5B3IVXA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIOBRGMYDAMRZHEZKM4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L#issuecomment-4813002992>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/B7DNCLFTHEGZ2WZYPIPEM735B3IVXAVCNFSNUABGKJSXA33TNF2G64TZHMYTCMRXHEYTMNBYGA5US43TOVSTWNBXGUYTINBTGU2TFILWAI>
.
Triage notifications, keep track of coding agent tasks and review pull
requests on the go with GitHub Mobile for iOS
<https://github.com/notifications/mobile/ios/B7DNCLGLHIHQUYSBM7EBX6T5B3IVXA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIOBRGMYDAMRZHEZKM4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KUZTPN52GK4S7NFXXG>
and Android
<https://github.com/notifications/mobile/android/B7DNCLAQ7KFVBZR5N7DDGC35B3IVXA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIOBRGMYDAMRZHEZKM4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2K4ZTPN52GK4S7MFXGI4TPNFSA>.
Download it today!
You are receiving this because you authored the thread.Message ID:
***@***.***>
--
06-01-91-71-23
Graphiste PAO/WEB
http://lou-portfolio.fr/
Lou Delestre
<http://www.flickr.com/photos/bbmarmotte>
|
…lags - Lead the README by crediting the upstream project (freddewitt/CorbeauSplat) that this Windows/CUDA edition is forked from. - Drop the obsolete --SiftExtraction.use_gpu / --SiftMatching.use_gpu mention (COLMAP 4.x rejects those flags; the CUDA build uses the GPU automatically) and note the video-tuned defaults (sequential matching, GPU bundle adjustment). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
After "Extraction des features terminé", two steps run with no output and make the pipeline look frozen on large datasets: - the sequential-matcher DB sort (image_id → temporal order), and - the matcher loading every image's descriptors into RAM before it prints its first "Matching block" line (several minutes on a big 4K set). Added log + status lines around both so the user knows it's working and not to close the program. No behavior change to the actual reconstruction. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
During mapping, COLMAP can print "Linear solver failure ... dense Cholesky factorization" as a glog W-level warning when one bundle-adjustment step fails — the solver retries and registration continues. It reads like a crash to users. Annotate it once in the log with a clear "non bloquant" note. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
Repeated "Linear solver failure" during the Global bundle adjustment signals a numerically hard scene. Extend the one-time note to point users at the GPU bundle adjustment (COLMAP 4.1.0) or robust mode to stabilise it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
The mapper has no fixed total, so there's no clean %, but num_reg_frames is the
real progress counter (images successfully placed). Surface it in the status
bar ("Reconstruction 3D : 404 images placées") instead of the raw image id, so
the user can see where they are without reading the log.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
Multi-dimension performance pass. All changes are pure speedups or tunable defaults — no reconstruction-quality regression. Startup (time-to-window): - app/cli/__init__.py: lazy-import the heavy CLI engine stack (.commands → ColmapEngine/BrushEngine/app.core.engine) only on the CLI branch; the GUI default path no longer pays for it. check_dependencies stays eager (cheap). - ply_cleaner.py: import numpy lazily inside the compute functions (cleaner_tab pulls this in at startup via resolve_params, which needs no numpy). - engine.py: lazy-import send2trash inside the delete path (off the startup chain). Confirmed: `import app.cli` no longer loads numpy/send2trash/cv2/plyfile. Image preparation: - Single-video fast path in resolution check: frames from one ffmpeg run share a resolution, so sample the first frame instead of header-reading every file (O(n) → O(1)). Multi-video/image-folder inputs keep the full scan. - Parallelize the resolution-resize pass (cv2 releases the GIL). - _same_image: short-circuit on matching size+mtime (copy2 preserves mtime) so no-op re-runs skip the full byte compare; filecmp remains the fallback. COLMAP mapper (the 10-15h bottleneck): - Pin OMP/OpenBLAS/MKL to 1 thread for COLMAP children. COLMAP already parallelizes via --*.num_threads; letting BLAS also spawn N threads caused N×N oversubscription that thrashes the CPU and worsens the "Linear solver failure" churn during global BA. One source of parallelism, not nested. - Bound bundle adjustment via new tunable ColmapParams fields (faster than COLMAP defaults, safe for 3DGS): ba_global_max_num_iterations=30, ba_global_function_tolerance=1e-6, ba_global_images_ratio/points_ratio=1.2, ba_local_max_num_iterations=20 — caps global-BA iterations and how often it re-runs. Tunable via config.json back toward COLMAP defaults if needed. Misc: - base_engine: line-buffer subprocess stdout (bufsize=1) so BA progress flushes promptly instead of looking frozen. Tests: - Repoint CLI dispatch patches to app.cli.commands.DISPATCH (now lazy). - Repoint send2trash patches to the real module (now a lazy import). - Fix pre-existing test_managers fixture gap: mock cleaner_tab (it was added to the save mapping earlier but never mocked). Verified: full app.* import sweep clean; 130 core + 89 affected tests pass; GUI constructs/runs headlessly; mapper builds the new BA flags; run_command pins BLAS env to 1. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
A focused correctness review of engine.py (heavily churned) surfaced three real
bugs in the video-frame / resolution-resize path:
1. _imwrite_unicode lossy/failing re-encode on resize. IMREAD_UNCHANGED can
return a 4-channel image; cv2.imencode(".jpg", rgba) FAILS → the frame was
left at its original size, leaking mixed resolutions into COLMAP. JPEG also
re-encoded at the default quality 95 (a second lossy generation). Fix: drop
the alpha channel before JPEG encode and encode at quality 100. Verified
RGBA→JPEG now succeeds and PNG alpha is preserved.
2. Video prefix collision. Two source videos whose stems sanitize to the same
string (e.g. "a.b" and "ab") — or to an empty string — produced the same
frame prefix, so the second extraction overwrote the first's frames and the
per-prefix count was wrong. Fix: prefix the enumeration index
(f"{i:03d}_{sanitized}", or f"{i:03d}" when empty) → distinct, non-empty.
3. Stale frames on video re-run. extract_frames_from_video never cleared old
frames, so re-running with a changed fps/source left higher-numbered stale
frames that COLMAP then ingested. Fix: unlink this prefix's existing
*.jpg before extracting (only this prefix is touched).
The review verified the recently-added perf code (mapper ba_* flags, run_command
BLAS pinning, _same_image, _is_single_video_input, DB sort) is correct.
Verified: prefix uniqueness, per-prefix frame clearing, and imwrite alpha/quality
all tested; 89 affected tests pass; full app.* import sweep clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
Frame extraction used bare '-hwaccel cuda', which frequently still decodes on the CPU. Now the input codec is probed (ffprobe) and the matching NVDEC cuvid decoder is selected (-c:v h264_cuvid / hevc_cuvid / …) when available, moving decode onto the GPU — 2-5x faster extraction on long/high-res video. Safe by construction: the standard CPU decode is always the last attempt, so a missing/incompatible cuvid decoder can never break extraction. The cuvid path is only chosen when both (a) ffprobe identifies a mappable codec and (b) the decoder is present in `ffmpeg -decoders` (probed once, cached). If the GPU attempt yields a non-zero return or zero frames, it clears partial frames and retries on CPU. Also drops audio (-an) and keeps the per-prefix frame cleanup from the previous fix (now run before each attempt). New helpers: _cuvid_decoder_for, _probe_video_codec, _ffprobe_bin (derived from ffmpeg's path), _available_cuvid_decoders (cached). Verified: decoder selection (codec→cuvid, availability gating), GPU→CPU fallback (2 attempts, CPU succeeds), and the no-CUDA single-attempt path are unit-tested; 64 affected tests pass; full app.* import sweep clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
…ions PR #26 added --Mapper.ba_global_* / ba_local_* flags unconditionally. COLMAP 4.1.0 renamed/removed some of them (ba_global_images_ratio / ba_global_points_ratio), so the mapper aborted with "unrecognised option '--Mapper.ba_global_images_ratio'" right after a successful match — losing the whole run. Now each optional --Mapper.* flag is gated on `colmap mapper -h` advertising it, using a single cached help-text probe shared with the GPU-BA check: - supported flags are passed (e.g. ba_global_max_num_iterations, ba_use_gpu), - unsupported ones are skipped with an info log, - if the help probe fails/returns nothing, all optional flags are skipped and the mapper runs with COLMAP defaults (correct, just unbounded). Verified across three simulated builds: 4.1.0 (skips *_ratio, keeps the rest + GPU BA), empty-help (skips all optional flags, base mapper intact), and 3.x (keeps *_ratio, omits GPU BA). 51 affected tests pass; import sweep clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
… run A failed/cancelled mapper used to force redoing feature extraction + matching from scratch (the DB was wiped at the start of every run). Now the pipeline reuses an existing database.db and jumps straight to the mapper when it is provably safe to do so. Reuse happens only when ALL hold (any mismatch → full rebuild): - a sidecar signature (database.db.sig.json) matches the current extraction/matching params AND image count — so changing any COLMAP setting (max_image_size, features, matcher, overlap, …) still forces a fresh run; - the DB's image names equal exactly the current image set; - every image has descriptors (extraction complete); - two_view_geometries (verified matches) is non-empty (matching complete). The signature is written after a successful match; the sparse model is still always rebuilt fresh. On any DB/SQLite/JSON error the guard returns False, so a stale or corrupt DB can never be silently reused. This directly helps the just-fixed mapper-abort case: re-running now skips the (already-done) extraction + matching and resumes at the mapper. Verified with a synthetic COLMAP-schema DB across 7 cases (valid→reuse; image mismatch, param change, no matches, partial features, no signature, no DB → all redo). 64 affected tests pass; import sweep clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
Re-running an image-folder project created duplicate files like "Photos-3-001_1_IMG_xxx.jpg" alongside "IMG_xxx.jpg", compounding every run. Root cause: the first run resizes mismatched-resolution images in place (_check_and_normalize_resolution), changing their bytes/size. On a re-run the old content-based dedup (_same_image) then saw source≠target and fell into the "<folder>_1_<name>" disambiguation path — re-importing every image under an alias. COLMAP then ingested both copies. Fix: import provenance is now tracked by SOURCE PATH in a manifest (project/import_manifest.json), not by content: - a source already imported (even after an in-place resize) is recognised by its absolute path and adopted, never re-imported under an alias; - target names are claimed deterministically; two genuinely different sources sharing a basename are still disambiguated; - self-heal: stale top-level images that no current source claims (old "<folder>_N_<name>" duplicates, or frames of a removed source) are deleted, so an already-messy project cleans itself on the next run. Subfolders (images_src, blurry) and mask files are left untouched. Removed the now-unused _same_image helper. Verified across 5 filesystem scenarios: fresh import; re-run after in-place resize (no duplicates); existing messy project (orphan duplicate removed); two distinct sources sharing a basename (both kept, re-run stable); source removed (its target dropped). 64 affected tests pass; import sweep clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
….1.0 The "Retriangulation and Global bundle adjustment" step is dominated by how many BA+filter refinement passes each global-BA trigger runs, and how often global BA is triggered. Two improvements (both probe-gated, version-safe): 1. Cap refinements — the single biggest in-mapper lever: - ba_global_max_refinements 5 → 3 (global BA+filter passes per trigger) - ba_local_max_refinements 2 → 1 (local BA runs after every image) 2. Fix global-BA frequency control on COLMAP 4.1.0: COLMAP 4.1.0 renamed --Mapper.ba_global_images_ratio → ba_global_frames_ratio (rig/frame terminology). We only sent the old name, which 4.1.0 rejects → it was silently dropped, leaving global BA at the slow default cadence (1.1). Now we pass BOTH names; the probe sends whichever the build advertises, so the "run global BA less often" tuning (1.2) actually applies on 4.1.0 and 3.x. Confirmed option names + defaults against COLMAP source (controllers/incremental_pipeline.h): ba_global_max_refinements=5, ba_local_max_refinements=2, ba_global_frames_ratio=1.1. Verified: new params round-trip; simulated 4.1.0 help → frames_ratio used (freq restored), refinements applied, images_ratio skipped; simulated 3.x → images_ratio used, frames_ratio skipped. 51 affected tests pass; import sweep clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
Brush is built on burn-fusion, which can panic mid-training with "Ordering is bigger than operations" (a known burn stream-ordering / tensor- handle bug). There is no runtime flag to disable fusion, but switching the wgpu backend (DX12 ↔ Vulkan) reschedules the op stream and usually dodges it. train() now detects the panic in Brush's output and, on a fusion crash, retries once on the alternate GPU backend (DX12 default → Vulkan). Non-fusion failures are NOT retried. The backend is also overridable via the `wgpu_backend` param (config.json) for users who want to pin Vulkan from the start. build_command gains an optional backend_override (backward compatible; default stays DX12). Verified: existing 17 brush tests pass; new tests cover fusion-panic→Vulkan retry, non-fusion failure→no retry, and success→single attempt. Import sweep clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
…fragment) With multiple_models on (robust mode), COLMAP writes sub-models sparse/0, sparse/1, … in no size order. Brush, the undistorter, and create_brush_config all use sparse/0, so when the scene fragments, training could land on a tiny fragment (observed: 11 of ~3000 images → a spiky, useless splat). After mapping, _promote_largest_sparse_model() now reads each sub-model's registered-image count (the uint64 header of images.bin, with an images.txt fallback) and swaps the largest model into the sparse/0 slot, so Brush trains on the full reconstruction. No sub-model is deleted — only reordered. Verified: fragment promotion (sparse/1=2000 → sparse/0, all models preserved), already-largest unchanged, single-model no-op, images.txt fallback count. 27 colmap tests pass; import sweep clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
The user's log showed the real trigger of the burn-fusion panic: on DX12 the device reports maxComputeInvocationsPerWorkgroup=768, but Brush's reduce kernels need 1024 → wgpu "Validation Error" in create_compute_pipeline → burn-fusion panic. Vulkan exposes the full 1024, so it trains fine. - Default wgpu backend on cuda/auto is now Vulkan (was DX12); DX12 becomes the automatic fallback (order reversed in train()). - Crash detection also matches "wgpu error" / "validation error", so even a non-panicking wgpu compute-limit error triggers the backend retry. - Still overridable via wgpu_backend / backend_override. Updated the two tests that asserted the DX12 default; added an override test. 18 brush tests pass; vulkan→dx12 fallback on a wgpu validation error verified; import sweep clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
A user's COLMAP 4.1.0 accepts --Mapper.ba_use_gpu but its Ceres was compiled without CUDA/cuDSS, so bundle adjustment silently falls back to CPU: "Requested to use GPU ... but Ceres was compiled without CUDA support." Our pre-run log claimed "Bundle adjustment : GPU (CUDA) ✅", which was wrong and hid the real cause of the slow mapper. - Soften the pre-run message: GPU BA is only effective if the build has Ceres+CUDA/cuDSS (confirmed at runtime). - Detect the runtime "compiled without CUDA/cuDSS" warning once and explain that BA is actually on the CPU, and that GLOMAP (or a cuDSS-enabled COLMAP) is the way to actually accelerate it. 27 colmap tests pass; import sweep clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
The user wants faster reconstruction (global SfM does one global BA instead of the repeated per-image one). The old path built GLOMAP from source (MSVC + CUDA + CMake/Ninja) — heavy and likely to fail. COLMAP 4.0+ merged GLOMAP in as the built-in `colmap global_mapper`, so the colmap.exe already installed can do it with no extra build. - mapper() now dispatches to _global_mapper (use_glomap) or _incremental_mapper. - _global_mapper prefers `colmap global_mapper` (probe-gated via `colmap help`), running `view_graph_calibrator` first to improve intrinsics (global SfM relies on them). Falls back to a standalone glomap binary, then to incremental — each gated so nothing aborts on an unsupported build. - GlomapEngineDep.is_installed() now also returns True when colmap advertises global_mapper, so enabling GLOMAP no longer triggers a source build. Drive-by (pre-existing breakage surfaced by the test run): - Re-export relax_requirements from setup_dependencies (the test imported it but it lived only in installers/tools) and add __all__ so ruff --fix never strips the compat re-exports again (a stripped re-export previously broke startup). Verified: global_mapper(+calibrator) used when available; fallbacks to standalone glomap then incremental; mapper() dispatch on use_glomap; relax_requirements test passes; 45 engine tests pass; import sweep clean; ruff clean on the shim. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
The user's reconstruction placed the same building in two different 3D spots — a loop-closure failure. Sequential matching only matches frames that are close in TIME, so when the camera revisits a place (common in video walkthroughs) the two visits aren't linked and the location gets reconstructed twice. No mapper (incl. GLOMAP) can fix matches that were never made. - New ColmapParams.loop_detection (default True) added to the DB-resume signature, so enabling it forces a fresh match. - feature_matching (sequential) now passes --SequentialMatching.loop_detection 1 with a COLMAP vocabulary tree, gated on: the param, the option existing in `colmap sequential_matcher -h`, and the vocab tree being available. - _ensure_vocab_tree() downloads vocab_tree_flickr100K_words32K.bin into engines/ once (streamed, atomic via .part rename); returns None on failure so matching proceeds without loop detection — never blocks. - _colmap_cmd_help() caches per-subcommand help for option probing. Verified: flags added only when supported + vocab present; skipped gracefully on missing vocab, unsupported option, or param off; loop_detection in the resume signature. 27 colmap tests pass; import sweep clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
The COLMAP params tab header read "Apple Silicon detected - N threads" — a leftover from the macOS origin, shown on Windows/NVIDIA machines. Replaced the info_cpu string in all 9 locales with an accurate, platform-neutral "CPU: N threads detected". Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
COLMAP switched its vocabulary-tree index from FLANN to FAISS in May 2025. The auto-downloaded flickr100K/32K tree was the legacy FLANN format, so COLMAP 4.1.0 aborted feature matching with "Failed to read faiss index" whenever loop detection was enabled — breaking the whole pipeline. - _ensure_vocab_tree() now fetches a FAISS-format tree (engines/ vocab_tree_flickr100K_words32K_faiss.bin) and validates the download size. - feature_matching() is now failure-proof: if the sequential matcher with loop detection fails, it automatically retries WITHOUT loop detection, so matching always produces correspondences instead of aborting. - matcher_type='vocab_tree' now runs the real vocab_tree_matcher (was silently falling through to exhaustive), with a graceful fallback to exhaustive matching if the tree is unreadable or unavailable. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
Multi-agent adversarial audit of the whole video→COLMAP→Brush pipeline. Confirmed defects fixed (false positives discarded after verifying against the actual code, e.g. quadratic_overlap is valid; the sparse-swap rename is already guarded by except OSError): - Frame extraction quality: ffmpeg -qscale:v 2 → 1 (highest JPEG quality) so frames feeding COLMAP/Brush keep maximum detail. - Mixed-resolution unification now resizes to the LARGEST resolution (INTER_CUBIC up / INTER_AREA down) instead of the smallest — the previous behaviour permanently discarded detail from the sharpest cameras. - WAL→DELETE DB conversion (GLOMAP path) now closes the SQLite connection explicitly before deleting -wal/-shm, fixing a Windows file-lock failure; checkpoint runs before the journal-mode switch and is committed. - _execute_command terminates the child process if streaming/parsing throws, so a COLMAP/Brush process is never orphaned (zombie holding GPU/port). - Glomap install verifies the binary copy actually landed before recording the version (a failed copy no longer marks it "installed" forever). - CLI matcher_type default aligned with the engine/GUI: exhaustive → sequential (fast and correct for ordered video frames); added --sequential_overlap and --no_loop_detection and wired them into run_colmap / run_pipeline. - GUI: added a "Détection de boucles" checkbox (loop_detection) to the COLMAP params tab so it can be seen/toggled, wired through get_params/set_params. All tests pass; compile + import sweep clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
… CLI ply_name Second adversarial audit pass over the subsystems the first didn't cover (export/PLY, viewer, upscale, GUI threading, param flow). Confirmed defects fixed (dangerous/incorrect agent suggestions discarded after verifying against the code, e.g. blocking empty-checksum downloads would break all installs; adding a --refine-mode flag Brush may not accept could break training): - SPZ export was 100% broken: export_engine.py used math.sqrt without importing math (NameError) and called .get() on a numpy.void structured scalar (AttributeError). Added the import; index fields directly (the enclosing `if 'scale_0' in data.dtype.names` guards guarantee presence). - Manual ASCII PLY parser returned zero points: it iterated the already- consumed header instead of the file body. Now reads the vertex rows from the remaining file stream (verified: 3-vertex ASCII PLY now parses). - SuperSplat data server reported success even when the port bind failed (error was only logged in the worker thread). It now waits on the bind and returns a real error; the Cleaner preview checks that return. - CLI --ply_name was silently ignored (only the GUI renamed the output). Added a shared rename_latest_ply() helper, wired into run_brush and run_pipeline. Targeted tests pass (78); compile + import sweep clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
…, phantom Brush flags Focused audit of camera placement (SfM) and Brush training. Fixes: MAPPING (camera placement): - GLOMAP/global_mapper was throttled to a SINGLE thread: run_command pins OMP_NUM_THREADS=1 (anti-oversubscription for the incremental mapper), but global_mapper has no --num_threads flag and derives its thread count from OMP_NUM_THREADS. run_command now takes omp_threads and the global-mapper path (view_graph_calibrator included) gets the full core count. - global_mapper/glomap received ZERO options — GUI settings (GPU BA, intrinsics refinement) were silently dropped. New probe-gated _global_mapper_opts() forwards BundleAdjustment/GlobalPositioning use_gpu + gpu_index, optimize_intrinsics/principal_point, and skips the view-graph calibration inside global_mapper when we already ran it as a separate step. ba_global_max_num_iterations is deliberately NOT mapped (tuned low for the repeated incremental BA; the single global BA keeps GLOMAP's own budget). - vocab_tree matching: raise retrieval breadth (num_images 150, nearest neighbors 8, probe-gated) and add a sequential TOP-UP pass after a successful vocab-tree pass — matches accumulate in the same database, and temporally-adjacent frames reconnect views the retrieval missed (the user's run left 88/2031 images unconnected). DB image-id sort now also runs for vocab_tree (required for sequential adjacency; the sort wipes matches, so it stays strictly before matching). - _sort_colmap_database_images now sorts NATURALLY (frame_9999 < frame_10000) instead of SQL lexicographic order. TRAINING (Brush): - ALLOWED_FLAGS contained flags that DO NOT EXIST in Brush v0.3.0 and make it abort via clap: --refine-pose, --test-split, --save-iterations, --log-level (the GUI placeholder even advertised --refine_pose). Verified against the v0.3.0 TrainConfig/LoadDataseConfig/ProcessConfig sources. Replaced with the real flag set: --eval-split-every, --ssim-weight, --lpips-loss-weight, --opac-loss-weight, lr-* etc. Tests updated (phantom flags now asserted rejected). - Resolution hint corrected in all locales: Brush's built-in default cap is 1920 px (not 1080) — with the spinbox at 0 the flag is omitted and Brush silently trains at 1920. - Undistortion was DEAD WEIGHT for training: dense/ was written but Brush always trained on the original distorted images (brush_config.json has no reader). The undistorted model is now normalized to dense/sparse/0 and the GUI + CLI pipeline route Brush to dense/ when it exists. - Robust mode's silent camera-model override (→ PINHOLE, no distortion modelling) is now surfaced in the logs with guidance. 95 targeted tests pass; locales JSON validated; natural sort verified. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Convert the macOS/Apple Silicon Gaussian Splatting tool into a Windows-only fork focused on the video -> frames -> COLMAP -> Brush splat training -> view pipeline, accelerated by NVIDIA CUDA.
Platform core
Launcher & installers
Removed Apple ML Sharp
Docs & tests
Claude-Session: https://claude.ai/code/session_01GCByXMmnQgpLN6LgSLhEZ3