feat: extras-based modular install & registry refactor (v1.5.0)

.github/workflows/ci-cd.yml

@@ -53,7 +53,7 @@ jobs:
 
       - name: Install dependencies
         run: |
-          uv sync --dev
+          uv sync --dev --all-extras
 
       - name: Lint with ruff
         run: |
@@ -69,6 +69,14 @@ jobs:
           uv run mypy src/videoannotator
 
       - name: Test with pytest
+        env:
+          # macOS ARM64: numpy 2.x + pyarrow (via datasets/pandas) each bundle their
+          # own OpenMP runtime, which can double-initialize alongside TensorFlow's
+          # and abort the process (Fatal Python error: Aborted) on import, not on
+          # any actual test failure. Same class of issue already documented for
+          # audio pipelines in docs/installation/INSTALLATION.md's macOS libomp
+          # note. No-op on Linux/Windows.
+          KMP_DUPLICATE_LIB_OK: "TRUE"
         run: |
           uv run pytest -q --cov=src/videoannotator --cov-report=xml --cov-report=term-missing --cov-fail-under=45
 
@@ -106,7 +114,7 @@ jobs:
 
       - name: Install dependencies
         run: |
-          uv sync --dev
+          uv sync --dev --all-extras
 
       - name: Run integration tests
         run: |
@@ -143,7 +151,7 @@ jobs:
 
       - name: Install dependencies
         run: |
-          uv sync --dev
+          uv sync --dev --all-extras
 
       - name: Run performance tests
         run: |

.specify/feature.json

@@ -0,0 +1,3 @@
+{
+  "feature_directory": "specs/004-extras-based-install"
+}

CHANGELOG.md

@@ -7,14 +7,144 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
-### Planned for v1.5.0
+### Planned
 
 - Queue position display for pending jobs
 - Deterministic test fixtures with synthetic video generation
 - Research workflow examples for JOSS paper
 - Benchmark results and performance validation
 - Additional contributor documentation improvements
 
+## [1.5.0] - 2026-07-19
+
+### Extras-Based Modular Install & Registry Refactor
+
+Implements `specs/004-extras-based-install/` — moves heavy ML pipeline dependencies out of the
+core install and into opt-in `pip` extras, and replaces the registry's hardcoded pipeline↔module
+mapping with metadata-driven resolution.
+
+#### Added
+
+- **Per-pipeline-family extras**: `pip install videoannotator[scene]` (or `face`, `person`,
+  `audio`, `face-laion`, `face-openface3`, `audio-laion`) now pulls in only that family's
+  dependencies; a plain `pip install videoannotator` installs no torch, no ML pipelines at all.
+  `videoannotator[all]` reproduces the pre-v1.5.0 "everything installed" behaviour.
+- **`PipelineMetadata.requires_extras`**: new field (`list[str]`, default `[]`) read from each
+  pipeline's YAML metadata; `module_path` is now a required field with no hardcoded fallback.
+- **Graceful degradation**: pipelines whose extras aren't installed are omitted from
+  `GET /api/v1/pipelines`/`videoannotator pipelines` by default (`?include_unavailable=true` /
+  `--all` shows them with an `install_hint`); submitting a job for an unavailable pipeline returns
+  `422` with the exact `pip install videoannotator[...]` command instead of a crash.
+- **Migration message**: a v1.4.x config referencing a pipeline demoted out of the default install
+  (`face_laion_clip`, `laion_voice`, `face_openface3_embedding`) gets a message naming the extras
+  group and explaining it's "no longer installed by default as of v1.5.0", distinct from the
+  generic unavailable-pipeline error.
+- Install-matrix documentation in `docs/installation/INSTALLATION.md` covering per-use-case
+  installs ("I want only scene labelling", "I want everything", "I want a slim API server").
+
+#### Changed
+
+- `registry/pipeline_loader.py`: removed `LEGACY_MAPPINGS`/`_infer_module_path`; pipeline classes
+  now resolve purely from metadata (`module_path`), gated by a cheap
+  `importlib.metadata`-based extras-availability check (no heavy imports at registry-load time).
+- `Dockerfile.cpu`/`Dockerfile.gpu`/`Dockerfile.dev`: CPU/GPU production images build slim (no
+  extras) by default; pass `--build-arg EXTRAS=<group[,group...]>` (or `EXTRAS=all`) for a
+  pipeline-enabled image. The dev image installs `--extra all` by default (unchanged behaviour).
+- Dropped the `numpy<2.0` pin; `numpy` now resolves per `numba`'s own declared ceiling (currently
+  numpy 2.2.x) instead of a hand-maintained upper bound. Retired the now-redundant `numpy2-test`
+  CI job — the default `test` job exercises numpy 2.x directly now that the pin is gone.
+
+#### Fixed
+
+- `face` extras group was missing `tf-keras`, which `deepface`'s `retinaface` backend requires
+  alongside Keras-3-era `tensorflow`; without it, importing `face_pipeline` raised `ValueError`
+  before any face-analysis code could run. Added `tf-keras>=2.15.0` to the `face` extras group.
+- `videoannotator/utils/audio.py` did a module-level `import librosa`, and `librosa` is an
+  `audio`/`audio-laion` extra, not a core dependency. Because `utils/__init__.py` re-exports
+  `find_f0` from that module, and `cli.py`/`version.py` import `utils` at startup, **the entire
+  CLI crashed with `ModuleNotFoundError: No module named 'librosa'` on any install without audio
+  extras** — including `[scene]`-only installs, defeating the whole point of this feature. Caught
+  via a real `pip install videoannotator[scene]` + `videoannotator pipelines list` run (quickstart
+  §1). Made the `librosa` import lazy (moved inside `find_f0`, the only caller) since `find_f0`
+  itself is unused elsewhere in the codebase.
+- **`[tool.setuptools.package-data]` never declared `registry/metadata/*.yaml`** — only
+  `viewer_static/**/*` was listed. Every pipeline's YAML metadata (`module_path`,
+  `requires_extras`, etc.) was silently absent from any real (non-editable) install; the registry
+  found **zero pipelines regardless of which extras were installed**. This was likely always true,
+  but harmless before this phase because `LEGACY_MAPPINGS` gave the loader a hardcoded fallback
+  `module_path` to fall back on. T013 removed that fallback, making the YAMLs a hard runtime
+  dependency — so this became a full-outage regression the moment the registry went
+  metadata-only. Caught via a real `pip install .[scene]` + `videoannotator pipelines` run
+  reporting `[OK] Pipelines: 0 found`. Added `"registry/metadata/*.yaml"` to `package-data`;
+  verified with `uv build --wheel` that all 9 metadata YAMLs are now present in the built wheel.
+- `scene` extras group declared `scenedetect[opencv]`, pulling in `opencv-python` (the full/GUI
+  build) redundantly alongside the `opencv-python-headless` already required at core — the exact
+  duplicate-`cv2`-distribution problem the `face` extras group's own comment says to avoid. Also
+  version-fragile: scenedetect 0.7 dropped the `opencv` extra name entirely, producing `WARNING:
+  scenedetect 0.7 does not provide the extra 'opencv'` on install. scenedetect has no unconditional
+  cv2 dependency of its own (only via that extra), so dropped it — core's `opencv-python-headless`
+  already satisfies it at runtime. Caught during a real `pip install videoannotator[scene]` run.
+- **The single biggest bug of this phase**: `videoannotator/pipelines/__init__.py` unconditionally
+  imported every pipeline family at package-init time (`AudioPipeline`, `FaceAnalysisPipeline`,
+  `LAIONFacePipeline`, `PersonTrackingPipeline`, `SceneDetectionPipeline`). Since Python always
+  runs a parent package's `__init__.py` before any of its submodules, loading *any single*
+  pipeline through the registry (`importlib.import_module("videoannotator.pipelines.scene_detection")`,
+  etc.) forced every other family's heavy deps to import too — regardless of which extras were
+  actually installed. In practice this meant **no pipeline could ever load successfully unless
+  every extras group was installed simultaneously**, silently defeating this entire phase's reason
+  for existing. `audio_processing/__init__.py` and `face_analysis/__init__.py` had the same bug one
+  level down: eagerly importing `LAIONVoicePipeline` (needs `audio-laion`, not a subset of `audio`'s
+  deps) and `LAIONFacePipeline` (needs `face-laion`'s torch/transformers, absent from plain `face`)
+  alongside their same-extras-group siblings. Caught live: a real `[scene]` install's `job submit
+  --pipelines scene_detection` failed server-side with `No module named 'librosa'` — from
+  `scene_detection`, which has nothing to do with audio. Fixed all three `__init__.py` files with
+  PEP 562 lazy (`__getattr__`-based) attribute resolution instead of eager imports, so importing one
+  pipeline no longer drags in siblings from other extras groups. Also surfaced a genuine (not just
+  packaging-level) coupling while fixing this: `LAIONFacePipeline` composes `FaceAnalysisPipeline`
+  as its internal face-detector backend, so `face-laion` alone was never actually functional without
+  `face`'s deepface stack — `pyproject.toml`'s `face-laion` group now depends on
+  `videoannotator[face]`. Verified with a mocked-import harness simulating five slim-install
+  scenarios (`scene`, `person`, `face`, `audio`, `face-openface3` — each with every *other* family's
+  heavy deps blocked at `__import__` level) — all five now import cleanly. Full suite still green
+  post-fix (1065 passed, 33 skipped) and noticeably faster (~4 min vs ~13 min) since test collection
+  no longer forces every pipeline family's imports for every test file.
+- Core declared `opencv-python-headless` unconditionally, while `face` (deepface/retina-face) and
+  `person` (ultralytics/supervision) transitively force plain `opencv-python`, uncapped, from their
+  own dependency declarations — so any `face`/`person` install ended up with both variants sharing
+  the same `cv2` install path, a documented upstream footgun
+  (github.com/opencv/opencv-python#note-1) that corrupts the compiled module. A real
+  `face_laion_clip` job hit it live: `AttributeError: module 'cv2' has no attribute
+  'CascadeClassifier'`. Worse, **removing one variant afterwards doesn't repair an
+  already-corrupted venv** — the leftover `cv2/` directory stays broken (confirmed hitting the same
+  corruption in this project's own dev venv mid-fix) until manually removed and reinstalled clean.
+  Fixed by never declaring `opencv-python-headless` anywhere and standardizing on plain
+  `opencv-python` project-wide. A second, independent bug surfaced immediately after: `opencv-python`
+  5.0 (newer than this project's original unbounded `>=4.11.0.86` pin) **removed
+  `cv2.CascadeClassifier` and all Haar-cascade data entirely**, replaced by the DNN-based
+  `FaceDetectorYN` — `face_pipeline.py`'s "opencv" backend still uses the legacy API. Pinned
+  `opencv-python<5.0` everywhere it's declared (`face`, `person`, `scene`, `face-openface3`);
+  migrating to `FaceDetectorYN` is real follow-up work, not a dependency-pin fix.
+- `scene_detection`'s CLIP-based scene classification failed with `too many values to unpack
+  (expected 2)`: `scene_pipeline.py` called `open_clip`'s `model(image, text)` expecting the legacy
+  2-tuple `(logits_per_image, logits_per_text)`; `open-clip-torch` 3.1.0 (unpinned upper bound)
+  returns `(image_features, text_features, logit_scale)` instead — a real open_clip API drift, not
+  a packaging issue. Fixed by computing the similarity logits directly
+  (`logit_scale * image_features @ text_features.T`), matching open_clip's current usage pattern.
+- `face_analysis`'s DeepFace "emotion" action logged repeated `No DNN in stream executor` warnings
+  on GPU installs. Root cause: torch hard-pins `nvidia-cudnn-cu12==9.1.0.70` (no version range);
+  `tensorflow` (deepface's transitive dependency) wants cuDNN `>=9.3.0.75` for GPU use via its own
+  `[and-cuda]` extra — which nothing here installs, so it just borrows torch's older cuDNN instead
+  and fails at runtime. These two pins can never both be satisfied in one venv (confirmed: no
+  tensorflow release targets the exact patch torch pins), so "find a compatible tensorflow build"
+  wasn't actually viable. Fixed by forcing TensorFlow onto CPU before `deepface` touches the GPU
+  (`tf.config.set_visible_devices([], "GPU")`, called from `face_pipeline.py` before importing
+  `deepface`) — this uses TensorFlow's own device-visibility API, not the `CUDA_VISIBLE_DEVICES` env
+  var, so torch's own GPU usage (scene/person/face-laion's CLIP/YOLO models) is untouched. Verified:
+  `torch.cuda.is_available()` still `True` post-fix; `DeepFace.analyze(..., actions=["emotion"])`
+  completes cleanly with zero cuDNN warnings. Escape hatch:
+  `VIDEOANNOTATOR_DEEPFACE_GPU=1` skips the forced-CPU behavior for anyone who's resolved the
+  mismatch themselves (e.g. a matching system-wide CUDA/cuDNN install).
+
 ## [1.4.4] - 2026-07-08
 
 ### Modularity, Viewer Integration, and JOSS Resubmission

CITATION.cff

@@ -28,5 +28,5 @@ authors:
     orcid: "https://orcid.org/0000-0001-5846-3444"
 license: "MIT"
 repository-code: "https://github.com/InfantLab/VideoAnnotator"
-version: "1.4.4"
-date-released: "2026-07-08"
+version: "1.5.0"
+date-released: "2026-07-19"

CLAUDE.md

@@ -0,0 +1,54 @@
+# VideoAnnotator Development Guidelines
+
+Auto-generated from feature plans by `.specify/scripts/bash/update-agent-context.sh`. Last updated: 2026-07-18
+
+## Active Technologies
+- Python 3.12 (`requires-python = ">=3.12,<3.13"`), FastAPI, SQLAlchemy, Pydantic, Typer/Click (core — stays required with no extras installed)
+- Per-pipeline extras (torch, ultralytics, pyannote.audio, transformers, deepface, open-clip-torch, openai-whisper, etc.) — being moved from required dependencies to `[project.optional-dependencies]` groups (`face`, `face-laion`, `face-openface3`, `audio`, `audio-laion`, `scene`, `person`, `all`) as of 004-extras-based-install
+- SQLite/SQLAlchemy for job/pipeline state; local filesystem model cache (HF/torch cache dirs)
+
+## Project Structure
+```
+src/videoannotator/
+├── api/            # FastAPI app, job submission/status endpoints
+├── batch/          # batch_orchestrator — CLI batch job execution path
+├── cli.py          # videoannotator CLI entry point
+├── pipelines/      # face_analysis/, audio_processing/, scene_detection/, person_tracking/
+├── registry/       # pipeline_registry.py, pipeline_loader.py, metadata/*.yaml
+├── storage/        # job/annotation storage backends
+└── exporters/      # COCO/RTTM/WebVTT/native-format writers
+tests/
+├── unit/ integration/ pipelines/ api/ contract/
+specs/<NNN>-<slug>/  # spec-kit feature specs (spec.md, plan.md, tasks.md, ...)
+docs/development/roadmap_v1.{5,6}.0.md, roadmap_v1.7_to_v2.0.md  # release roadmap
+```
+
+## Commands
+```bash
+pytest tests/                 # full suite
+pytest tests/ -k acceptance   # v1.4.x behaviour-parity fixtures
+ruff check .                  # lint (see pyproject.toml [tool.ruff])
+mypy src/videoannotator        # type check
+pre-commit run --all-files    # full pre-commit gate (used on every commit)
+videoannotator pipelines list # CLI: list available pipelines
+videoannotator job submit <video> --pipelines <name>
+```
+
+## Code Style
+Python 3.12, ruff-enforced (line-length 88, see `[tool.ruff]` in `pyproject.toml` for the
+per-file-ignore exceptions). Follow standard conventions; no comments explaining *what* code does,
+only non-obvious *why*.
+
+## Constitution
+`.specify/memory/constitution.md` (v1.0.0) is binding — five core principles (Local-First
+Execution, Stable Pipeline Contract, Provenance & Reproducibility, Modular by Construction,
+Backward Compatibility by Default). `/speckit-plan`'s Constitution Check gate evaluates every plan
+against it.
+
+## Recent Changes
+- 004-extras-based-install: extras-based modular install + metadata-driven registry loading
+  (removes `LEGACY_MAPPINGS`, adds `requires_extras` to `PipelineMetadata`), scoped to leave room
+  for v1.6.0's Ollama backend and v1.7+'s remote/HPC dispatch without another schema migration.
+
+<!-- MANUAL ADDITIONS START -->
+<!-- MANUAL ADDITIONS END -->

Dockerfile.cpu

@@ -1,8 +1,15 @@
 # VideoAnnotator Production Docker Image - CPU Version
 # This image does NOT include models/weights - they download automatically on first use
 #
+# By default this builds a SLIM image with no pipeline extras (no torch, no
+# ML pipelines) — 004-extras-based-install / SC-002. Pass --build-arg
+# EXTRAS=<group[,group...]> to include one or more pipeline families, or
+# EXTRAS=all to reproduce the pre-v1.5.0 "everything installed" image.
+#
 # Usage:
-#   docker build -f Dockerfile.cpu -t videoannotator:cpu .
+#   docker build -f Dockerfile.cpu -t videoannotator:cpu .                        # slim (no extras)
+#   docker build -f Dockerfile.cpu --build-arg EXTRAS=scene -t videoannotator:cpu-scene .
+#   docker build -f Dockerfile.cpu --build-arg EXTRAS=all -t videoannotator:cpu-all .
 #   docker run --rm -p 8000:8000 -v ${PWD}/data:/app/data videoannotator:cpu
 
 FROM ubuntu:24.04
@@ -34,25 +41,45 @@ ENV PATH="/root/.local/bin:${PATH}"
 
 WORKDIR /app
 
+# Extras group(s) to install, e.g. "scene" or "scene,person" or "all".
+# Empty (default) = slim image, core dependencies only, no torch.
+ARG EXTRAS=""
+
 # Copy source code explicitly (exclude models/weights)
 COPY pyproject.toml uv.lock ./
 COPY api_server.py ./
 COPY src/ ./src/
 COPY configs/ ./configs/
 COPY scripts/ ./scripts/
 
-# Install dependencies, then override with CPU-only PyTorch wheels.
-# Keep torch/torchvision/torchaudio versions compatible.
-RUN uv sync --frozen --no-editable \
+# Install dependencies (extras opt-in via $EXTRAS), then, only if any extras
+# were requested, override with CPU-only PyTorch wheels (matched versions).
+# hadolint ignore=SC2086
+RUN set -eu; \
+    if [ -n "$EXTRAS" ]; then \
+        FLAGS=""; \
+        for e in $(echo "$EXTRAS" | tr ',' ' '); do FLAGS="$FLAGS --extra $e"; done; \
+        uv sync --frozen --no-editable $FLAGS; \
+    else \
+        uv sync --frozen --no-editable; \
+    fi \
     && HADOLINT_DEST_DIR=/usr/local/bin bash scripts/install_hadolint.sh \
-    && uv pip install "torch==2.6.0+cpu" "torchvision==0.21.0+cpu" "torchaudio==2.6.0+cpu" --index-url https://download.pytorch.org/whl/cpu
+    && if [ -n "$EXTRAS" ]; then \
+        uv pip install "torch==2.6.0+cpu" "torchvision==0.21.0+cpu" "torchaudio==2.6.0+cpu" --index-url https://download.pytorch.org/whl/cpu; \
+    else \
+        echo "[BUILD] EXTRAS not set — slim image, skipping CPU torch install"; \
+    fi
 
-# Verify CPU setup (no GPU packages installed)
-RUN uv run python3 -c "\
+# Verify CPU setup (only meaningful when extras/torch were installed)
+RUN if [ -n "$EXTRAS" ]; then \
+        uv run python3 -c "\
 import torch; \
 print(f'[CPU BUILD] CUDA available: {torch.cuda.is_available()}'); \
 print(f'[CPU BUILD] PyTorch version: {torch.__version__}'); \
-print('[CPU BUILD] Production image ready - models will download on first use');"
+print('[CPU BUILD] Production image ready - models will download on first use');"; \
+    else \
+        echo "[BUILD] Slim image (no extras) — skipping Torch verification"; \
+    fi
 
 # Set environment for production
 ENV PYTHONUNBUFFERED=1

Dockerfile.dev

@@ -52,9 +52,12 @@ COPY scripts/ ./scripts/
 COPY models/ /app/models/
 COPY weights/ /app/weights/
 
-# Install dependencies (torch/torchvision/torchaudio come from the configured
-# `pytorch-cu124` uv index in pyproject.toml).
-RUN uv sync --frozen --no-editable \
+# Install dependencies. As of 004-extras-based-install, torch/pipeline deps
+# are opt-in extras rather than core dependencies, so the dev image (which
+# is meant to exercise every pipeline against the mounted model cache)
+# installs the `all` extras group explicitly. torch/torchvision/torchaudio
+# come from the configured `pytorch-cu124` uv index in pyproject.toml.
+RUN uv sync --frozen --no-editable --extra all \
     && HADOLINT_DEST_DIR=/usr/local/bin bash scripts/install_hadolint.sh \
     && uv tool install specify-cli --from git+https://github.com/github/spec-kit.git