[output] Add optional TwelveLabs scene labelling

[mohit-twelvelabs]

Hi! I'm Mohit, I work at TwelveLabs (@mohit-twelvelabs).

What this adds

An opt-in scenedetect.output.label_scenes helper that attaches a short natural-language description to each detected scene using the TwelveLabs Pegasus video-understanding model.

PySceneDetect's detectors find where the cuts are (pixel/histogram/hash based); this complements that by answering what is in each scene. Each detected scene's start/end timecode is forwarded to Pegasus via its start_time/end_time parameters, so each description covers only that portion of the video and the labels line up 1:1 with the scene list:

from scenedetect import detect, ContentDetector
from scenedetect.output import label_scenes

scenes = detect("my_video.mp4", ContentDetector())
for label in label_scenes(scenes, video_url="https://example.com/my_video.mp4"):
    print(label.index, label.label)

Why it helps

A common ask after shot detection is a quick semantic summary per scene (for editing, search, dataset labelling, highlight reels). This wires that in without the user having to build their own pipeline, while keeping PySceneDetect's pixel-based detection as the source of truth for cut boundaries.

Opt-in / non-breaking

• New code only; touches no detection paths and changes no defaults.
• Gated behind a new optional extra: pip install scenedetect[twelvelabs]. The twelvelabs package is not a hard dependency; if it's missing, label_scenes raises a friendly ImportError with install instructions.
• Exported alongside the other scenedetect.output helpers, following the existing re-export convention.

How it was tested

• No-network unit tests (tests/test_labels.py) using a fake client assert the per-scene timecode wiring and source-argument validation; these run in CI without a key.
• An opt-in integration test is skipped unless TWELVELABS_API_KEY (and a TWELVELABS_TEST_VIDEO_URL) are set.
• Verified against the live API: a Marengo text embedding returns the expected 512-dim vector, and the Pegasus analyze request wiring (model pegasus1.5, video source, prompt, per-scene start_time/end_time, max_tokens) passes all server-side parameter validation. Full per-scene generation against a hosted sample is pending a clean public sample URL.
• ruff check and ruff format --check pass on all changed files; pytest tests/test_labels.py passes (2 passed, 1 skipped).

You can grab a free API key at https://twelvelabs.io — there's a generous free tier.

[frueter]

that is exciting! the pip install isn't working for me though. user error?

(.venv) (base) frank@MacBookPro cutdetector_test % pip install scenedetect[twelvelabs]
zsh: no matches found: scenedetect[twelvelabs]

[mohit-twelvelabs]

Not user error — that's zsh treating the square brackets as a glob pattern. Just quote the extra:

pip install 'scenedetect[twelvelabs]'

One caveat: since this is still an open PR (not yet on PyPI), pip won't find the twelvelabs extra from the released package. To try it out, install straight from the branch:

pip install 'scenedetect[twelvelabs] @ git+https://github.com/mohit-twelvelabs/PySceneDetect.git@feat/twelvelabs-integration'

or clone the branch and pip install '.[twelvelabs]'. Let me know if that gets you unblocked!

— Mohit (@mohit-twelvelabs, TwelveLabs)