Add video encoding support with SYCL RGB→NV12 kernel and CPU fallback - #103
Open
eromomon wants to merge 9 commits into
Open
Add video encoding support with SYCL RGB→NV12 kernel and CPU fallback#103eromomon wants to merge 9 commits into
eromomon wants to merge 9 commits into
Conversation
eromomon
requested review from
Copilot
and removed request for
dvrogozh and
luis-real
July 27, 2026 22:58
There was a problem hiding this comment.
Pull request overview
Adds an Intel XPU (VAAPI) hardware video encoding path to the torchcodec-xpu plugin, including an on-device SYCL RGB→NV12 conversion kernel with a CPU/libswscale fallback, and updates upstream TorchCodec encoder tests (via patch) to run on XPU as well as CUDA.
Changes:
- Implement XPU encoding overrides in
XpuDeviceInterface(pixel format selection, VAAPI hw_frames_ctx setup, tensor→AVFrame upload path with SYCL fast-path + CPU fallback). - Add SYCL
convertRGBToNV12kernel supporting BT.601/BT.709 and full/limited range, including tiled (Intel Tile-Y) surface handling. - Extend/parameterize upstream encoder tests (patch) to exercise XPU and avoid VAAPI flush-on-empty segfault by priming with a frame.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/torchcodec-xpu/src/torchcodec_xpu/XpuDeviceInterface.h | Declares XPU encoding overrides and helper conversion entry points. |
| packages/torchcodec-xpu/src/torchcodec_xpu/XpuDeviceInterface.cpp | Implements VAAPI frames context setup, VAAPI encoder discovery logic, and SYCL/CPU tensor→NV12 surface conversion. |
| packages/torchcodec-xpu/src/torchcodec_xpu/ColorConversionKernel.h | Exposes the new SYCL RGB→NV12 conversion API. |
| packages/torchcodec-xpu/src/torchcodec_xpu/ColorConversionKernel.cpp | Implements RGB→NV12 SYCL kernel and supporting YUV conversion matrices. |
| packages/torchcodec-xpu/patches/0002-Add-XPU-support-to-video-encoder-tests.patch | Updates upstream TorchCodec encoder tests to include XPU device coverage and VAAPI-specific priming behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
dvrogozh
requested changes
Jul 27, 2026
Contributor
|
@eromomon , please, rebase on top of latest main and resolve conflicts. |
eromomon
force-pushed
the
eromomon/encoding_no_patch
branch
from
August 5, 2026 16:01
5386ae9 to
652caef
Compare
Signed-off-by: Edgar Romo Montiel <edgar.romo.montiel@intel.com>
eromomon
force-pushed
the
eromomon/encoding_no_patch
branch
from
August 5, 2026 16:51
652caef to
2023c4f
Compare
dvrogozh
reviewed
Aug 5, 2026
Signed-off-by: Edgar Romo Montiel <edgar.romo.montiel@intel.com>
Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com>
Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com>
Contributor
|
Last concern I have on this PR is that non-SYCL path might not be working: |
dvrogozh
reviewed
Aug 6, 2026
Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com>
Signed-off-by: Edgar Romo Montiel <edgar.romo.montiel@intel.com>
…te DG2. Signed-off-by: Edgar Romo Montiel <edgar.romo.montiel@intel.com>
Signed-off-by: Edgar Romo Montiel <edgar.romo.montiel@intel.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a hardware video encoding path to the torchcodec-xpu plugin (supersedes #58)
What changes:
convert_tensor_to_av_frame_for_encoding dispatching to:
ColorConversionKernel.{h,cpp} — adds convertRGBToNV12 SYCL kernel (BT.601 / BT.709, limited + full range); get_tile_offset reused for encode.
patches/0002-Add-XPU-support-to-video-encoder-tests.patch — parametrizes upstream torchcodec encoder tests over CUDA + XPU, and primes the encoder with one frame in test_write_frames_different_devices_errors (VAAPI segfaults on empty flush; NVENC doesn't). Depends on 0001-Add-XPU-support-to-tests.patch.
Compatibility
No change to decode behavior, plugin loading, or public API.
Testing
Encoder test suite (patches 0001 + 0002) passes on Intel Arc / Battlemage.