feat(m5stack-tab5): add MIPI-CSI camera support and a live Camera tab#676
Merged
Conversation
Bring up the Tab5's on-board camera (SC202CS on MIPI-CSI) via Espressif's
esp_video (V4L2) pipeline and show the live feed, with settings, on a new
Camera tab.
BSP:
- initialize_camera(callback): reset the sensor (IO expander 0x43 P6),
esp_video_init() with the sensor SCCB sharing the internal I2C bus (at
400 kHz, so the per-frame auto-exposure sensor writes don't starve the
shared-bus touch reads), then the V4L2 flow (open /dev/video0, S_FMT RGB565,
REQBUFS/mmap, QBUF, STREAMON) and a capture task that DQBUFs frames to the
callback and requeues them. stop_camera(), camera_width/height(),
camera_reset(). Pulls in managed espressif/esp_video + esp_cam_sensor (the
ISP converts the sensor RAW8 to RGB565).
- Each frame is run through the PPA in one hardware pass: downscaled (Full /
Half / Quarter), rotated to match the display orientation (display rotation +
a fixed 90-deg mount offset), and mirrored / flipped on request.
- CameraControls {scale, hmirror, vflip} + set_camera_controls() /
camera_controls(): the GUI (any thread) posts the desired state under a mutex
and the capture task applies it, so the PPA scale, mirror/flip and the
preview-buffer realloc all happen on one thread. (Exposure / white balance /
color are left to the ISP auto pipeline; they are driven every frame and are
not usefully overridable, so no manual controls are exposed.)
espp::I2c: expose the native i2c_master_bus_handle_t so the camera SCCB can
reuse the internal bus.
Example: a Camera tab showing the live feed in a framed, draggable widget (an
lv_canvas bound to a PSRAM RGB565 buffer, with a border and rounded corners,
draggable within the tab), plus a collapsible gear-button settings overlay
(image size + mirror/flip). Enables the IDF component manager for the managed
camera components. The ISP pipeline controller (auto exposure / white balance)
is enabled so the feed is correctly white-balanced.
Two benign upstream quirks, both handled/documented (the ISP rejects the bad
value and keeps the previous one, so the image is unaffected): the stock
SC202CS auto color-correction occasionally exceeds the ISP's +/-4.0 CCM limit
(the per-frame error logging is silenced, which also fixes frame drops), and
occasional 'ISP: gamma xcoord error' from the auto-enhancement gamma (an ISR
early-log that can't be runtime-silenced; left as-is). See camera.cpp and the
example README.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
✅Static analysis result - no issues found! ✅ |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds end-to-end MIPI-CSI camera support for the M5Stack Tab5 BSP (SC202CS via esp_video/V4L2 + ISP) and extends the example with a live “Camera” tab UI, while also enabling camera-related managed components/config and exposing a native I2C bus handle for SCCB sharing.
Changes:
- Implemented camera bring-up, V4L2 streaming, and PPA-based downscale/rotate/mirror/flip processing in the Tab5 BSP.
- Added a live Camera tab to the example (LVGL canvas feed, draggable framed widget, and a settings overlay for scale/mirror/flip).
- Enabled/declared required dependencies and config (managed
esp_video/esp_cam_sensor, sdkconfig defaults) and added I2C native bus handle accessors.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| components/m5stack-tab5/src/camera.cpp | New camera pipeline implementation: esp_video init + V4L2 capture + PPA processing + control application + cleanup. |
| components/m5stack-tab5/README.md | Updates BSP documentation to reflect SC202CS camera support and usage. |
| components/m5stack-tab5/include/m5stack-tab5.hpp | Adds camera API, callback type, controls, and camera-related members. |
| components/m5stack-tab5/idf_component.yml | Declares espressif/esp_video and espressif/esp_cam_sensor dependencies for the BSP. |
| components/m5stack-tab5/example/sdkconfig.defaults | Enables esp_video MIPI-CSI device + SC202CS sensor + ISP pipeline controller. |
| components/m5stack-tab5/example/README.md | Documents the new Camera tab behavior and known upstream ISP quirks. |
| components/m5stack-tab5/example/main/m5stack_tab5_example.cpp | Initializes camera and forwards frames into the GUI. |
| components/m5stack-tab5/example/main/idf_component.yml | Enables managed-component fetching for esp_video + esp_cam_sensor in the example. |
| components/m5stack-tab5/example/main/gui.hpp | Adds camera tab initialization and camera frame/control UI plumbing declarations. |
| components/m5stack-tab5/example/main/gui.cpp | Implements Camera tab UI (canvas, drag, overlay controls) and frame copy into PSRAM buffer. |
| components/m5stack-tab5/example/CMakeLists.txt | Adjusts example build to use component manager flow and includes esp_video in components list. |
| components/m5stack-tab5/CMakeLists.txt | Adds esp_video to the BSP component requirements. |
| components/i2c/include/i2c.hpp | Exposes I2c::native_bus_handle() for sharing the internal I2C bus with esp_video SCCB. |
| components/i2c/include/i2c_master.hpp | Exposes I2cMasterBus::native_handle() used by the higher-level I2c accessor. |
Comments suppressed due to low confidence (1)
components/m5stack-tab5/src/camera.cpp:283
- apply_camera_controls() ignores allocate_camera_preview_buffer() failures and clears camera_controls_dirty_. If allocation fails (e.g., PSRAM OOM), the requested scale change is silently dropped and will never be retried.
// Scale: reallocate the preview buffer if the requested scale changed.
if (c.scale != camera_active_scale_) {
allocate_camera_preview_buffer(c.scale);
}
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Remaining review findings on top of the Copilot autofix: - initialize_camera() set camera_initialized_ before the capture task started; if start() failed the object stayed 'initialized' and could not be retried. Start first, tear down and return false on failure, mark initialized only on success. - If open(/dev/video0) (or a later V4L2 step) failed after esp_video_init() succeeded, the esp_video pipeline was left initialized, leaking it and blocking a later retry. Track esp_video init state and esp_video_deinit() it in stop_camera(); route the open-failure path through stop_camera(). - apply_camera_controls() ignored allocate_camera_preview_buffer() failure and cleared the dirty flag, silently dropping a scale change on OOM. Log the failure (the current buffer is kept, scale not marked applied). - The example's Gui allocated a persistent PSRAM canvas buffer but never freed it; deinit_ui() (called from ~Gui) only cleaned the LVGL tree. Free the buffer there so recreating the Gui does not leak PSRAM. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The camera managed components come in transitively via the m5stack-tab5 BSP, so the example does not need to re-declare them: - Remove the redundant example/main/idf_component.yml (esp_video + esp_cam_sensor were already declared by the BSP's idf_component.yml and are fetched transitively). - Drop esp_video from the example's COMPONENTS list (m5stack-tab5 REQUIRES it, and esp_video REQUIRES esp_cam_sensor, so both are already in the build closure). - Add esp_cam_sensor to the BSP's CMakeLists REQUIRES so it matches the BSP manifest and the closure gets both camera components directly from the BSP. Verified with a clean build: both managed components are still fetched and the example builds. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- The ISP log-tag silencing (esp_log_level_set to NONE) was never undone, so a failed initialize_camera() or a later stop_camera() left those tags muted for the rest of the app. Cache the prior levels (esp_log_level_get) before muting and restore them in stop_camera(). - set_camera_frame() deleted the canvas and freed the PSRAM buffer before confirming the new allocation, blanking the tab on OOM. Allocate the new buffer first; on failure keep the current canvas / preview. - initialize_camera() ignored camera_reset()'s return value, so it would continue (and let esp_video probe a possibly-held-in-reset sensor) even if the IO-expander write failed. Treat a reset failure as an init error. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ding) - The capture task treated every VIDIOC_DQBUF failure as 'no frame yet' and ignored VIDIOC_QBUF's result. Now only EAGAIN sleeps/retries; any other DQBUF error, or a QBUF failure (which would drain the queue and stall), is logged and stops the task rather than spinning silently. - The camera task read the display rotation via LVGL (lv_display_get_rotation) from its own thread, which is not thread-safe. The display flush already reads the rotation on the GUI thread, so cache it there into an atomic and have the camera task read that instead. - stop_camera() restored the muted ISP log tags after esp_video_deinit(), hiding any teardown diagnostics; restore them before deinit. - apply_camera_controls() cleared the dirty flag before applying the scale, so an OOM scale change was dropped, never retried. Apply mirror/flip (which cannot fail) first, and on a failed scale realloc re-queue the request (with a rate-limited warning) so it retries when memory frees. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…, docs) - VIDIOC_REQBUFS may allocate fewer buffers than requested; use the count it actually returns (require >=1, capped to the array) and iterate that instead of assuming CAMERA_BUFFER_COUNT. - Include errno / strerror in the camera bring-up failure logs (open, QUERYCAP, S_FMT, REQBUFS, QUERYBUF, mmap, QBUF, STREAMON) for easier field debugging. - stop_camera() now resets camera_width_/height_ and the preview dims to 0 so camera_width()/height() honor their documented '0 when not initialized' contract after a stop or failure. - Gui::deinit_ui() only nulled camera_canvas_; lv_obj_clean() deletes the whole tree, so a camera frame arriving during teardown could build objects under a freed camera_tab_ (use-after-free). Add a ui_ready_ flag (checked in set_camera_frame) and null all the tab/overlay pointers on teardown. - Enable the IDF component manager explicitly in the example CMakeLists (it is on by default, but the env could disable it), and fix the example README to point at the BSP's idf_component.yml where the camera deps are now declared. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
components/m5stack-tab5/src/camera.cpp:87
- This comment claims the shared internal I2C bus runs other devices at 1 MHz, but the Tab5 BSP configures its internal_i2c_ clock at 100 kHz (and device configs use that value). That mismatch is confusing when debugging bus contention issues.
// bus-hold time. (The bus itself runs the other devices at 1 MHz; the I2C
// master reprograms the clock per transaction.)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds support for the M5Stack Tab5's on-board MIPI-CSI camera (SC202CS) and a
live Camera tab in the example, with a framed/draggable feed and a small
settings overlay.
BSP (
espp::M5StackTab5):initialize_camera(callback)brings up the ESP32-P4 camera pipeline (CSIreceiver → ISP → sensor) via Espressif's
esp_video(V4L2) framework: resetsthe sensor (IO expander 0x43 P6),
esp_video_init()with the sensor SCCBsharing the internal I2C bus, then the V4L2 capture flow (
/dev/video0,S_FMTRGB565,REQBUFS/mmap,QBUF,STREAMON) and a capture task thatDQBUFs each frame to the callback and requeues it. Plusstop_camera(),camera_width/height(),camera_reset().(Full / Half / Quarter), rotate to match the current display orientation
(display rotation + a fixed 90° sensor-mount offset), and mirror / flip on
request.
CameraControls {scale, hmirror, vflip}+set_camera_controls()/camera_controls(). The GUI (any thread) posts the desired state under amutex; the capture task applies it, so the PPA scale, mirror/flip and the
preview-buffer reallocation all happen on one thread.
espp::I2cgains anative_bus_handle()accessor so the camera SCCB canreuse the internal I2C bus.
Example:
lv_canvasbound to a PSRAM RGB565 buffer, with a border + rounded corners;it can be dragged around the tab, and pans when larger than the tab), plus a
collapsible gear-button settings overlay (image size + mirror/flip).
espressif/esp_video+espressif/esp_cam_sensorcomponents (the ISP converts the sensor RAW8 → RGB565) and enables the IDF
component manager. The ISP pipeline controller (auto exposure / white balance)
is enabled so the feed is correctly white-balanced.
Exposure / white balance / color are left to the ISP auto pipeline (it drives
them every frame), so no manual image-quality controls are exposed.
Motivation and Context
The Tab5 BSP already anticipated the camera (pin defines, a "not yet
implemented" note) but had no camera code. This implements it end-to-end so the
BSP can stream frames and the example can display them, matching the other Tab5
peripherals.
Notes on things worked through on hardware, handled and documented in-code:
ISP's auto-exposure writes the sensor over SCCB every frame; the SCCB is
clocked at 400 kHz so those writes don't starve the (short, fail-fast)
touch reads.
rejects the bad value and keeps the previous one, so the image is unaffected):
the auto color-correction occasionally exceeds the ISP's
±4.0CCM limit —the per-frame error logging is silenced (which also fixes the frame drops it
caused); and an occasional
ISP: gamma xcoord errorfrom the auto-enhancementgamma, which is an ISR early-log that can't be runtime-silenced and is left
as-is.
How has this been tested?
m5stack-tab5example with ESP-IDF for the ESP32-P4 — clean build,no warnings.
mirror / flip work (via the PPA).
Screenshots (if appropriate, e.g. schematic, board, console logs, lab pictures):
Types of changes
Checklist:
Software
.github/workflows/build.ymlfile to add my new test to the automated cloud build