DO NOT MERGE — Diagnose Plan Path Along Surface CI hang (moveit_pro#20427)

[griswaldbrooks]

[written by AI]

Diagnostic branch — do not merge. Instrumentation to identify which joint hangs the Plan Path Along Surface execute goal in hangar_sim CI (PickNikRobotics/moveit_pro#20427).

Background

Analysis on the moveit_pro issue established the failure is a silent controller hang, not slowness: constraints.goal_time: 0.0 makes upstream JTC's goal-tolerance abort branch unreachable, so a goal outside tolerance after the last trajectory point neither succeeds nor aborts — and the per-joint tolerance error only prints on the abort path. Incidence over 21 mined runs: ≥1 failure in 70% of runs, strictly bimodal (~55 s pass or exactly the 90 s fixture cap), only this objective ever fails.

Instruments

1. goal_time: 20.0 in picknik_ur.ros2_control.yaml — converts the silent hang into a GOAL_TOLERANCE_VIOLATED abort at trajectory-end + 20 s (~T+75 s, inside the 90 s fixture cap) whose log line names the offending joint and error magnitude. This is also the candidate permanent regression guard.
2. Per-test rosbag capture (conftest.py) — records /joint_states, /tf(_static), /clock, JTC controller_state, FollowJointTrajectory action status/feedback, and base odometry as a subprocess (same fork/DDS isolation rule as capture_rosout). Bags for failing tests and for the two surface objectives (pass or fail, for healthy-vs-hung diffing) are kept under build/hangar_sim/test_results/bags/, which the reusable workflow already uploads in the test-results-* artifact; all other bags are deleted on pass. Playback: ros2 bag play <dir> --clock + RViz with use_sim_time.
3. CI matrix trimmed to hangar_sim on this branch only, so repeated evidence-gathering runs don't burn lab_sim runners.

Method

Sequential workflow_dispatch runs (-f image_tag=main) on this branch until several failure samples are captured (~70% per-run incidence → a handful of runs suffices). Each failure yields the JTC abort line in captured stdout plus a scrubbable bag artifact. Findings and the resulting fix PR go to moveit_pro#20427.

Release notes

None

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 63fd7647-0fc9-4e75-a57e-1c831df5ca8d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

⚠️ This PR modifies 1 file(s) that also exist in PickNikRobotics/moveit_pro_empty_ws.

Consider whether the change should land upstream in moveit_pro_empty_ws first so downstream forks pick it up on the next sync.

Overlapping files

• .github/workflows/ci.yaml

[github-actions]

MoveIt Pro Example WS - Objectives Integration Test Report

• lab_sim
  • ❌ jazzy: no report produced — see run logs
• hangar_sim
  • ❌ jazzy: view HTML report — 1 failed, 16 passed

[griswaldbrooks]

[written by AI]

Diagnostics served their purpose — root cause identified and fixed by #776. The instrumentation recipe (goal_time abort + per-test rosbag capture riding the test-results artifact) is documented in moveit_pro#20427 for future use.