Grasping Anygrasp

This meta package brings together the grasping stack, AnyGrasp, anygrasp_ros, and the gripper stack into one integration flow.

It does the following tasks:

1. Call the configured AnyGrasp service for grasp poses.
2. Forward the selected pose to grasping stack's grasping_control/motion_execution_node through grasping_msgs/action/MoveToPose.
3. Trigger gripper open/close actions and optionally request the post-grasp move.

Read here for known issues and how we fixed them

System Architecture

Camera Driver

Camera setup, configuration, and customization are documented in the linked file. The current supported device is:

• Realsense Camera

RGBD to Pointcloud (via external devcontainer)

As shown in the system architecture diagram, the RGB and depth images are combined into a colored point cloud before grasp detection. This is done by anygrasp_ros/rgbd_to_pointcloud_node, which subscribes to the RGB and depth image topics, synchronizes them, and publishes the resulting colored point cloud for grasp pose detection. Check the Anygrasp Node section for instructions on connecting to the external devcontainer.

• RGBD to Pointcloud Node

AnyGrasp Node (via external devcontainer)

This grasping stack uses AnyGrasp for grasp pose detection. This container does not include AnyGrasp itself, and expects a companion devcontainer to provide AnyGrasp. The AnyGrasp node subscribes to the colored point cloud and publishes the detected grasp poses. The anygrasp_grasping node connects to the AnyGrasp node and requests a grasp pose when triggered.

• Detection Node
• Tracking Node

To connect with anygrasp_ros external devcontainer, follow the instructions on CollaborativeRoboticsLab/anygrasp_ros. Use the host.xml file since this container is configured for network=host mode.

Gripper Controller

This grasping stack supports multiple gripper types. The gripper controller focuses on custom-built grippers using servo configurations documented in CollaborativeRoboticsLab/grippers. Setup, configuration, and customization details are in the linked documents.

• Dynamixel Grippers
• Feetech Grippers

Manipulator

This grasping stack supports multiple manipulators. The moveit stack uses custom config that contains standard manipulators attached with custom-built grippers and are documented in CollaborativeRoboticsLab/grasping. Following are the links to the setup, configuration, and customization details for each manipulator.

UR10 Manipulator

• UR10 and Devcontainer connection
• UR10 calibration
• UR10 tf frames for gripper compatibility
• UR10 startup

TM12S Manipulator

• TM12S and Devcontainer connection
• TM12S tf frames for gripper compatibility

LD250 &TM12x Mobile Manipulator

• LD250 & TM12X and Devcontainer connection
• LD250 & TM12X tf frames for gripper compatibility

Common

• Attaching new gripper and components
• Moveit Servo and Keyboard Teleop

Manipulator Control and Workspace Creation

This component transforms grasp poses, applies workspace obstacles to MoveIt, visualizes the calibrated workspace area, and rejects poses outside that area. This is also available in CollaborativeRoboticsLab/grasping. Read the linked documents for setup, configuration, and customization details.

• Workspace Creation
• Manipulator Control
• Control stack overview

AnyGrasp Grasping Pipeline

This package implements the grasping pipeline and is the main component that orchestrates the AnyGrasp based grasping process: it requests a grasp pose from AnyGrasp, calls the arm-control action, closes the gripper, and optionally runs a post-grasp move.

• Grasping pipeline

Quick Commands

Starting the camera

Use the following command to start the realsense D435 camera.

source install/setup.bash
ros2 launch anygrasp_realsense d435.launch.py

Starting the AnyGrasp Detection System

Use the following command to start the AnyGrasp system.

source install/setup.bash
ros2 launch anygrasp_ros detection.launch.py

Start the UR10 Manipulator and Gripper with MoveIt

For UR10 with soft two-finger gripper

source install/setup.bash
ros2 launch grasping_control ur10_soft_two_fingers.launch.py

Then Start the AnyGrasp Bridge in server mode

source install/setup.bash
ros2 launch anygrasp_grasping grasping_bridge.launch.py

and Finally Trigger One Grasp Cycle (this can be done multiple times to test the grasping pipeline)

source install/setup.bash
ros2 service call /anygrasp_grasping_node/run_grasp std_srvs/srv/Trigger {}

Or Start the AnyGrasp Bridge in single trigger mode

source install/setup.bash
ros2 launch anygrasp_grasping grasping_bridge.launch.py server_mode:=false

Typical Run

1. Start MoveIt and the robot driver.
2. Calibrate or update workspace.yaml with workspace_creation if needed.
3. Start the gripper action server.
4. Start AnyGrasp.
5. Launch a grasping_control robot bringup such as ur10.launch.py or ur10_soft_two_fingers.launch.py.
6. Launch grasping_bridge.launch.py.
7. Trigger /anygrasp_grasping_node/run_grasp, or run the node with server_mode:=false.