Perception of Rock Bolt – Detection & Pose Estimation

Organization: ABB Robotics, Umeå University, Sweden
Timeline: Mar 2025 – Jun 2025

Overview

Developed a perception-to-control pipeline for ABB’s industrial mining robots to identify, locate, and estimate the 6-DoF pose of rock bolts on conveyor systems.
The project aimed to enable autonomous grasp planning under dusty and uneven mining environments using RGB-D data.

My Role

  • Led the vision and integration pipeline, combining 2D detection and 3D pose estimation.

  • Designed and trained Mask R-CNN for segmentation on 1,000 + RGB-D images.

  • Integrated PointNet++ for 3D point-cloud regression and implemented the full system in ROS2.

  • Validated grasp planning in RViz simulation with automated control feedback.

Results & Impact

  • 92 % detection accuracy and 0.028 average pose error on unseen RGB-D datasets.

  • Achieved 0.401 majority-voting robustness score across noisy test conditions.

  • Demonstrated closed-loop grasping validation for potential industrial automation deployment at ABB.

  • Strengthened collaboration between Umeå University and ABB for future perception-driven automation studies.

Multi-Stage Perception-to-control Pipeline

Pose-Estimate Workflow

(a) segmented Mask by Mask RCNN; (b) point cloud generation; (c) pose-estimation with colour code (white- centroid, green- bolt’s end points, yellow- camera centre)

Full demonstration of End-to-end perception workflow showing detection of single and multiple bolts, partial occlusions, and rock coverage.

Next

Autonomous EV Charging Arm