Pose-free object classification from surface contact features in sequences of Robotic grasps

Introduction

The paper discusses the development of two different approaches, PN (Point and Normal) based and P (Point) based, for object identification using a robotic hand with proprioceptive and/or tactile sensing capabilities. The PN method relies on measuring 3D positions and surface normals at the finger contact points, while the P method only uses the positions of the contacts.

The researchers conducted experiments in a robotic simulator with the Barrett Hand, investigating an active learning strategy for exploration around the object to find the grasp with the highest information gain. This active exploration approach was compared to passive/random exploration under the same conditions.

The proposed methods were validated in the GraspIt! simulator, and their robustness was assessed by adding random noise to the sensor values and checking the corresponding performance. The methods maintain a likelihood score for each object at each grasp, allowing a decision on the object's identity to be made based on a certainty threshold.

The evaluation criteria included efficiency (the number of grasps required to reach a certain level of certainty) and accuracy (the fraction of correct decisions made by the system).

Related Work

The text discusses various methods for object identification using tactile sensing with robotic grippers and hands. Some key points:

• Previous methods require full 3D reconstruction of objects, which can be computationally expensive. Humans can recognize objects with just a few grasps without full reconstruction.

• The bag-of-features approach uses a two-fingered gripper to grasp and examine an object's height profile with tactile sensors on each finger. Neural networks can also analyze the sensor shapes for object identification.

• The GraspIt! simulator can compute stable grasps for objects and provide contact poses. It has been used with real robotic hands like the Barrett Hand.

• One approach uses motor values of a Barrett Hand's fingers and orientation to create tables and apply Bayesian inference for object identification, without using tactile sensor values.

• This approach implements active learning to find the grasp orientation with the most information gain, but requires the object to be static.

• Vision-based methods like using point clouds from RGBD cameras and point pair features can also be adapted for robotic grasp object identification.

• A limitation of existing methods is that they are not pose-invariant and require the object to remain static.

Approach

The paper describes a haptic object identification approach based on using contact point information from a robotic hand's fingers. The core method calculates Point Pair Features (PPFs) from the relative positions and orientations of pairs of contact points. These PPFs are used as keys in a hash table that stores object models. There are two variants:

The PN method uses both the positions and normals at the contact points to calculate PPFs, making it robust to the object's pose.

The P method only uses the contact point positions, allowing identification with robots that cannot measure contact normals, but at the cost of some pose invariance.

During data collection, multiple grasps are performed on each object to account for sensor noise. The testing phase accumulates votes for objects based on how well the observed contact point data matches the hash tables.

The paper explores two exploration strategies - passive random exploration and active exploration that selects the next grasp to maximize the differentiation between the top object candidates. Different probability thresholds are evaluated as stopping criteria for identification.

Overall, it presents a contact geometry based approach for pose-invariant haptic object recognition, with options to trade off between accuracy and sensor requirements through the PN and P variants. The key technical details covered are the PPF feature calculation, voting scheme, exploration strategies, and probabilistic inference.

V Experiments

The paper discusses using the GraspIt! simulator to generate data for object classification. The simulator allows extracting contact pose information as a robotic hand rotates 360 degrees around objects. Five objects from the YCB database were chosen: a tuna can, mug, bowl, baseball, and foam brick.

To create training data, Gaussian noise was added to the simulated contact poses 50 times for each grasp to simulate sensor noise. This generated 50 sample grasps which were used as keys to populate hash tables for two different classification methods.

For testing, noise was again added to the simulated grasps to generate 50 sample keys. The existence of these keys was checked in each object's hash table to calculate the probability that grasp belongs to that object using an equation provided.

The section on processing grasp values does not provide any specific details about the classification methods themselves, only the data generation process.

Results

The paper presents results comparing two methods (PN and P) for object classification through grasping, using both passive and active exploration approaches. Key findings include:

Passive Learning:

• The PN method generally required fewer grasps than the P method for object classification, except for the foam brick object.
• For a 99% confidence threshold, the PN method allowed single-grasp identification for the bowl object and required 2-3 grasps for other objects, outperforming the P method.
• The baseball was the most difficult object to identify, requiring up to 327 grasps for the P method.
• Perception errors were generally lower for the PN method compared to P, except at low confidence thresholds where the P method performed slightly better for some objects.

Active Learning:

• Both methods required far fewer grasps compared to passive learning.
• The PN method outperformed P for most objects, requiring fewer grasps on average. The bowl showed similar results for both methods.
• The active P method struggled with classifying the mug object, exhibiting around 15% error even at 99% confidence.
• Perception errors were substantially lower with active learning compared to passive for both methods.

Overall:

• Active exploration with the PN method achieved the best performance, requiring the lowest median number of grasps (5) and exhibiting low perception errors.
• Passive exploration with the P method performed the worst, demanding the highest median grasps (16) and highest perception errors.

The results demonstrate the advantages of incorporating active exploration and using the PN method for efficient and accurate object classification through grasping.

Conclusions

The paper presents two methods for classifying objects from a sequence of grasps without requiring knowledge of the relative pose between the hand and the object. The first method (PN) uses the position and orientation of the contacts between the fingers and the object relative to the hand's reference frame. The second method (P) uses only the position of the contacts, catering to robots with less haptic sensing abilities.

Experiments demonstrated that both methods effectively recognize objects in a limited set with good accuracy using a small number of grasps. The PN method, utilizing richer features, achieves better performance, requiring approximately half the number of grasps compared to the P method on average.

In cases where the relative pose between the hand and object can be measured (e.g., with external sensors), an active exploration method is proposed to further reduce the number of required grasps. The PN method still outperforms the P method, but the difference is smaller, suggesting that simpler contact sensing can be sufficient when the object's pose relative to the hand is known.

Future work will focus on a more comprehensive characterization of the methods, involving more objects, increased hand movement degrees of freedom, and more natural environments, both in simulation and with a real robot hand.

The paper introduces a pose-free method for haptic recognition, improving the efficiency of haptic sensing methods for robots in recognizing objects under natural conditions.