Additional experiments for the paper: Automatic self-contained calibration of an industrial dual-arm robot with cameras using self-contact, planar constraints, and self-observation http://karlastepanova.cz/publications/self-calibration-industry/)
Estimating uncertainty from repeatability
First, we studied whether the repeatability is sufficient to approximate the uncertainty associated with the different calibration approaches. Prior to the analysis, we have updated the error distribution pertaining to this (new Fig. 16), taking into account that self-contact measurements involved both large movements of the robot as well as local movements similar to those used in the contact with planes.
We fitted the Euclidean distances from the repeatability measurement by a Gaussian distribution and evaluated the corresponding standard deviation (S.D.) as 1.07 x10-5m for planar contact and 4.04 x10-5m for self-touch, respectively. Then, we performed calibration and used the adjusted model (f as defined in Eq. 6): f = gT Σ-1 g, where g=c–q (Eq. 7) for self-touch, and g=c–r (Eq. 10) for planar contact; Σ is a diagonal matrix with S.D. for corresponding approaches on the diagonal:
The resulting error evaluated on the independent Leica dataset is the following:
Original calibration without weighting (using forig): 0.0030
Calibration where uncertainty is used to weigh individual approaches (using f): 0.0052
Grid search to find the best scaling factor for our real setup
Second, we performed a grid search over different ratio of scale factors (k in Eq. 15; here only η reflecting uncertainty is considered) for planar constraints and self-touch (0.01 to 100) in our objective function. Here a ratio greater than 1 on the x-axis gives more weight to plane constraints measurements, emulating that they would have a lower uncertainty. The resulting mean RMSE over 10 runs on the independent laser tracker dataset can be seen below. When incorporating a planes/self-touch scale factors ratio of 0.79, we achieve a better RMSE on the independent dataset. This result above is not in line with the scale factor that was predicted from the ratio of uncertainty obtained from the repeatability above (planes/self-touch = sqrt(4.04e-5 / 1.07e-5 ) ~ 2), for which the RMSE would be significantly higher. In fact, the grid search revealed that the self-contact data are more reliable.
These results suggest that the uncertainty cannot be determined from our repeatability measurements alone and that self-touch measurements have effectively a lower uncertainty than contact with planes. There might be several factors, which contribute to uncertainty of individual approaches and which are difficult to evaluate (different responses of force sensors for variable orientations of end-effectors during contact detection, not perfectly flat planes, etc.).
The uncertainty may also be different in every measurement point in the workspace.
Simulation of individual measurements’ uncertainty
Third, to get additional insight if perfect information was available, we prepared a simulation of our setup. In this simulated environment we created:
- Independent dataset which we used for validation.
- A set of emulated measurements for self-touch and planar touch experiment, with a known uncertainty of the measurement for each point. We added noise to the measurements of end effector position as follows: For the self-touch approach, the errors were sampled from a normal distribution; eventually, the mean of the 3D errors over all measurements was 0.0001, with S.D. of 0.0005. More noise was added to the planar constraints measurements: mean error 0.0001, S.D. 0.008.
Using the set of measurements with the given uncertainty, we evaluated calibration of offset parameters by combination of self-touch and planar contact where:
- Uncertainty is not included to the evaluation (all-standard).
- A same value of uncertainty is considered for each of the combined approaches (this value is the same for all measurements, i.e. 0.0005 for self-touch and 0.008 for planes) (all-std).
- Known uncertainty for each measurement point is included: gT Σ-1 g, where Σ is a diagonal matrix with uncertainty of each measurement on the diagonal.
The results of the calibration (for 10 runs) evaluated on the independent simulated dataset can be seen in the following figure. Incorporating the uncertainty improves the RMSE evaluated on the independent dataset, but increases the standard deviation of the results. Including uncertainty for each measurement separately is further slightly improving the RMSE, while further increasing the distribution of the RMSE errors.
