Time Synchronization in modular collaborative robots

Introducing M-cobots, modular collaborative robots

The content of this article comes from “Time Synchronization in modular collaborative robots” available at  . Peer written with Carlos San Vicente Gutiérrez, Lander Usategui San Juan, , Iñigo Muguruza Goenaga, Laura Alzola Kirschgens, and yours truly.

A new generation of robot systems which are modular, flexible and safe for human-robot interaction are needed. Existing cobots seem to meet only the later and require a modular approach to improve their reconfigurability and interoperability. We propose a new sub-class of cobots named M-cobots which tackle these problems. In particular, we discuss the relevance of synchronization for these systems, analyze it and demonstrate how with a properly configured M-cobot, we are able to obtain a) distributed sub-microsecond clock synchronization accuracy among modules, b) timestamping accuracy of ROS 2.0 messages under 100 microseconds and c) millisecond-level end-to-end communication latencies, even when disturbed with networking overloads of up to 90 % of the network capacity.

Introduction

With the growing popularity of robots, we are starting to observe how these machines are entering new areas. The degree of automation in a variety of applications is rapidly increasing. The food industry is one of the latest hypes in robotics and current technology is showing limitations to cope with the demands that arise when preparing a salad. Moving beyond new robot applications and looking back at traditional industrial automation, technical hurdles are starting to be identified as critical for further growth. Geenen [1] indicates that previous work conducted at Fraunhofer [2] showed that full automation of production cycles is often inefficient due to the challenges presented when mixing high numbers of product varieties combined in small batch quantities. He remarks that achieving efficiency in this landscape requires the development of flexible and adaptable production systems that can be applied to a variety of tasks [3] and proposes a new generation of robot systems which are modular, flexible and safe for Human-Robot-Collaboration (HRC).

Collaborative robots (commonly referred as cobots), combine the benefits of human intelligence and skills with the advantage of sophisticated robotic technical systems. This generation of robots focus on establishing a joint workspace between humans and machines. Cobots have demonstrated their advantages in many areas. For example, Gambao et al. highlight this aspect for material handling tasks [4] however, according to the authors, in order to improve the reconfigurability and flexibility of existing robots, the modular approach is the best solution. Current available-in-the-market collaborative robots lack of modularity and have followed a similar approach than that of traditional robots enforcing vendor lock-in through a variety of techniques. Beyond small agrupations of manufacturers around an individual cobot vendor, there is no real attempt to ease the integration process with peripherals such as end-effectors or sensors in a vendor-agnostic manner.

Inspired by prior art [4], we present a new subclass of collaborative robots that include native modularity and reconfigurability capabilities to tackle new market demands. We call this new class M-cobots; modular collaborative robots. M-cobots aim to tackle several of the relevant problems that exist in current cobot solutions such as the lack of interoperability among vendors, the lack of flexibility, extensibility or even synchronization with third party modules. In this particular paper, we discuss the problem of synchronization, relevant for time-critical robotic tasks and with direct implications on communication latencies.

Traditional approaches for synchronization involve the use of certain vendor-specific fieldbuses and a variety of components using different synchronization primitives. This, typically leads to complex and hard to maintain hybrid synchronization architectures. For M-cobots, we propose an integrated synchronization approach based on the IEEE 1588 Precision Time Protocol (PTP) standard. We demonstrate how with this approach, we manage to obtain robust millisecond-level end-to-end communication latencies. We show our setup and provide experimental results about the timestamping accuracy obtained with ROS 2.0. In particular, we show how our distributed and modular setup is able to deliver sub-microsecond clock synchronization accuracy. We finalize by challenging our setup with a variety of actions and showing how appropriate configurations can mitigate the impact to the overall communication latency.

Background

A. Why do we need synchronization in modular robotics?

There is abundant literature on the importance of synchronization in robotics applications and how different methods are applied in order to achieve better performance. A modular robotic system is, in essence, a distributed robotic system which requires synchronization between components in order to coordinate operations for the sensors and the actuators. A common time reference is required for a variety of reasons. For example, often, data logging requires precise timestamping. In high speed and accurate motion control systems, synchronization between different actuators is critical. In the case of multiple sensor data fusion, inaccurate synchronization usually leads to inaccurate data inference. Furthermore and as it will be seen through our experiments, synchronization is also critical for decreasing overall response times in distributed systems.

A common case in robotics where synchronization is needed are robot-laser scanner systems. Voges et al. [5] explain the significance of a proper synchronization between a laser scanner and an actuator within a motor-actuated 3D laser scanner. In this case, the authors propose a passive method to estimate the sensor timestamp offset. In [6], a similar method is applied for a multi-sensor system consisting in an actuated laser scanner, its motor and a camera for a Simultaneous Localization and Mapping (SLAM) application. As the author explains, if no synchronization is present, the offset for the corresponding encoder values for each set of scan points can lead to a large distortion in the resulting point cloud that is constructed by the SLAM technique.

In [7], it is explained how the synchronization between a laser scanner and a robot is critical for laser welding. An incorrect synchronization would lead to geometric error between the desired pattern and the welded pattern. In this case, the proposed method consisted in characterizing experimentally and compensating the delay between measurements and robot operations. Graaf et al. [8] tackles a similar problem for real-time seam-tracking in robotic laser welding using a UDP communication to synchronize the laser data acquisition with the robot pose.

Zaman et al. [9] introduces a method for time synchronization of odometry and vision data in a mobile robot. The author explains how in systems without synchronization techniques, the combination of image data with odometry usually requires speed limitations in order to reduce measurements errors. The proposed method use motion events, detectable by the odometry and image data.

In [10], a humanoid robot is built as a distributed system using Ethernet and a communication middleware (Corba) for the communications. This work addresses the importance of synchronization and uses the IEEE 1588 Precision Time Protocol (PTP) to synchronize the clock of all the robot components in the network. Particularly, within the example discussed, the nodes need to be synchronized in order to get data from the sensors and rotate the motors at the right time.

In [11] different synchronization strategies for multi-sensor data fusion for advanced driver assistant systems (ADAS) applications are discussed. The aim of this work is to analyze in a formal way the impact of the sensor fusion process in the system latency. One of the final conclusions is that having the sensors synchronized helps to reduce the fusion process latency greatly in scenarios with very different sensor measuring frequencies.

In [12], the use of real-time sensor data for robot control is analyzed. In this work, the author breaks down the delays involved in a sensor based control with a Universal Robot (UR) arm controlled using the Robot Operating System (ROS) [13]. For the characterization, the worst case synchronization delays are taken into account. These delays are as high as the period of the control cycle. In order to compensate for sampling effects due to the lack of synchronization between the sensors and the control cycle, an oversampling strategy is adopted. With this strategy the sensors values are updated with a sample frequency higher than that of the robot control cycle.

As we have seen, synchronization is required in a wide range of robotic applications. Depending on the use case, it is common to find different solutions; some of them are based on clock offset estimation and compensation, calibration, trigger signals and time synchronization standards, among others.

B. Synchronization in ROS

In ROS based systems, the common approach for synchronization is to use the system clock as a time source and to synchronize the clocks of different components with network synchronization tools such as the Network Time Protocol (NTP), as explained in ROS clock documentation [14]. In addition, ROS provides several topic synchronization methods as part of the Transform Library [15]. tf or the newer tf2 libraries have methods such as TimeSynchronizer and ApproximateTimeSynchronizer, which allows to synchronize incoming messages based on the timestamps in the message headers. ROS 2.0 Clock and Transform Library implementations are currently under development [16] but it is expected to work in a similar way as in ROS.

One of the problems when building a ROS based system is that synchronizing the system clocks is not always feasible and in other cases the method used is not accurate enough. For this reason, it is common to find different approaches to achieve synchronization between components. Olson et al. [17] give an overview of some of these methods and propose a passive synchronization algorithm. This method aims to reduce the synchronization error in robot sensors when the sensors do not provide a synchronization method. In this work, one of the main problems in robotic system construction is addressed; heterogeneous sensors which lack of synchronization standards. For example, in many cases, synchronization methods such as NTP are not applicable because sensors are connected trough serial interfaces. This problem is also addressed in the 2017 Roscon talk Building a Computer Vision Research Vehicle with ROS [18] where Fregin explained the method used to solve the synchronization problem between four cameras for computer vision system for an ADAS (Advanced driver-assistance systems) system. Because the cameras lack of a common reference time, they needed to implement a hybrid solution based on the PTP and a trigger signal approach. This exemplifies the complexity of integrating non-synchronized components in a ROS distributed system.

As discussed, synchronization can be time consuming and the implemented solutions tend to be application specific. The key problem is that many robot components do not provide standard synchronization methods or they do not provide any synchronization method at all. To the best of our knowledge, the simplest approach for roboticists would be to use synchronous ROS native components which can be easily synchronized using enabled methods such as NTP or PTP. This way, ROS applications will simply rely on a correct system clock synchronization regardless of the method used to synchronize the clocks.

C. Precision Time Protocol (PTP)

NTP is a well established clock synchronization method which allows synchronization over unreliable networks such as the Internet. On the other hand, when higher accuracy is needed, clock synchronization protocols such as Global Positioning System (GPS) are typically used. However, to overcome some limitations of these protocols the IEEE 1588 Precision Time Protocol (PTP) was created. PTP provides higher acuracy than NTP because it makes use of hardware timestamping, which provides sub-microsecond accuracy [19]. Also, unlike GPS, PTP does not require to have access to satellite signals, which makes it suitable for indoors applications and at lower cost.

In order to simplify the construction of synchronous robots powered by ROS, we make use of the H-ROS [20] software and hardware infrastructure and include the IEEE 1588 Precision Time Protocol (PTP) at each module. In addition, the H-ROS modules make use of the HRIM [21] model which allows modules to have consistent messages and interoperate seamlessly. The PTP standard included on each module allows sub-microsecond synchronization. This provides a common global clock for all the H-ROS modules allowing, for example, to timestamp all the messages in a coherent time reference.

D. Time synchronization and TSN

As explained in previous work [22], the Time Sensitive Networking (TSN) standards aim to make Ethernet more deterministic. As part of these standards, the IEEE 802.1 TSN task group has defined a formal profile of IEEE 1588, in order to synchronize time in standard Ethernet. The IEEE 802.1AS-2011 [23] standard, also known as gPTP (Generalized Precision Time Protocol), defines a specific profile of IEEE 1588–2008 with additional specifications which improve timing accuracy and reliability. Following the completion of 802.1AS-2011, the IEEE 802.1 TSN task group began work on a revision of gPTP in order to expand its use for industrial time-sensitive applications. For example 802.1AS-REV will support multiple simultaneous master clocks to allow for quick recovery of time synchronization in case of failure. The standard also aims to provide plug and play synchronizations, an interesting feature for modular robotic components.

Time synchronization provided by the 802.1AS standard is also the base for other standards such as the time aware traffic scheduling, defined in 802.1Qbv [24]. This standard enables time-triggered communications at the egress ports of Ethernet devices by defining a periodic schedule.

Experimental results, conclusion and future work

Our group researched the problem of time synchronization for modular robots. We describe how the lack of clock synchronization methods in robotic components is a critical problem that hampers the system integration effort and can affect the overall robot performance. In the original paper, we present the complete results and accordingly, we claim that the best approach for modular robotics is to build native synchronous robot modules which rely on standardized network-based synchronization protocols. We do so building on top of previous work [22], [26], [27] integrated in the H-ROS[25] infrastructure and prove how ROS-native robot components can turn into robot modules that provide synchronous responses and well defined communication latencies. In particular, we demonstrate how ROS 2.0-enabled hardware is able to obtain

a) sub-microsecond clock synchronization accuracy,

b) ROS 2.0 timespamping accuracies below 100 microseconds and

c) end-to-end communication latencies between 1–2 milliseconds.

Overall, the results presented show that we achieved accurate modular and distributed synchronization using the setup described. We are able to timestamp ROS 2.0 messages with an accuracy under 100 microseconds. This is specially interesting for sensor data fusion based on timestamping, such as the ROS synchronization filters. We also showed how by using PTP, we are able to synchronize end-to-end ROS 2.0 communications with high accuracy, mainly limited by communications delays. As we showed in [27], the communications overhead is mainly caused by the reception path of the Linux Network Stack and the lack of optimization of the ROS-DDS layers. Similar experiments with pub/sub communications for OPC UA over TSN are being conducted [28] [29], which show low latencies can be achieved when optimizations for time-sensitive traffic are made. In addition, we demonstrated how delays in the modules’ Ethernet links can increase the synchronization jitter for ROS 2.0 messages in congested networks and showed that applying switched Ethernet QoS priorization solves the problem. Finally, we show that TSN Time Aware Shaper can be used in order to control the message transmission and achieve accurate reception times when ROS 2.0 nodes publish.

Read the full article at .

References

[1] “Designing a new generation of modular, flexible and collaborative robots ,” designing-a-new-generation-of-modular-flexible-and-collaborative-robots/, accessed: 2018–09–12.

[2] R. Müller et al., “Wandlungsfähiges montagesystem für großbauteile am beispiel der flugzeugstrukturmontage,” Zukunftsfähige Montagesysteme. Wirtschaftlich, wandlungsfähig und rekonfigurierbar. Stuttgart: Fraunhofer Verlag, 2013.

[3] B. Corves, R. Müller, M. Esser, M. Hüsing, M. Janßen, M. Riedel, and M. Vette, “Reconfigurable assembly systems for handling large components,” Integrative Production Technology for High-Wage Countries. Berlin, Heidelberg: Springer-Verlag, pp. 946–971, 2012.

[4] E. Gambao, M. Hernando, and D. Surdilovic, “A new generation of collaborative robots for material handling,” in ISARC. Proceedings of the International Symposium on Automation and Robotics in Construction, vol. 29. Vilnius Gediminas Technical University, Department of Construction Economics & Property, 2012, p. 1.

[5] R. Voges, C. S. Wieghardt, and B. Wagner, “Timestamp offset determination between an actuated laser scanner and its corresponding motor,” May 2018.

[6] “Finding timestamp offsets for a multi-sensor system using sensor observations,” Photogrammetric Engineering and Remote Sensing, vol. 84, p. 357–366, May 2018. [Online]. Available:

[7] H. Kang, J. W. Noh, and S. J. Kwak, “Synchronization method for laser scanner and robot,” in 2012 12th International Conference on Control, Automation and Systems, Oct 2012, pp. 952–955.

[8] M. Graaf, R. Aarts, J. Meijer, B. Jonker, E. Beyer, F. Dausinger, A. Ostendorf, and A. Otto, “Robot-sensor synchronization for realtime seamtracking in robotic laser welding,” 01 2005.

[9] M. Zaman and J. Illingworth, “Interval-based time synchronisation of sensor data in a mobile robot,” in Proceedings of the 2004 Intelligent Sensors, Sensor Networks and Information Processing Conference, 2004., Dec 2004, pp. 463–468.

[10] F. Kanehiro, Y. Ishiwata, H. Saito, K. Akachi, G. Miyamori, T. Isozumi, K. Kaneko, and H. Hirukawa, “Distributed control system of humanoid robots based on real-time ethernet,” in 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, Oct 2006, pp. 2471–2477.

[11] N. Kaempchen and K. Dietmayer, “Data synchronization strategies for multi-sensor fusion,” 08 2018.

[12] T. Andersen, O. Ravn, and N. Andersen, “Sensor based real-time control of robots,” Ph.D. dissertation, Technical University of Denmark, Department of Electrical Engineering, 2015.

[13] M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, and A. Y. Ng, “Ros: an open-source robot operating system,” in ICRA workshop on open source software, vol. 3, no. 3.2. Kobe, Japan, 2009, p. 5.

[14] “Clock — ROS Documentation, OSRF,” accessed: 2018–09–31.

[15] T. Foote, “tf: The transform library,” in Technologies for Practical Robot Applications (TePRA), 2013 IEEE International Conference on, ser. Open-Source Software workshop, April 2013, pp. 1–6.

[16] “ROS 2.0 Clock and Time — ROS 2 Design, OSRF,” ros2.org/articles/clock_and_time.html, accessed: 2018–09–31.

[17] E. Olson, “A passive solution to the sensor synchronization problem,” in 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, Oct 2010, pp. 1059–1064.

[18] “ROSCon 2017: Building a Computer Vision Research Vehicle with ROS — Andreas Fregin,” ROSCon%202017%20Computer%20Vision%20Research%20Vehicle. pdf, accessed: 2018–09–31.

[19] T. Neagoe, V. Cristea, and L. Banica, “Ntp versus ptp in com puter networks clock synchronization,” in 2006 IEEE International Symposium on Industrial Electronics, vol. 1, July 2006, pp. 317–362.

[20] V. Mayoral, A. Hernández, R. Kojcev, I. Muguruza, I. Zamalloa, A. Bilbao, and L. Usategi, “The shift in the robotics paradigm — the hardware robot operating system (h-ros); an infrastructure to create interoperable robot components,” in 2017 NASA/ESA Conference on Adaptive Hardware and Systems (AHS), July 2017, pp. 229–236.

[21] I. Zamalloa, I. Muguruza, A. Hernández, R. Kojcev, and V. Mayoral, “An information model for modular robots: the hardware robot information model (hrim),” arXiv preprint arXiv:1802.01459, 2018.

[22] C. S. V. Gutiérrez, L. U. S. Juan, I. Z. Ugarte, and V. M. Vilches, “Time-sensitive networking for robotics,” CoRR, vol. abs/1804.07643, 2018. [Online]. Available:

[23] IEEE, “Ieee standard for local and metropolitan area networks — timing and synchronization for time-sensitive applications in bridged local area networks,” IEEE Std 802.1AS-2011, pp. 1–292, March 2011.

[24] — — , “Ieee standard for local and metropolitan area networks — bridges and bridged networks — amendment 25: Enhancements for scheduled traffic,” IEEE Std 802.1Qbv-2015 (Amendment to IEEE Std 802.1Q-2014 as amended by IEEE Std 802.1Qca-2015, IEEE Std 802.1Qcd-2015, and IEEE Std 802.1Q-2014/Cor 1–2015), pp. 1–57, March 2016.

[25] I. Zamalloa, I. Muguruza, A. Hernández, R. Kojcev, and V. Mayoral, “An information model for modular robots: the hardware robot information model (hrim),” arXiv preprint arXiv:1802.01459, 2018.

[26] C. S. V. Gutiérrez, L. Usategui San Juan, I. Zamalloa Ugarte, and V. Mayoral Vilches, “Real-time Linux communications: an evaluation of the Linux communication stack for real-time robotic applications,” ArXiv e-prints, Aug. 2018.

[27] “Time-Sensitive Networking for robotics,” ArXiv e-prints, Apr. 2018.

[28] J. Pfrommer, A. Ebner, S. Ravikumar, and B. Karunakaran, “Open source opc ua pubsub over tsn for realtime industrial communication,” 07 2018.

[29] J. Diesilva Sagaya, B. Jayachandran Ramesh, F. Melvin, and K. Pavithra, “Real-time open source solution for industrial communication using opc ua pubsub over tsn,” 072018. [Online]. Available: Real-time-Open-Source-Solution-for-Industrial-Communication-Using-OPC-UA-PubSub-over-TSN. pdf