TY - JOUR A1 - Engemann, Heiko A1 - Cönen, Patrick A1 - Dawar, Harshal A1 - Du, Shengzhi A1 - Kallweit, Stephan T1 - A robot-assisted large-scale inspection of wind turbine blades in manufacturing using an autonomous mobile manipulator JF - Applied Sciences N2 - Wind energy represents the dominant share of renewable energies. The rotor blades of a wind turbine are typically made from composite material, which withstands high forces during rotation. The huge dimensions of the rotor blades complicate the inspection processes in manufacturing. The automation of inspection processes has a great potential to increase the overall productivity and to create a consistent reliable database for each individual rotor blade. The focus of this paper is set on the process of rotor blade inspection automation by utilizing an autonomous mobile manipulator. The main innovations include a novel path planning strategy for zone-based navigation, which enables an intuitive right-hand or left-hand driving behavior in a shared human–robot workspace. In addition, we introduce a new method for surface orthogonal motion planning in connection with large-scale structures. An overall execution strategy controls the navigation and manipulation processes of the long-running inspection task. The implemented concepts are evaluated in simulation and applied in a real-use case including the tip of a rotor blade form. KW - mobile manipulation KW - large-scale inspection KW - wind turbine production KW - autonomous navigation KW - surface-orthogonal path planning Y1 - 2021 U6 - https://doi.org/10.3390/app11199271 SN - 2076-3417 N1 - Belongs to the Special Issue "Advances in Industrial Robotics and Intelligent Systems" VL - 11 IS - 19 SP - 1 EP - 22 PB - MDPI CY - Basel ER - TY - CHAP A1 - Engemann, Heiko A1 - Wiesen, Patrick A1 - Kallweit, Stephan A1 - Deshpande, Harshavardhan A1 - Schleupen, Josef T1 - Autonomous mobile manipulation using ROS T2 - Advances in Service and Industrial Robotics Y1 - 2018 SN - 978-3-319-61276-8 U6 - https://doi.org/10.1007/978-3-319-61276-8_43 N1 - International Conference on Robotics in Alpe-Adria Danube Region RAAD 2017; Mechanisms and Machince Science book series, Vol 49. SP - 389 EP - 401 PB - Springer CY - Cham ER - TY - JOUR A1 - Franko, Josef A1 - Du, Shengzhi A1 - Kallweit, Stephan A1 - Duelberg, Enno Sebastian A1 - Engemann, Heiko T1 - Design of a Multi-Robot System for Wind Turbine Maintenance JF - Energies N2 - The maintenance of wind turbines is of growing importance considering the transition to renewable energy. This paper presents a multi-robot-approach for automated wind turbine maintenance including a novel climbing robot. Currently, wind turbine maintenance remains a manual task, which is monotonous, dangerous, and also physically demanding due to the large scale of wind turbines. Technical climbers are required to work at significant heights, even in bad weather conditions. Furthermore, a skilled labor force with sufficient knowledge in repairing fiber composite material is rare. Autonomous mobile systems enable the digitization of the maintenance process. They can be designed for weather-independent operations. This work contributes to the development and experimental validation of a maintenance system consisting of multiple robotic platforms for a variety of tasks, such as wind turbine tower and rotor blade service. In this work, multicopters with vision and LiDAR sensors for global inspection are used to guide slower climbing robots. Light-weight magnetic climbers with surface contact were used to analyze structure parts with non-destructive inspection methods and to locally repair smaller defects. Localization was enabled by adapting odometry for conical-shaped surfaces considering additional navigation sensors. Magnets were suitable for steel towers to clamp onto the surface. A friction-based climbing ring robot (SMART— Scanning, Monitoring, Analyzing, Repair and Transportation) completed the set-up for higher payload. The maintenance period could be extended by using weather-proofed maintenance robots. The multi-robot-system was running the Robot Operating System (ROS). Additionally, first steps towards machine learning would enable maintenance staff to use pattern classification for fault diagnosis in order to operate safely from the ground in the future. Y1 - 2020 U6 - https://doi.org/10.3390/en13102552 SN - 1996-1073 VL - 13 IS - 10 SP - Article 2552 PB - MDPI CY - Basel ER - TY - CHAP A1 - Engemann, Heiko A1 - Du, Shengzhi A1 - Kallweit, Stephan A1 - Ning, Chuanfang A1 - Anwar, Saqib T1 - AutoSynPose: Automatic Generation of Synthetic Datasets for 6D Object Pose Estimation T2 - Machine Learning and Artificial Intelligence. Proceedings of MLIS 2020 N2 - We present an automated pipeline for the generation of synthetic datasets for six-dimension (6D) object pose estimation. Therefore, a completely automated generation process based on predefined settings is developed, which enables the user to create large datasets with a minimum of interaction and which is feasible for applications with a high object variance. The pipeline is based on the Unreal 4 (UE4) game engine and provides a high variation for domain randomization, such as object appearance, ambient lighting, camera-object transformation and distractor density. In addition to the object pose and bounding box, the metadata includes all randomization parameters, which enables further studies on randomization parameter tuning. The developed workflow is adaptable to other 3D objects and UE4 environments. An exemplary dataset is provided including five objects of the Yale-CMU-Berkeley (YCB) object set. The datasets consist of 6 million subsegments using 97 rendering locations in 12 different UE4 environments. Each dataset subsegment includes one RGB image, one depth image and one class segmentation image at pixel-level. Y1 - 2020 SN - 978-1-64368-137-5 U6 - https://doi.org/10.3233/FAIA200770 N1 - Frontiers in Artificial Intelligence and Applications. Vol 332 SP - 89 EP - 97 PB - IOS Press CY - Amsterdam ER - TY - JOUR A1 - Engemann, Heiko A1 - Du, Shengzhi A1 - Kallweit, Stephan A1 - Cönen, Patrick A1 - Dawar, Harshal T1 - OMNIVIL - an autonomous mobile manipulator for flexible production JF - Sensors Y1 - 2020 SN - 1424-8220 U6 - https://doi.org/10.3390/s20247249 N1 - Special issue: Sensor Networks Applications in Robotics and Mobile Systems VL - 20 IS - 24, art. no. 7249 SP - 1 EP - 30 PB - MDPI CY - Basel ER - TY - JOUR A1 - Coll-Perales, Baldomero A1 - Schulte-Tigges, Joschua A1 - Rondinone, Michele A1 - Gozalvez, Javier A1 - Reke, Michael A1 - Matheis, Dominik A1 - Walter, Thomas T1 - Prototyping and evaluation of infrastructure-assisted transition of control for cooperative automated vehicles JF - IEEE Transactions on Intelligent Transportation Systems N2 - Automated driving is now possible in diverse road and traffic conditions. However, there are still situations that automated vehicles cannot handle safely and efficiently. In this case, a Transition of Control (ToC) is necessary so that the driver takes control of the driving. Executing a ToC requires the driver to get full situation awareness of the driving environment. If the driver fails to get back the control in a limited time, a Minimum Risk Maneuver (MRM) is executed to bring the vehicle into a safe state (e.g., decelerating to full stop). The execution of ToCs requires some time and can cause traffic disruption and safety risks that increase if several vehicles execute ToCs/MRMs at similar times and in the same area. This study proposes to use novel C-ITS traffic management measures where the infrastructure exploits V2X communications to assist Connected and Automated Vehicles (CAVs) in the execution of ToCs. The infrastructure can suggest a spatial distribution of ToCs, and inform vehicles of the locations where they could execute a safe stop in case of MRM. This paper reports the first field operational tests that validate the feasibility and quantify the benefits of the proposed infrastructure-assisted ToC and MRM management. The paper also presents the CAV and roadside infrastructure prototypes implemented and used in the trials. The conducted field trials demonstrate that infrastructure-assisted traffic management solutions can reduce safety risks and traffic disruptions. KW - Automated driving KW - automated vehicles KW - connected automated vehicles KW - CAV KW - experimental evaluation Y1 - 2021 U6 - https://doi.org/10.1109/TITS.2021.3061085 SN - 1524-9050 (Print) SN - 1558-0016 (Online) VL - 23 IS - 7 SP - 6720 EP - 6736 PB - IEEE ER - TY - CHAP A1 - Schulte-Tigges, Joschua A1 - Matheis, Dominik A1 - Reke, Michael A1 - Walter, Thomas A1 - Kaszner, Daniel ED - Krömker, Heidi T1 - Demonstrating a V2X enabled system for transition of control and minimum risk manoeuvre when leaving the operational design domain T2 - HCII 2023: HCI in Mobility, Transport, and Automotive Systems N2 - Modern implementations of driver assistance systems are evolving from a pure driver assistance to a independently acting automation system. Still these systems are not covering the full vehicle usage range, also called operational design domain, which require the human driver as fall-back mechanism. Transition of control and potential minimum risk manoeuvres are currently research topics and will bridge the gap until full autonomous vehicles are available. The authors showed in a demonstration that the transition of control mechanisms can be further improved by usage of communication technology. Receiving the incident type and position information by usage of standardised vehicle to everything (V2X) messages can improve the driver safety and comfort level. The connected and automated vehicle’s software framework can take this information to plan areas where the driver should take back control by initiating a transition of control which can be followed by a minimum risk manoeuvre in case of an unresponsive driver. This transition of control has been implemented in a test vehicle and was presented to the public during the IEEE IV2022 (IEEE Intelligent Vehicle Symposium) in Aachen, Germany. KW - V2X KW - Transiton of Control KW - Minimum Risk Manoeuvre KW - Operational Design Domain KW - Connected Automated Vehicle Y1 - 2023 SN - 978-3-031-35677-3 (Print) SN - 978-3-031-35678-0 (Online) U6 - https://doi.org/10.1007/978-3-031-35678-0_12 N1 - 5th International Conference, MobiTAS 2023, Held as Part of the 25th HCI International Conference, HCII 2023, Copenhagen, Denmark, July 23–28, 2023. SP - 200 EP - 210 PB - Springer CY - Cham ER - TY - CHAP A1 - Ferrein, Alexander A1 - Maier, Christopher A1 - Mühlbacher, Clemens A1 - Niemüller, Tim A1 - Steinbauer, Gerald A1 - Vassos, Stravros T1 - Controlling logistics robots with the action-based language YAGI T2 - Intelligent Robotics and Applications: 9th International Conference, ICIRA 2016, Tokyo, Japan, August 22-24, 2016, Proceedings, Part I Y1 - 2016 SN - 978-3-319-43505-3 (Print) SN - 978-3-319-43506-0 (Online) U6 - https://doi.org/10.1007/978-3-319-43506-0_46 N1 - Series: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) VL - 9834 SP - 525 EP - 537 PB - Springer ER - TY - CHAP A1 - Niemueller, Tim A1 - Ferrein, Alexander A1 - Reuter, Sebastian A1 - Jeschke, Sabina A1 - Lakemeyer, Gerhard T1 - The RoboCup Logistics League as a Holistic Multi-Robot Smart Factory Benchmark T2 - Proceedings of the IROS 2015 Open forum on evaluation of results, replication of experiments and benchmarking in robotics research N2 - With autonomous mobile robots receiving increased attention in industrial contexts, the need for benchmarks becomes more and more an urgent matter. The RoboCup Logistics League (RCLL) is one specific industry-inspired scenario focusing on production logistics within a Smart Factory. In this paper, we describe how the RCLL allows to assess the performance of a group of robots within the scenario as a whole, focusing specifically on the coordination and cooperation strategies and the methods and components to achieve them. We report on recent efforts to analyze performance of teams in 2014 to understand the implications of the current grading scheme, and derived criteria and metrics for performance assessment based on Key Performance Indicators (KPI) adapted from classic factory evaluation. We reflect on differences and compatibility towards RoCKIn, a recent major benchmarking European project. Y1 - 2015 ER - TY - CHAP A1 - Eichenbaum, Julian A1 - Nikolovski, Gjorgji A1 - Mülhens, Leon A1 - Reke, Michael A1 - Ferrein, Alexander A1 - Scholl, Ingrid T1 - Towards a lifelong mapping approach using Lanelet 2 for autonomous open-pit mine operations T2 - 2023 IEEE 19th International Conference on Automation Science and Engineering (CASE) N2 - Autonomous agents require rich environment models for fulfilling their missions. High-definition maps are a well-established map format which allows for representing semantic information besides the usual geometric information of the environment. These are, for instance, road shapes, road markings, traffic signs or barriers. The geometric resolution of HD maps can be as precise as of centimetre level. In this paper, we report on our approach of using HD maps as a map representation for autonomous load-haul-dump vehicles in open-pit mining operations. As the mine undergoes constant change, we also need to constantly update the map. Therefore, we follow a lifelong mapping approach for updating the HD maps based on camera-based object detection and GPS data. We show our mapping algorithm based on the Lanelet 2 map format and show our integration with the navigation stack of the Robot Operating System. We present experimental results on our lifelong mapping approach from a real open-pit mine. Y1 - 2023 SN - 979-8-3503-2069-5 (Online) SN - 979-8-3503-2070-1 (Print) U6 - https://doi.org/10.1109/CASE56687.2023.10260526 N1 - 19th International Conference on Automation Science and Engineering (CASE), 26-30 August 2023, Auckland, New Zealand. PB - IEEE ER -