TY - CHAP A1 - Wittig, M. A1 - Rütters, René A1 - Bragard, Michael ED - Reiff-Stephan, Jörg ED - Jäkel, Jens ED - Schwarz, André T1 - Application of RL in control systems using the example of a rotatory inverted pendulum T2 - Tagungsband AALE 2024 : Fit für die Zukunft: praktische Lösungen für die industrielle Automation N2 - In this paper, the use of reinforcement learning (RL) in control systems is investigated using a rotatory inverted pendulum as an example. The control behavior of an RL controller is compared to that of traditional LQR and MPC controllers. This is done by evaluating their behavior under optimal conditions, their disturbance behavior, their robustness and their development process. All the investigated controllers are developed using MATLAB and the Simulink simulation environment and later deployed to a real pendulum model powered by a Raspberry Pi. The RL algorithm used is Proximal Policy Optimization (PPO). The LQR controller exhibits an easy development process, an average to good control behavior and average to good robustness. A linear MPC controller could show excellent results under optimal operating conditions. However, when subjected to disturbances or deviations from the equilibrium point, it showed poor performance and sometimes instable behavior. Employing a nonlinear MPC Controller in real time was not possible due to the high computational effort involved. The RL controller exhibits by far the most versatile and robust control behavior. When operated in the simulation environment, it achieved a high control accuracy. When employed in the real system, however, it only shows average accuracy and a significantly greater performance loss compared to the simulation than the traditional controllers. With MATLAB, it is not yet possible to directly post-train the RL controller on the Raspberry Pi, which is an obstacle to the practical application of RL in a prototyping or teaching setting. Nevertheless, RL in general proves to be a flexible and powerful control method, which is well suited for complex or nonlinear systems where traditional controllers struggle. KW - Rotatory Inverted Pendulum KW - MPC KW - LQR KW - PPO KW - Reinforcement Learning Y1 - 2024 SN - 978-3-910103-02-3 U6 - https://doi.org/10.33968/2024.53 N1 - 20. AALE-Konferenz. Bielefeld, 06.03.-08.03.2024. (Tagungsband unter https://doi.org/10.33968/2024.29) SP - 241 EP - 248 PB - le-tex publishing services GmbH CY - Leipzig ER - TY - JOUR A1 - Schulte-Tigges, Joschua A1 - Förster, Marco A1 - Nikolovski, Gjorgji A1 - Reke, Michael A1 - Ferrein, Alexander A1 - Kaszner, Daniel A1 - Matheis, Dominik A1 - Walter, Thomas T1 - Benchmarking of various LiDAR sensors for use in self-driving vehicles in real-world environments JF - Sensors N2 - Abstract In this paper, we report on our benchmark results of the LiDAR sensors Livox Horizon, Robosense M1, Blickfeld Cube, Blickfeld Cube Range, Velodyne Velarray H800, and Innoviz Pro. The idea was to test the sensors in different typical scenarios that were defined with real-world use cases in mind, in order to find a sensor that meet the requirements of self-driving vehicles. For this, we defined static and dynamic benchmark scenarios. In the static scenarios, both LiDAR and the detection target do not move during the measurement. In dynamic scenarios, the LiDAR sensor was mounted on the vehicle which was driving toward the detection target. We tested all mentioned LiDAR sensors in both scenarios, show the results regarding the detection accuracy of the targets, and discuss their usefulness for deployment in self-driving cars. KW - Lidar KW - Benchmark KW - Self-driving Y1 - 2022 U6 - https://doi.org/10.3390/s22197146 SN - 1424-8220 N1 - This article belongs to the Special Issue "Sensor Fusion for Vehicles Navigation and Robotic Systems" VL - 22 IS - 19 PB - MDPI CY - Basel ER - TY - JOUR A1 - Heinrichs, Uwe A1 - Utting, Jane F. A1 - Frauenrath, Tobias A1 - Hezel, Fabian A1 - Krombach, Gabriele A. A1 - Hodenius, Michael A. J. A1 - Kozerke, Sebastian A1 - Niendorf, Thoralf T1 - Myocardial T2 mapping free of distortion using susceptibility-weighted fast spin-echo imaging: A feasibility study at 1.5 T and 3.0 T JF - Magnetic Resonance in Medicine N2 - This study demonstrates the feasibility of applying free-breathing, cardiac-gated, susceptibility-weighted fast spin-echo imaging together with black blood preparation and navigator-gated respiratory motion compensation for anatomically accurate T₂ mapping of the heart. First, T₂ maps are presented for oil phantoms without and with respiratory motion emulation (T₂ = (22.1 ± 1.7) ms at 1.5 T and T₂ = (22.65 ± 0.89) ms at 3.0 T). T₂ relaxometry of a ferrofluid revealed relaxivities of R2 = (477.9 ± 17) mM⁻¹s⁻¹ and R2 = (449.6 ± 13) mM⁻¹s⁻¹ for UFLARE and multiecho gradient-echo imaging at 1.5 T. For inferoseptal myocardial regions mean T₂ values of 29.9 ± 6.6 ms (1.5 T) and 22.3 ± 4.8 ms (3.0 T) were estimated. For posterior myocardial areas close to the vena cava T₂-values of 24.0 ± 6.4 ms (1.5 T) and 15.4 ± 1.8 ms (3.0 T) were observed. The merits and limitations of the proposed approach are discussed and its implications for cardiac and vascular T₂-mapping are considered. Y1 - 2009 U6 - https://doi.org/10.1002/mrm.22054 SN - 1522-2594 VL - 62 IS - 3 SP - 822 EP - 828 PB - Wiley-Liss CY - New York ER - TY - JOUR A1 - Orzada, Stephan A1 - Fiedler, Thomas M. A1 - Bitz, Andreas A1 - Ladd, Mark E. A1 - Quick, Harald H. T1 - Local SAR compression with overestimation control to reduce maximum relative SAR overestimation and improve multi-channel RF array performance JF - Magnetic Resonance Materials in Physics, Biology and Medicine N2 - Objective In local SAR compression algorithms, the overestimation is generally not linearly dependent on actual local SAR. This can lead to large relative overestimation at low actual SAR values, unnecessarily constraining transmit array performance. Method Two strategies are proposed to reduce maximum relative overestimation for a given number of VOPs. The first strategy uses an overestimation matrix that roughly approximates actual local SAR; the second strategy uses a small set of pre-calculated VOPs as the overestimation term for the compression. Result Comparison with a previous method shows that for a given maximum relative overestimation the number of VOPs can be reduced by around 20% at the cost of a higher absolute overestimation at high actual local SAR values. Conclusion The proposed strategies outperform a previously published strategy and can improve the SAR compression where maximum relative overestimation constrains the performance of parallel transmission. Y1 - 2020 SN - 1352-8661 U6 - https://doi.org/10.1007/s10334-020-00890-0 IS - 34 (2021) SP - 153 EP - 164 PB - Springer CY - Heidelberg ER - TY - JOUR A1 - Fiedler, Thomas M. A1 - Ladd, Mark E. A1 - Bitz, Andreas T1 - SAR Simulations & Safety JF - NeuroImage Y1 - 2017 U6 - https://doi.org/10.1016/j.neuroimage.2017.03.035 SN - 1053-8119 IS - Epub ahead of print PB - Elsevier CY - Amsterdam ER - TY - JOUR A1 - Schiffer, Stefan A1 - Ferrein, Alexander A1 - Lakemeyer, Gerhard T1 - Caesar: an intelligent domestic service robot JF - Intelligent service robotics N2 - In this paper we present CAESAR, an intelligent domestic service robot. In domestic settings for service robots complex tasks have to be accomplished. Those tasks benefit from deliberation, from robust action execution and from flexible methods for human–robot interaction that account for qualitative notions used in natural language as well as human fallibility. Our robot CAESAR deploys AI techniques on several levels of its system architecture. On the low-level side, system modules for localization or navigation make, for instance, use of path-planning methods, heuristic search, and Bayesian filters. For face recognition and human–machine interaction, random trees and well-known methods from natural language processing are deployed. For deliberation, we use the robot programming and plan language READYLOG, which was developed for the high-level control of agents and robots; it allows combining programming the behaviour using planning to find a course of action. READYLOG is a variant of the robot programming language Golog. We extended READYLOG to be able to cope with qualitative notions of space frequently used by humans, such as “near” and “far”. This facilitates human–robot interaction by bridging the gap between human natural language and the numerical values needed by the robot. Further, we use READYLOG to increase the flexible interpretation of human commands with decision-theoretic planning. We give an overview of the different methods deployed in CAESAR and show the applicability of a system equipped with these AI techniques in domestic service robotics Y1 - 2012 U6 - https://doi.org/10.1007/s11370-012-0118-y SN - 1861-2776 N1 - Special Issue on Artificial Intelligence Techniques for Robotics: Sensing, Representation and Action, Part I VL - 5 IS - 4 SP - 259 EP - 276 PB - Springer CY - Berlin ER - TY - CHAP A1 - Schuba, Marko A1 - Höfken, Hans-Wilhelm A1 - Linzbach, Sophie T1 - An ICS Honeynet for Detecting and Analyzing Cyberattacks in Industrial Plants T2 - 2021 International Conference on Electrical, Computer and Energy Technologies (ICECET) N2 - Cybersecurity of Industrial Control Systems (ICS) is an important issue, as ICS incidents may have a direct impact on safety of people or the environment. At the same time the awareness and knowledge about cybersecurity, particularly in the context of ICS, is alarmingly low. Industrial honeypots offer a cheap and easy to implement way to raise cybersecurity awareness and to educate ICS staff about typical attack patterns. When integrated in a productive network, industrial honeypots may not only reveal attackers early but may also distract them from the actual important systems of the network. Implementing multiple honeypots as a honeynet, the systems can be used to emulate or simulate a whole Industrial Control System. This paper describes a network of honeypots emulating HTTP, SNMP, S7communication and the Modbus protocol using Conpot, IMUNES and SNAP7. The nodes mimic SIMATIC S7 programmable logic controllers (PLCs) which are widely used across the globe. The deployed honeypots' features will be compared with the features of real SIMATIC S7 PLCs. Furthermore, the honeynet has been made publicly available for ten days and occurring cyberattacks have been analyzed KW - Conpot KW - honeypot KW - honeynet KW - ICS KW - cybersecurity Y1 - 2022 SN - 978-1-6654-4231-2 SN - 978-1-6654-4232-9 U6 - https://doi.org/10.1109/ICECET52533.2021.9698746 N1 - 2021 International Conference on Electrical, Computer and Energy Technologies (ICECET). 09-10 December 2021. Cape Town, South Africa. PB - IEEE ER - TY - JOUR A1 - Lagemaat, Miriam W. A1 - Breukels, Vincent A1 - Vos, Eline K. A1 - Kerr, Adam B. A1 - Uden, Mark J. van A1 - Orzada, Stephan A1 - Bitz, Andreas A1 - Maas, Marnix C. A1 - Scheenen, Tom W. J. T1 - ¹H MR spectroscopic imaging of the prostate at 7T using spectral-spatial pulses JF - Magnetic Resonance in Medicine N2 - Purpose To assess the feasibility of prostate ¹H MR spectroscopic imaging (MRSI) using low-power spectral-spatial (SPSP) pulses at 7T, exploiting accurate spectral selection and spatial selectivity simultaneously. Methods A double spin-echo sequence was equipped with SPSP refocusing pulses with a spectral selectivity of 1 ppm. Three-dimensional prostate ¹H-MRSI at 7T was performed with the SPSP-MRSI sequence using an 8-channel transmit array coil and an endorectal receive coil in three patients with prostate cancer and in one healthy subject. No additional water or lipid suppression pulses were used. Results Prostate ¹H-MRSI could be obtained well within specific absorption rate (SAR) limits in a clinically feasible time (10 min). Next to the common citrate signals, the prostate spectra exhibited high spermine signals concealing creatine and sometimes also choline. Residual lipid signals were observed at the edges of the prostate because of limitations in spectral and spatial selectivity. Conclusion It is possible to perform prostate ¹H-MRSI at 7T with a SPSP-MRSI sequence while using separate transmit and receive coils. This low-SAR MRSI concept provides the opportunity to increase spatial resolution of MRSI within reasonable scan times. Y1 - 2016 U6 - https://doi.org/10.1002/mrm.25569 SN - 1522-2594 VL - 75 IS - 3 SP - 933 EP - 945 PB - International Society for Magnetic Resonance in Medicine ER - TY - JOUR A1 - Ferrein, Alexander A1 - Steinbauer, Gerald A1 - Vassos, Stavros T1 - Action-Based Imperative Programming with YAGI JF - AAAI Technical Report N2 - Many tasks for autonomous agents or robots are best described by a specification of the environment and a specification of the available actions the agent or robot can perform. Combining such a specification with the possibility to imperatively program a robot or agent is what we call the actionbased imperative programming. One of the most successful such approaches is Golog. In this paper, we draft a proposal for a new robot programming language YAGI, which is based on the action-based imperative programming paradigm. Our goal is to design a small, portable stand-alone YAGI interpreter. We combine the benefits of a principled domain specification with a clean, small and simple programming language, which does not exploit any side-effects from the implementation language. We discuss general requirements of action-based programming languages and outline YAGI, our action-based language approach which particularly aims at embeddability. Y1 - 2012 N1 - Cognitive Robotics, Papers from the 2012 AAAI Workshop, CogRob@AAAI 2012, Toronto, Ontario, Canada, July 22-23, 2012 SP - 24 EP - 31 PB - AAAI CY - Menlo Park ER - TY - CHAP A1 - Chajan, Eduard A1 - Schulte-Tigges, Joschua A1 - Reke, Michael A1 - Ferrein, Alexander A1 - Matheis, Dominik A1 - Walter, Thomas T1 - GPU based model-predictive path control for self-driving vehicles T2 - IEEE Intelligent Vehicles Symposium (IV) N2 - One central challenge for self-driving cars is a proper path-planning. Once a trajectory has been found, the next challenge is to accurately and safely follow the precalculated path. The model-predictive controller (MPC) is a common approach for the lateral control of autonomous vehicles. The MPC uses a vehicle dynamics model to predict the future states of the vehicle for a given prediction horizon. However, in order to achieve real-time path control, the computational load is usually large, which leads to short prediction horizons. To deal with the computational load, the control algorithm can be parallelized on the graphics processing unit (GPU). In contrast to the widely used stochastic methods, in this paper we propose a deterministic approach based on grid search. Our approach focuses on systematically discovering the search area with different levels of granularity. To achieve this, we split the optimization algorithm into multiple iterations. The best sequence of each iteration is then used as an initial solution to the next iteration. The granularity increases, resulting in smooth and predictable steering angle sequences. We present a novel GPU-based algorithm and show its accuracy and realtime abilities with a number of real-world experiments. KW - Heuristic algorithms KW - Computational modeling KW - model-predictive control KW - GPU KW - autonomous driving Y1 - 2021 SN - 978-1-7281-5394-0 U6 - https://doi.org/10.1109/IV48863.2021.9575619 N1 - 2021 IEEE Intelligent Vehicles Symposium (IV), July 11-17, 2021. Nagoya, Japan SP - 1243 EP - 1248 PB - IEEE CY - New York, NY ER -