[논문]COLREGs-compliant multiship collision avoidance based on deep reinforcement learning

Zhao, Luman; Roh, Myung-Il

doi:10.1016/j.oceaneng.2019.106436

COLREGs-compliant multiship collision avoidance based on deep reinforcement learning

Ocean engineering, v.191, 2019년, pp.106436 -

Zhao, Luman (Department of Ocean Operations and Civil Engineering, Norwegian University of Science and Technology) , Roh, Myung-Il (Department of Naval Architecture and Ocean Engineering, Research Institute of Marine Systems Engineering, Seoul National University)

Abstract ▼ AI-Helper

Abstract Developing a high-level autonomous collision avoidance system for ships that can operate in an unstructured and unpredictable environment is challenging. Particularly in congested sea areas, each ship should make decisions continuously to avoid collisions with other ships in a busy and complex waterway. Furthermore, recent reports indicate that a large number of marine collision accidents are caused by or are related to human decision failures concerning a lack of situational awareness and failure to comply with the Convention on the International Regulations for Preventing Collisions at Sea (COLREGs). In this study, we propose an efficient method to overcome multiship collision avoidance problems based on the Deep Reinforcement Learning (DRL) algorithm by expanding our previous study (Zhao et al., 2019). The proposed method directly maps the states of encountered ships to an ownship's steering commands in terms of rudder angle using the Deep Neural Network (DNN). This DNN is trained over multiple ships in rich encountering situations using the policy-gradient based DRL algorithm. To address multiple encountered ships, we classify them into four regions based on COLREGs, and consider only the nearest ship in each region. We validate the proposed collision avoidance method in a variety of simulated scenarios with thorough performance evaluations, and demonstrate that the final DRL controller can obtain time efficient and collision-free paths for multiple ships. Simulation results indicate that multiple ships can avoid collisions with each other while following their own predefined paths simultaneously. In addition, the proposed approach demonstrates its excellent adaptability to unknown complex environments with various encountered ships. Highlights This study deals with a multiship collision avoidance problem in congested sea areas. A COLREGs-compliant method for multiship collision avoidance is proposed based on deep reinforcement learning. A novel strategy is used to solve the problem by categorizing target ships into four regions defined by COLREGs. The proposed method is applied to a wide range of problems, from single collision avoidance to multiple collision avoidance.

주제어

참고문헌 (27)

Ocean Eng. Abdelaal 160 168 2018 10.1016/j.oceaneng.2018.04.026 Nonlinear model predictive control for trajectory tracking and collision avoidance of underactuated vessels with disturbances

상세보기
J. Mar. Sci. Appl. Bhopale 18 2 228 2019 10.1007/s11804-019-00089-3 Reinforcement learning based obstacle avoidance for autonomous underwater vehicle

상세보기
IEEE Conf. Intell. Transp. Syst. Proc. ITSC Chae 1 2018 Autonomous braking system via deep reinforcement learning
Neurocomputing Cheng 272 63 2018 10.1016/j.neucom.2017.06.066 Concise deep reinforcement learning obstacle avoidance for underactuated unmanned marine vessels

상세보기
Cui vol. 11 4 2019 Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems, Macau, China Reinforcement learning ship autopilot: sample-efficient and model predictive control-based approach
1st Annu. IEEE Conf. Control Technol. Appl. CCTA 2017 Eriksen 766 2017 MPC-Based mid-level collision avoidance for ASVs using nonlinear programming
Everett vol. 10 1 2018 Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, Madrid, Spain Motion planning among dynamic, decision-making agents with deep reinforcement learning
Fossen 2011 Handbook of Marine Craft Hydrodynamics and Motion Control
Ocean Eng. He 140 281 2017 10.1016/j.oceaneng.2017.05.029 Quantitative analysis of COLREG rules and seamanship for autonomous collision avoidance at open sea

상세보기
IEEE Trans. Intell. Transp. Syst. Johansen 17 3407 2016 10.1109/TITS.2016.2551780 Ship collision avoidance and COLREGS compliance using simulation-based control behavior selection with predictive hazard assessment

상세보기
Kahn vol. 9 24 2017 Proceedings of the 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vancouver, Canada Uncertainty-aware reinforcement learning for collision avoidance
Kingma vol. 4 14 2014 Proceedings of the 3rd International Conference on Learning Representations, Banff, Canada Adam: a method for stochastic optimization
Long vol. 5 21 2018 Towards optimally decentralized multi-robot collision avoidance via deep reinforcement learning
Ocean Eng. Mou 37 483 2010 10.1016/j.oceaneng.2010.01.012 Study on collision avoidance in busy waterways by using AIS data

상세보기
Blanke 1 2002 Mathematical Ship Modeling for Control Applications. Technical Report
Schulman 2015 High-Dimensional Continuous Control Using Generalized Advantage Estimation 1-14
Schulman 2017 Proximal Policy Optimization Algorithms 1-12
Appl. Ocean Res. Shen 86 268 2019 10.1016/j.apor.2019.02.020 Automatic collision avoidance of multiple ships based on deep Q-learning

상세보기
Singla 2018 Memory-based Deep Reinforcement Learning for Obstacle Avoidance in Uav with Limited Environment Knowledge
mierzchalski 2005 Information Processing and Security Systems Ships' domains as collision risk at sea in the evolutionary method of trajectory planning
Sutton 2017 Reinforcement Learning : an Introduction
J. Mar. Sci. Technol. Tam 15 257 2010 10.1007/s00773-010-0089-7 Collision risk assessment for ships

상세보기
IEEE Int. Conf. Robot. Autom. Wang 6189 2018 Design, modeling, and nonlinear model predictive tracking control of a novel autonomous surface vehicle
Ocean Eng. Wang 146 486 2017 10.1016/j.oceaneng.2017.08.034 The ship maneuverability based collision avoidance dynamic support system in close-quarters situation

상세보기
Ocean Eng. Zhang 105 336 2015 10.1016/j.oceaneng.2015.06.054 A distributed anti-collision decision support formulation in multi-ship encounter situations under COLREGs

상세보기
Neurocomputing Zhao 182 255 2016 10.1016/j.neucom.2015.12.028 A real-time collision avoidance learning system for unmanned surface vessels

상세보기
Accepted for Publication and appears in J. Mar. Sci. Technol. Taiwan Zhao 27 4 2019 Control method for path following and collision avoidance of autonomous ship based on deep reinforcement learning

LOADING...

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증

COLREGs-compliant multiship collision avoidance based on deep reinforcement learning

Abstract ▼ AI-Helper

주제어

참고문헌 (27)

이 논문을 인용한 문헌

연구과제 타임라인

관련 콘텐츠

원문 URL 링크

이 논문과 함께 이용한 콘텐츠

AI-Helper ※ AI-Helper는 오픈소스 모델을 사용합니다.

선택된 텍스트

연합인증

COLREGs-compliant multiship collision avoidance based on deep reinforcement learning

Abstract ▼ AI-Helper

주제어

참고문헌 (27)

이 논문을 인용한 문헌

연구과제 타임라인

전체(0) 논문(0) 특허(0) 보고서(0)

전체(0) 논문(0) 특허(0) 보고서(0)

관련 콘텐츠

원문 URL 링크

이 논문과 함께 이용한 콘텐츠

AI-Helper ※ AI-Helper는 오픈소스 모델을 사용합니다.

선택된 텍스트