[논문]Assistive Grasping Based on Laser-point Detection with Application to Wheelchair-mounted Robotic Arms

Zhong, Ming; Zhang, Yanqiang; Yang, Xi; Yao, Yufeng; Guo, Junlong; Wang, Yaping; Liu, Yaxin

doi:10.3390/s19020303

Assistive Grasping Based on Laser-point Detection with Application to Wheelchair-mounted Robotic Arms 원문보기

Sensors, v.19 no.2, 2019년, pp.303 -

Zhong, Ming (Industrial Research Institute of Robotics and Intelligent Equipment, Harbin Institute of Technology, Weihai 264209, China) , Zhang, Yanqiang (zhongming@hit.edu.cn (M.Z.)) , Yang, Xi (15732031132@163.com (Y.Z.)) , Yao, Yufeng (yangxi_hit@163.com (X.Y.)) , Guo, Junlong (Yyf1023@163.com (Y.Y.)) , Wang, Yaping (junlongg@hit.edu.cn (J.G.)) , Liu, Yaxin (Industrial Research Institute of Robotics and Intelligent Equipment, Harbin Institute of Technology, Weihai 264209, China)

Abstract ▼ AI-Helper

As the aging of the population becomes more severe, wheelchair-mounted robotic arms (WMRAs) are gaining an increased amount of attention. Laser pointer interactions are an attractive method enabling humans to unambiguously point out objects and pick them up. In addition, they bring about a greater sense of participation in the interaction process as an intuitive interaction mode. However, the issue of human–robot interactions remains to be properly tackled, and traditional laser point interactions still suffer from poor real-time performance and low accuracy amid dynamic backgrounds. In this study, combined with an advanced laser point detection method and an improved pose estimation algorithm, a laser pointer is used to facilitate the interactions between humans and a WMRA in an indoor environment. Assistive grasping using a laser selection consists of two key steps. In the first step, the images captured using an RGB-D camera are pre-processed, and then fed to a convolutional neural network (CNN) to determine the 2D coordinates of the laser point and objects within the image. Meanwhile, the centroid coordinates of the selected object are also obtained using the depth information. In this way, the object to be picked up and its location are determined. The experimental results show that the laser point can be detected with almost 100% accuracy in a complex environment. In the second step, a compound pose-estimation algorithm aiming at a sparse use of multi-view templates is applied, which consists of both coarse- and precise-matching of the target to the template objects, greatly improving the grasping performance. The proposed algorithms were implemented on a Kinova Jaco robotic arm, and the experimental results demonstrate their effectiveness. Compared with commonly accepted methods, the time consumption of the pose generation can be reduced from 5.36 to 4.43 s, and synchronously, the pose estimation error is significantly improved from 21.31% to 3.91%.

주제어

참고문헌 (34)

1. Tang Y. Dong F. Yamazaki Y. Shibata T. Hirota K. Deep Level Situation Understanding for Casual Communication in Humans-Robots Interaction Int. J. Fuzzy Log. Intell. Syst. 2015 15 1 11 10.5391/IJFIS.2015.15.1.1

원문보기 상세보기
2. Wu Q. Wu H. Development, Dynamic Modeling, and Multi-Modal Control of a Therapeutic Exoskeleton for Upper Limb Rehabilitation Training Sensors 2018 18 3611 10.3390/s18113611 30356005

상세보기
3. Lee H.K. Kim J.H. An HMM-based threshold model approach for gesture recognition IEEE Trans. Pattern Anal. Mach. Intell. 1999 21 961 973

상세보기
4. Tanaka H. Sumi Y. Matsumoto Y. Assistive robotic arm autonomously bringing a cup to the mouth by face recognition Proceedings of the 2010 IEEE Advanced Robotics and ITS Social Impacts Seoul, Korea 26–28 October 2010 34 39
5. Kazi Z. Foulds R. Knowledge driven planning and multimodal control of a telerobot Robotica 1998 16 509 516 10.1017/S0263574798000666

상세보기
6. Rouanet P. Oudeyer P.Y. Danieau F. Filliat D. The impact of human–robot interfaces on the learning of visual objects IEEE Trans. Robot. 2013 29 525 541 10.1109/TRO.2012.2228134

상세보기
7. Choi K. Min B.K. Future directions for brain-machine interfacing technology Recent Progress in Brain and Cognitive Engineering Springer Dordrecht, The Netherlands 2015
8. Imtiaz N. Mustafa M.M. Hussain A. Scavino E. Laser pointer detection based on intensity profile analysis for application in teleconsultation J. Eng. Sci. Technol. 2017 12 2238 2253
9. Kang S.H. Yang C.K. Laser-pointer human computer interaction system Proceedings of the IEEE International Conference on Multimedia & Expo Workshops Turin, Italy 29 June–3 July 2015 1 6
10. Karvelis P. Roijezon U. Faleij R. Georgoulas G. Mansouri S.S. Nikolakopoulos G. A laser dot tracking method for the assessment of sensorimotor function of the hand Proceedings of the Mediterranean Conference on Control and Automation Valletta, Malta 3–6 July 2017 217 222
11. Fukuda Y. Kurihara Y. Kobayashi K. Watanabe K. Development of electric wheelchair interface based on laser pointer Proceedings of the ICCAS-SICE Fukuoka, Japan 18–21 August 2009 1148 1151
12. Gualtieri M. Kuczynski J. Shultz A.M. Pas A.T. Platt R. Yanco H. Open world assistive grasping using laser selection Proceedings of the IEEE International Conference on Robotics and Automation Singapore 29 May–3 June 2017 4052 4057
13. Kemp C.C. Anderson C.D. Hai N. Trevor A.J. Xu Z. A point-and-click interface for the real world: Laser designation of objects for mobile manipulation Proceedings of the 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction Amsterdam, The Netherlands 12–15 March 2008
14. Hai N. Anderson C. Trevor A. Jain A. Xu Z. Kemp C.C. EL-E: An assistive robot that fetches objects from flat surfaces Proceedings of the Robotic Helpers Workshop at HRI’08 Amsterdam, The Netherlands 12 March 2008
15. Jain A. Kemp C.C. EL-E: An assistive mobile manipulator that autonomously fetches objects from flat surfaces Auton. Robot. 2010 28 45 10.1007/s10514-009-9148-5

상세보기
16. Lapointe J.F. Godin G. On-screen laser spot detection for large display interaction Proceedings of the IEEE International Workshop on Haptic Audio Visual Environments & Their Applications Ottawa, ON, Canada 1 October 2005
17. Nguyen H. Jain A. Anderson C. Kemp C.C. A clickable world: behavior selection through pointing and context for mobile manipulation Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems Nice, France 22–26 September 2008 787 793
18. Zhou P. Wang X. Huang Q. Ma C. Laser spot center detection based on improved circle fitting algorithm Proceedings of the 2018 2nd IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC) Xi’an, China 25–27 May 2016
19. Stauffer C. Grimson W.E.L. Adaptive background mixture models for real-time tracking Proceedings of the 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149) Fort Collins, CO, USA 23–25 June 1999 Volume 2 2246
20. Geng L. Xiao Z. Real time foreground-background segmentation using two-layer codebook model Proceedings of the 2011 International Conference on Control, Automation and Systems Engineering Singapore 30–31 July 2011 1 5
21. Zhang B. Gu J. Chen C. Han J. Su X. Cao X. Liu J. One-two-one networks for compression artifacts reduction in remote sensing ISPRS J. Photogramm. Remote Sens. 2018 145 184 196 10.1016/j.isprsjprs.2018.01.003

상세보기
22. Zhang B. Li Z. Cao X. Ye Q. Chen C. Shen L. Perina A. Jill R. Output Constraint Transfer for Kernelized Correlation Filter in Tracking IEEE Trans. Syst. Man Cybern. Syst. 2017 47 693 703 10.1109/TSMC.2016.2629509

상세보기
23. Jeon W.-S. Rhee S.-Y. Plant Leaf Recognition Using a Convolution Neural Network Int. J. Fuzzy Log. Intell. Syst. 2017 17 26 34 10.5391/IJFIS.2017.17.1.26

상세보기
24. Shin M. Lee J.-H. CNN Based Lithography Hotspot Detection Int. J. Fuzzy Log. Intell. Syst. 2016 16 208 215 10.5391/IJFIS.2016.16.3.208

원문보기 상세보기
25. Chu J. Guo Z. Leng L. Object Detection Based on Multi-Layer Convolution Feature Fusion and Online Hard Example Mining IEEE Access 2018 6 19959 19967 10.1109/ACCESS.2018.2815149

상세보기
26. Jiang S. Yao W. Hong Z. Li L. Su C. Kuc T.-Y. A Classification-Lock Tracking Strategy Allowing a Person-Following Robot to Operate in a Complicated Indoor Environment Sensors 2018 18 3903 10.3390/s18113903 30424577

상세보기
27. Choi H. CNN Output Optimization for More Balanced Classification Int. J. Fuzzy Log. Intell. Syst. 2017 17 98 106 10.5391/IJFIS.2017.17.2.98

상세보기
28. Redmon J. Farhadi A. YOLOv3: An Incremental Improvement arXiv 2018 1804.02767
29. Luan S. Zhang B. Zhou S. Chen C. Han J. Yang W. Liu J. Gabor Convolutional Networks IEEE Trans. Image Process. 2018 27 4357 4366 10.1109/TIP.2018.2835143 29870353

상세보기
30. Zhang B. Perina A. Li Z. Murino V. Liu J. Ji R. Bounding Multiple Gaussians Uncertainty with Application to Object Tracking Int. J. Comput. Vis. 2016 118 364 379 10.1007/s11263-016-0880-y

상세보기
31. Rusu R.B. Bradski G. Thibaux R. Hsu J. Fast 3D recognition and pose using the Viewpoint Feature Histogram Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems Taipei, Taiwan 18–22 October 2014 2155 2162
32. Aldoma A. Vincze M. Blodow N. Gossow D. Gedikli S. Rusu R.B. Bradski G. CAD-model recognition and 6DOF pose estimation using 3D cues Proceedings of the IEEE International Conference on Computer Vision Workshops Barcelona, Spain 6–13 November 2011 585 592
33. Filipe S. Alexandre L.A. A comparative evaluation of 3D keypoint detectors in a RGB-D object dataset Proceedings of the International Conference on Computer Vision Theory and Applications Lisbon, Portugal 5–8 January 2014 476 483
34. Tombari F. Stefano L.D. Object recognition in 3D scenes with occlusions and clutter by Hough Voting Proceedings of the 2010 Fourth Pacific-Rim Symposium on Image and Video Technology Singapore 14–17 November 2010 349 355

내보내기 구분	파일저장 인쇄 메일전송
구성항목	기본정보 상세정보 관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관 관리번호, 논문명, 대등논문명, 저자 , 저널/프로시딩명, 발행기관, 발행년, 발행언어, 권, 호, 시작페이지, 끝페이지, ISBN, ISSN, 주제분야, 키워드, 초록(한글), 초록(영문), 저자(소속기관)
저장형식	Text(ASCII format) Excel format RefWorks Direct Export RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley
메일정보	받는사람 (필수) @ 보내는사람 (선택) @ 제목 내용 KISTI 검색결과 이메일 서비스
안내	총 건의 자료가 검색되었습니다. 다운받으실 자료의 인덱스를 입력하세요. (1-10,000) 검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다. 데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요) 다운로드 파일은 UTF-8 형태로 저장됩니다. 파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오. ~ Text(ASCII format) Excel format

연합인증

Assistive Grasping Based on Laser-point Detection with Application to Wheelchair-mounted Robotic Arms 원문보기

Abstract ▼ AI-Helper

주제어

참고문헌 (34)

이 논문을 인용한 문헌

관련 콘텐츠

원문 보기

원문 URL 링크

오픈액세스(OA) 유형

이 논문과 함께 이용한 콘텐츠

AI-Helper ※ AI-Helper는 오픈소스 모델을 사용합니다.

선택된 텍스트