[논문]LoBSTr: Real‐time Lower‐body Pose Prediction from Sparse Upper‐body Tracking Signals

Yang, Dongseok; Kim, Doyeon; Lee, Sung‐; Hee

doi:10.1111/cgf.142631

[해외논문] LoBSTr: Real‐time Lower‐body Pose Prediction from Sparse Upper‐body Tracking Signals 원문보기

Computer graphics forum : journal of the European Association for Computer Graphics, v.40 no.2, 2021년, pp.265 - 275

Yang, Dongseok (Korea Advanced Institute of Science and Technology (KAIST)) , Kim, Doyeon (Korea Advanced Institute of Science and Technology (KAIST)) , Lee, Sung‐Hee (Korea Advanced Institute of Science and Technology (KAIST))

Abstract ▼ AI-Helper

AbstractWith the popularization of games and VR/AR devices, there is a growing need for capturing human motion with a sparse set of tracking data. In this paper, we introduce a deep neural network (DNN) based method for real‐time prediction of the lower‐body pose only from the tracking signals of the upper‐body joints. Specifically, our Gated Recurrent Unit (GRU)‐based recurrent architecture predicts the lower‐body pose and feet contact states from a past sequence of tracking signals of the head, hands, and pelvis. A major feature of our method is that the input signal is represented by the velocity of tracking signals. We show that the velocity representation better models the correlation between the upper‐body and lower‐body motions and increases the robustness against the diverse scales and proportions of the user body than position‐orientation representations. In addition, to remove foot‐skating and floating artifacts, our network predicts feet contact state, which is used to post‐process the lower‐body pose with inverse kinematics to preserve the contact. Our network is lightweight so as to run in real‐time applications. We show the effectiveness of our method through several quantitative evaluations against other architectures and input representations with respect to wild tracking data obtained from commercial VR devices.

Keyword

참고문헌 (35)

“Carnegie-Mellon Motion Capture Database”. (2013). url: %5Curl%7Bhttps://mocap.cs.cmu.edu%7D 2 10.
10.1109/WACV.2019.00156 Chiu Hsu-kuang Adeli Ehsan Wang Borui et al. “Action-agnostic human pose forecasting”.2019 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE.2019 1423-14323.
Chung Junyoung Gulcehre Caglar Cho KyungHyun andBengio Yoshua. “Empirical evaluation of gated recurrent neural networks on sequence modeling”.arXiv preprint arXiv:1412.3555(2014) 4.
10.1145/1186822.1073248 Chai JinxiangandHodgins Jessica K.“Performance animation from low-dimensional control signals”.ACM SIGGRAPH 2005 Papers.2005 686-6962.
10.1109/ICCV.2019.00081 Cheng Yu Yang Bo Wang Bo et al. “Occlusion-aware networks for 3d human pose estimation in video”.Proceedings of the IEEE International Conference on Computer Vision.2019 723-7323.
DeepMotion. “How To Make 3 Point Tracked Full-Body Avatars in VR”. (Apr.2018). url:https://blog.deepmotion.com/2018/04/30/how-to-make-3-point-tracked-full-body-avatars-in-vr/2 3.
10.1145/1186562.1015755 Grochow Keith Martin Steven L Hertzmann Aaron andPopović Zoran. “Style-based inverse kinematics”.ACM SIGGRAPH 2004 Papers.2004 522-5312.
ACM Transactions on Graphics (TOG) Huang Yinghao 1 37 2018 10.1145/3272127.3275108 Deep inertial poser: learning to reconstruct human pose from sparse inertial measurements in real time

상세보기
ACM Transactions on Graphics (TOG) Holden Daniel 53 39 2020 Learned motion matching
Holden, Daniel, Komura, Taku, Saito, Jun. Phase-functioned neural networks for character control. ACM transactions on graphics, vol.36, no.4, 1-13.

상세보기
Holden, Daniel. Robust solving of optical motion capture data by denoising. ACM transactions on graphics, vol.37, no.4, 1-12.

상세보기
Hochreiter, Sepp, Schmidhuber, Jürgen. Long Short-Term Memory. Neural computation, vol.9, no.8, 1735-1780.

상세보기
Holden, Daniel, Saito, Jun, Komura, Taku. A deep learning framework for character motion synthesis and editing. ACM transactions on graphics, vol.35, no.4, 1-11.

상세보기
10.1145/2820903.2820918 Holden Daniel Saito Jun Komura Taku andJoyce Thomas. “Learning motion manifolds with convolutional autoencoders”.SIGGRAPH Asia 2015 Technical Briefs.2015 1-46.
Jang Deok-KyeongandLee Sung-Hee. “Constructing Human Motion Manifold With Sequential Networks”.Computer Graphics Forum. Wiley Online Library.20202.
10.1145/3013971.3013987 Jiang Fan Yang Xubo andFeng Lele. “Real-time full-body motion reconstruction and recognition for off-the-shelf VR devices”.Proceedings of the 15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry-Volume 1.2016 309-3183.
Kim, SangBin, Park, Inbum, Kwon, Seongsu, Han, JungHyun. Motion Retargetting based on Dilated Convolutions and Skeleton‐specific Loss Functions. Computer graphics forum : journal of the European Association for Computer Graphics, vol.39, no.2, 497-507.

상세보기
Kreylos Oliver. “Lighthouse tracking examined”. (May2016). url:http://doc-ok.org/?page_id=610.
Lin JamesandO'Brien James. “Temporal IK: Data-Driven Pose Estimation for Virtual Realiity”. (2019) 2 3.
10.1145/1944745.1944768 Liu Huajun Wei Xiaolin Chai Jinxiang et al. “Realtime human motion control with a small number of inertial sensors”.Symposium on interactive 3D graphics and games.2011 133-1402.
Levine, Sergey, Wang, Jack M., Haraux, Alexis, Popović, Zoran, Koltun, Vladlen. Continuous character control with low-dimensional embeddings. ACM transactions on graphics, vol.31, no.4, 1-10.

상세보기
ACM Transactions on Graphics (TOG) Ling Hung Yu 40 39 2020 Character controllers using motion vaes
10.1145/1111411.1111418 Liu Guodong Zhang Jingdan Wang Wei andMcMillan Leonard. “Human motion estimation from a reduced marker set”.Proceedings of the 2006 symposium on Interactive 3D graphics and games.2006 35-422.
10.1109/CVPR.2017.497 Martinez Julieta Black Michael J andRomero Javier. “On human motion prediction using recurrent neural networks”.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017 2891-29006 8.
10.1109/WACV.2013.6474999 Ofli Ferda Chaudhry Rizwan Kurillo Gregorij et al. “Berkeley mhad: A comprehensive multimodal human action database”.2013 IEEE Workshop on Applications of Computer Vision (WACV). IEEE.2013 53-602 10.
10.1007/s11263-019-01245-6 Pavllo Dario Feichtenhofer Christoph Auli Michael andGrangier David. “Modeling human motion with quaternion-based neural networks”.International Journal of Computer Vision(2019) 1-184.

상세보기
Pavllo Dario Grangier David andAuli Michael. “Quaternet: A quaternion-based recurrent model for human motion”.arXiv preprint arXiv:1805.06485(2018) 4.
Rockwell ChrisandFouhey David F.“Full-body awareness from partial observations”.arXiv preprint arXiv:2008.06046(2020) 3.
Root-Motion. “FINAL-IK”. (2017) 5.
ACM Transactions on Graphics (TOG) Shi Mingyi 1 40 2020 MotioNet: 3D Human Motion Reconstruction from Monocular Video with Skeleton Consistency
Safonova, Alla, Hodgins, Jessica K., Pollard, Nancy S.. Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces. ACM transactions on graphics, vol.23, no.3, 514-521.

상세보기
ACM Transactions on Graphics (TOG) Starke Sebastian 54 39 2020 Local motion phases for learning multi-contact character movements
10.1145/3359996.3364240 Thomasset Vincent Caron Stéphane andWeistroffer Vincent. “Lower body control of a semi-autonomous avatar in Virtual Reality: Balance and Locomotion of a 3D Bipedal Model”.25th ACM Symposium on Virtual Reality Software and Technology.2019 1-113.
10.1109/CVPR.2018.00901 Villegas Ruben Yang Jimei Ceylan Duygu andLee Honglak. “Neural kinematic networks for unsupervised motion retargetting”.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018 8639-86484.
Sensors Wouda Frank J 19 2019 Time Coherent Full-Body Poses Estimated Using Only Five Inertial Sensors: Deep versus Shallow Learning

상세보기

LOADING...

활용도 분석정보

상세보기

다운로드

내보내기

활용도 Top5 논문

해당 논문의 주제분야에서 활용도가 높은 상위 5개 콘텐츠를 보여줍니다.
더보기 버튼을 클릭하시면 더 많은 관련자료를 살펴볼 수 있습니다.

원문 보기

AccessON 원문보기

원문 URL 링크

DOI : 10.1111/cgf.142631
EBSCOhost-Academic Search Premier : 저널
Blackwell Publishing_STM : 저널 > 논문
EBSCOHost-Business Source Premier : 저널
EBSCOHost-Business Source Corporate : 저널
EBSCO Industries, Inc. : 저널
AccessON : 저널

*원문 PDF 파일 및 링크정보가 존재하지 않을 경우 KISTI DDS 시스템에서 제공하는 원문복사서비스를 사용할 수 있습니다.

오픈액세스(OA) 유형

GREEN

저자가 공개 리포지터리에 출판본, post-print, 또는 pre-print를 셀프 아카이빙 하여 자유로운 이용이 가능한 논문

저작권 관리 안내

내보내기 메뉴

내보내기 구분

파일저장
인쇄
메일전송

구성항목

기본정보
상세정보

관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관

저장형식

Text(ASCII format)
Excel format
RefWorks Direct Export
RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley

메일정보

받는사람 (필수): @
보내는사람 (선택): @
제목
내용: KISTI 검색결과 이메일 서비스

안내

총 건의 자료가 검색되었습니다.

다운받으실 자료의 인덱스를 입력하세요. (1-10,000)

검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다.

데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요)

다운로드 파일은 UTF-8 형태로 저장됩니다.
파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오.

Text(ASCII format)
Excel format

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

AI-Helper ※ AI-Helper는 을 사용합니다.

AI-Helper

안녕하세요, AI-Helper입니다. 좌측 "선택된 텍스트"에서 텍스트를 선택하여 요약, 번역, 용어설명을 실행하세요.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.

연합인증