[논문]Knowledge Transfer for On-Device Deep Reinforcement Learning in Resource Constrained Edge Computing Systems

Jang, Ingook; Kim, Hyunseok; Lee, Donghun; Son, Young-Sung; Kim, Seonghyun

doi:10.1109/access.2020.3014922

[해외논문] Knowledge Transfer for On-Device Deep Reinforcement Learning in Resource Constrained Edge Computing Systems 원문보기

IEEE access : practical research, open solutions, v.8, 2020년, pp.146588 - 146597

Jang, Ingook (Autonomous IoT Research Section, Electronics and Telecommunications Research Institute (ETRI), Daejeon, South Korea) , Kim, Hyunseok (Autonomous IoT Research Section, Electronics and Telecommunications Research Institute (ETRI), Daejeon, South Korea) , Lee, Donghun (Autonomous IoT Research Section, Electronics and Telecommunications Research Institute (ETRI), Daejeon, South Korea) , Son, Young-Sung (Autonomous IoT Research Section, Electronics and Telecommunications Research Institute (ETRI), Daejeon, South Korea) , Kim, Seonghyun (Autonomous IoT Research Section, Electronics and Telecommunications Research Institute (ETRI), Daejeon, South Korea)

Abstract ▼ AI-Helper

Deep reinforcement learning (DRL) is a promising approach for developing control policies by learning how to perform tasks. Edge devices are required to control their actions by exploiting DRL to solve tasks autonomously in various applications such as smart manufacturing and autonomous driving. However, the resource limitations of edge devices make it unfeasible for them to train their policies from scratch. It is also impractical for such an edge device to use the policy with a large number of layers and parameters, which is pre-trained by a centralized cloud infrastructure with high computational power. In this paper, we propose a method, on-device DRL with distillation (OD3), to efficiently transfer distilled knowledge of how to behave for on-device DRL in resource-constrained edge computing systems. Our proposed method makes it possible to simultaneously perform knowledge transfer and policy model compression in a single training process on edge devices with considering their limited resource budgets. The novelty of our method is to apply a knowledge distillation approach to DRL based edge device control in integrated edge cloud environments. We analyze the performance of the proposed method by implementing it on a commercial embedded system-on-module equipped with limited hardware resources. The experimental results show that 1) edge policy training with the proposed method achieves near-cloud-performance in terms of average rewards, although the size of the edge policy network is significantly smaller compared to that of the cloud policy network and 2) the training time elapsed for edge policy training with our method is reduced significantly compared to edge policy training from scratch.

참고문헌 (38)

arXiv 1711 07478 Implementing the deep Q-network roderick 2017
10.1109/ICCAD45719.2019.8942147
Proc IEEE/ACM ICCAD On Neural Architecture Search for Resource-Constrained Hardware Platforms lu 2019 1
ArXiv 1503 02531 Distilling the knowledge in a neural network hinton 2015
Proc NIPS Do deep nets really need to be deep ba 2014 2654
arXiv 1804 07612 Revisiting small batch training for deep neural networks masters 2018
Proc ICLR Adam: A method for stochastic optimization kingma 2015 1
arXiv 1603 04467 TensorFlow: Large-scale machine learning on heterogeneous distributed systems abadi 2016
10.1109/ICCV.2019.00199
Mohammadi, Mehdi, Al-Fuqaha, Ala, Sorour, Sameh, Guizani, Mohsen. Deep Learning for IoT Big Data and Streaming Analytics: A Survey. IEEE Communications surveys and tutorials, vol.20, no.4, 2923-2960.

상세보기
Proc ICLR Policy distillation rusu 2016 1
Nature Human-level control through deep reinforcement learning mnih 2015 10.1038/nature14236 518 529

상세보기
Nature Mastering the game of go with deep neural networks and tree search silver 2016 10.1038/nature16961 529 484

상세보기
Proc ICLR Continuous control with deep reinforcement learning lillicrap 2016 1
Liu, Chi Harold, Chen, Zheyu, Tang, Jian, Xu, Jie, Piao, Chengzhe. Energy-Efficient UAV Control for Effective and Fair Communication Coverage: A Deep Reinforcement Learning Approach. IEEE journal on selected areas in communications : a publication of the IEEE Communications Society, vol.36, no.9, 2059-2070.

상세보기
Savaglio, Claudio, Pace, Pasquale, Aloi, Gianluca, Liotta, Antonio, Fortino, Giancarlo. Lightweight Reinforcement Learning for Energy Efficient Communications in Wireless Sensor Networks. IEEE access : practical research, open solutions, vol.7, 29355-29364.

상세보기
Chen, Xianfu, Zhang, Honggang, Wu, Celimuge, Mao, Shiwen, Ji, Yusheng, Bennis, Medhi. Optimized Computation Offloading Performance in Virtual Edge Computing Systems Via Deep Reinforcement Learning. IEEE Internet of things journal : a joint publication of the IEEE Sensors Council, the IEEE Communications Society, the IEEE Computer Society, the IEEE Signal Processing Society, vol.6, no.3, 4005-4018.

상세보기
Min, Minghui, Xiao, Liang, Chen, Ye, Cheng, Peng, Wu, Di, Zhuang, Weihua. Learning-Based Computation Offloading for IoT Devices With Energy Harvesting. IEEE transactions on vehicular technology, vol.68, no.2, 1930-1941.

상세보기
Chen, Xianfu, Zhao, Zhifeng, Wu, Celimuge, Bennis, Mehdi, Liu, Hang, Ji, Yusheng, Zhang, Honggang. Multi-Tenant Cross-Slice Resource Orchestration: A Deep Reinforcement Learning Approach. IEEE journal on selected areas in communications : a publication of the IEEE Communications Society, vol.37, no.10, 2377-2392.

상세보기
10.1109/ICCV.2017.541
IEEE Trans Ind Informat Internet of Things for enterprise systems of modern manufacturing bi 2014 10.1109/TII.2014.2300338 10 1537

상세보기
Proc NIPS Dynamic network surgery for efficient dnns guo 2016 1379
Ma, Huadong, Liu, Wu. A Progressive Search Paradigm for the Internet of Things. IEEE multimedia, vol.25, no.1, 76-86.

상세보기
Shi, Weisong, Cao, Jie, Zhang, Quan, Li, Youhuizi, Xu, Lanyu. Edge Computing: Vision and Challenges. IEEE Internet of things journal : a joint publication of the IEEE Sensors Council, the IEEE Communications Society, the IEEE Computer Society, the IEEE Signal Processing Society, vol.3, no.5, 637-646.

상세보기
10.1145/1150402.1150464
Pan, Jianli, McElhannon, James. Future Edge Cloud and Edge Computing for Internet of Things Applications. IEEE Internet of things journal : a joint publication of the IEEE Sensors Council, the IEEE Communications Society, the IEEE Computer Society, the IEEE Signal Processing Society, vol.5, no.1, 439-449.

상세보기
10.1109/IROS.2017.8202244
10.1109/ICDCS.2017.123
Guerrero-ibanez, Juan Antonio, Zeadally, Sherali, Contreras-Castillo, Juan. Integration challenges of intelligent transportation systems with connected vehicle, cloud computing, and internet of things technologies. IEEE wireless communications, vol.22, no.6, 122-128.

상세보기
10.1109/ICRA.2018.8461233
Zanella, Andrea, Bui, Nicola, Castellani, Angelo, Vangelista, Lorenzo, Zorzi, Michele. Internet of Things for Smart Cities. IEEE Internet of things journal : a joint publication of the IEEE Sensors Council, the IEEE Communications Society, the IEEE Computer Society, the IEEE Signal Processing Society, vol.1, no.1, 22-32.

상세보기
Zhu, Hao, Cao, Yang, Wang, Wei, Jiang, Tao, Jin, Shi. Deep Reinforcement Learning for Mobile Edge Caching: Review, New Features, and Open Issues. IEEE network, vol.32, no.6, 50-57.

상세보기
Xu, Mengwei, Qian, Feng, Mei, Qiaozhu, Huang, Kang, Liu, Xuanzhe. DeepType : On-Device Deep Learning for Input Personalization Service with Minimal Privacy Concern. Proceedings of acm on interactive, mobile, wearable and ubiquitous technologies, vol.2, no.4, 1-26.

상세보기
10.1145/3241539.3241559
Li, He, Ota, Kaoru, Dong, Mianxiong. Learning IoT in Edge: Deep Learning for the Internet of Things with Edge Computing. IEEE network, vol.32, no.1, 96-101.

상세보기
Proc IEEE Int Conf Edge Comput (EDGE) Are existing knowledge transfer techniques effective for deep learning with edge devices? sharma 2018 29
Proc ICLR Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding han 2016 1
Proc NIPS Learning both weights and connections for efficient neural network han 2015 1135

LOADING...

활용도 분석정보

상세보기

다운로드

내보내기

활용도 Top5 논문

해당 논문의 주제분야에서 활용도가 높은 상위 5개 콘텐츠를 보여줍니다.
더보기 버튼을 클릭하시면 더 많은 관련자료를 살펴볼 수 있습니다.

원문 보기

AccessON 원문보기

원문 URL 링크

DOI : 10.1109/ACCESS.2020.3014922 [무료]
IEEE : 저널 > 논문 [무료]
AccessON : 저널

*원문 PDF 파일 및 링크정보가 존재하지 않을 경우 KISTI DDS 시스템에서 제공하는 원문복사서비스를 사용할 수 있습니다.

오픈액세스(OA) 유형

GOLD

오픈액세스 학술지에 출판된 논문

저작권 관리 안내

내보내기 메뉴

내보내기 구분

파일저장
인쇄
메일전송

구성항목

기본정보
상세정보

관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관

저장형식

Text(ASCII format)
Excel format
RefWorks Direct Export
RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley

메일정보

받는사람 (필수): @
보내는사람 (선택): @
제목
내용: KISTI 검색결과 이메일 서비스

안내

총 건의 자료가 검색되었습니다.

다운받으실 자료의 인덱스를 입력하세요. (1-10,000)

검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다.

데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요)

다운로드 파일은 UTF-8 형태로 저장됩니다.
파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오.

Text(ASCII format)
Excel format

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

AI-Helper ※ AI-Helper는 을 사용합니다.

AI-Helper

안녕하세요, AI-Helper입니다. 좌측 "선택된 텍스트"에서 텍스트를 선택하여 요약, 번역, 용어설명을 실행하세요.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.

연합인증