[논문]Differentiable Architecture Search Based on Coordinate Descent

Ahn, Pyunghwan; Hong, Hyeong Gwon; Kim, Junmo

doi:10.1109/access.2021.3068766

[해외논문] Differentiable Architecture Search Based on Coordinate Descent 원문보기

IEEE access : practical research, open solutions, v.9, 2021년, pp.48544 - 48554

Ahn, Pyunghwan (Korea Advanced Institute of Science and Technology, School of Electrical Engineering, Daejeon, South Korea) , Hong, Hyeong Gwon (Korea Advanced Institute of Science and Technology, Graduate School of AI, Daejeon, South Korea) , Kim, Junmo (Korea Advanced Institute of Science and Technology, School of Electrical Engineering, Daejeon, South Korea)

Abstract ▼ AI-Helper

Neural architecture search (NAS) is an automated method searching for the optimal network architecture by optimizing the combinations of edges and operations. For efficiency, recent differentiable architecture search methods adopt a one-shot network, containing all the candidate operations in each edge, instead of sampling and training individual architectures. However, a recent study doubts the effectiveness of differentiable methods by showing that random search can achieve comparable performance with differentiable methods using the same search cost. Therefore, there is a need to reduce the search cost even for previous differentiable methods. For more efficient differentiable architecture search, we propose a differentiable architecture search based on coordinate descent (DARTS-CD) that searches for optimal operation over only one sampled edge per training step. DARTS-CD is proposed based on the coordinate descent algorithm, which is an efficient learning method for resolving large-scale problems by updating only a subset of parameters. In DARTS-CD, one edge is randomly sampled, in which all the operations are performed, whereas only one operation is applied to the other edges. Weight update is also performed only at the sampled edge. By optimizing each edge separately, as in the coordinate descent that optimizes each coordinate individually, DARTS-CD converges much faster than DARTS while using the network architecture similar to that used for evaluation. We experimentally show that DARTS-CD performs comparably to the state-of-the-art efficient architecture search algorithms, with an extremely low search cost of 0.125 GPU days (1/12 of the search cost of DARTS) on CIFAR-10 and CIFAR-100. Furthermore, a warm-up regularization method is introduced to improve the exploration capability, which further enhances the performance.

참고문헌 (32)

arXiv 1412 6980 Adam: A method for stochastic optimization kingma 2015
J Mach Learn Res Adaptive subgradient methods for online learning and stochastic optimization duchi 2011 12 2121

상세보기
arXiv 2004 08996 Local search is a remarkably strong baseline for neural architecture search den ottelander 2020
10.1109/ICCV.2015.178
Chen, Liang-Chieh, Papandreou, George, Kokkinos, Iasonas, Murphy, Kevin, Yuille, Alan L.. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE transactions on pattern analysis and machine intelligence, vol.40, no.4, 834-848.

상세보기
10.1109/CVPR.2018.00907
Real, Esteban, Aggarwal, Alok, Huang, Yanping, Le, Quoc V.. Regularized Evolution for Image Classifier Architecture Search. Proceedings of the ... aaai conference on artificial intelligence, vol.33, 4780-4789.

상세보기
Proc Int Conf Learn Represent Darts: Differentiable architecture search liu 2019 1
Proc Int Conf Mach Learn Understanding and simplifying one-shot architecture search bender 2018 549
Proc Uncertainty Artif Intell Random search and reproducibility for neural architecture search li 2020 367
Nesterov, Y.. Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems. SIAM journal on optimization : a publication of the Society for Industrial and Applied Mathematics, vol.22, no.2, 341-362.

상세보기
arXiv 1909 00918 Efficiency of coordinate descent methods for structured nonconvex optimization deng 2019
Patrascu, Andrei, Necoara, Ion. Efficient random coordinate descent algorithms for large-scale structured nonconvex optimization. Journal of global optimization : an international journal dealing with theoretical and computational aspects of seeking global optima and their applications in science, management and engineering, vol.61, no.1, 19-46.

상세보기
10.1007/978-3-030-01246-5_2
10.1109/CVPR.2016.90
Proc Int Conf Learn Represent Hierarchical representations for efficient architecture search liu 2018 1
10.1109/CVPR.2015.7298594
10.1109/CVPR.2014.81
arXiv 2005 02960 Local search is state of the art for neural architecture search benchmarks white 2020
10.1109/CVPR.2017.243
10.1109/CVPR.2016.91
Proc Adv Neural Inf Process Syst Faster R-CNN: Towards real-time object detection with region proposal networks ren 2015 91
Proc Int Conf Learn Represent Very deep convolutional networks for large-scale image recognition simonyan 2015 1
10.1109/CVPR.2015.7298965
Proc Adv Neural Inf Process Syst Imagenet classification with deep convolutional neural networks krizhevsky 2012 1097
Proc 13th AAAI Conf Artif Intell Fast lasso algorithm via selective coordinate descent fujiwara 2016 1561
arXiv 1605 07146 Wide residual networks zagoruyko 2016
Proc Int Conf Learn Represent SNAS: Stochastic neural architecture search xie 2019 1
10.1109/ICCV.2019.00138
Proc Int Conf Mach Learn Efficient neural architecture search via parameters sharing pham 2018 4095
arXiv 1708 04552 Improved regularization of convolutional neural networks with cutout devries 2017
Proc Int Conf Learn Represent Proxylessnas: Direct neural architecture search on target task and hardware cai 2018 1

LOADING...

활용도 분석정보

상세보기

다운로드

내보내기

활용도 Top5 논문

해당 논문의 주제분야에서 활용도가 높은 상위 5개 콘텐츠를 보여줍니다.
더보기 버튼을 클릭하시면 더 많은 관련자료를 살펴볼 수 있습니다.

원문 보기

AccessON 원문보기

원문 URL 링크

DOI : 10.1109/ACCESS.2021.3068766 [무료]
IEEE : 저널 > 논문 [무료]
AccessON : 저널

*원문 PDF 파일 및 링크정보가 존재하지 않을 경우 KISTI DDS 시스템에서 제공하는 원문복사서비스를 사용할 수 있습니다.

오픈액세스(OA) 유형

GOLD

오픈액세스 학술지에 출판된 논문

저작권 관리 안내

내보내기 메뉴

내보내기 구분

파일저장
인쇄
메일전송

구성항목

기본정보
상세정보

관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관

저장형식

Text(ASCII format)
Excel format
RefWorks Direct Export
RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley

메일정보

받는사람 (필수): @
보내는사람 (선택): @
제목
내용: KISTI 검색결과 이메일 서비스

안내

총 건의 자료가 검색되었습니다.

다운받으실 자료의 인덱스를 입력하세요. (1-10,000)

검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다.

데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요)

다운로드 파일은 UTF-8 형태로 저장됩니다.
파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오.

Text(ASCII format)
Excel format

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

AI-Helper ※ AI-Helper는 을 사용합니다.

AI-Helper

안녕하세요, AI-Helper입니다. 좌측 "선택된 텍스트"에서 텍스트를 선택하여 요약, 번역, 용어설명을 실행하세요.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.

연합인증