[논문]Spatial Shift Point-Wise Quantization

Kim, Eunhui; Lee, Kyong-Ha

doi:10.1109/access.2020.3038164

[해외논문] Spatial Shift Point-Wise Quantization 원문보기

IEEE access : practical research, open solutions, v.8, 2020년, pp.207683 - 207690

Kim, Eunhui (Korea Institute of Science and Technology Information, Daejeon, South Korea) , Lee, Kyong-Ha (Korea Institute of Science and Technology Information, Daejeon, South Korea)

Abstract ▼ AI-Helper

Deep neural networks (DNN) have been applied to numerous artificial-intelligence applications because of their remarkable accuracy. However, computational requirements for deep neural networks are recently skyrocketing far beyond the Moore’s Law. In addition to the importance of accuracy, the industry’s demand for efficiency in model learning process is increasing. This has led to various attempts to make DNNs more lightweight. Hence, we propose a modeling technique that applies lightweight convolutional neural networks (CNN) to handle the model-learning processes for DNNs. The proposed spatial-shift pointwise quantization (SSPQ) model elegantly combines compact network-design techniques to revitalize DNN quantization efficiency with little accuracy loss. We set the depths of our SSPQ model to 20, 34, and 50 to test against CIFAR10, CIFAR100, and ImageNet datasets, respectively. By applying SSPQ20 to the CIFAR10 dataset, we reduced accuracy degradation by 2.95%, while reducing the number of parameters $8.7\times $. For the same dataset, our “wide” SSPQ20 variant reduced training parameters $1.96\times $, compared with the ResNet20 model architecture, which provided a 0.2% improvement. By applying SSPQ34 to the CIFAR100 dataset, we successfully reduced the number of learning parameters $8\times $, compared with the ResNet34 model with an accuracy degradation of 4.57%. By applying SSPQ50 to the ImageNet dataset, we successfully reduced the number of parameters $10.2\times $ over ResNet50 with an accuracy degradation of only 2.68%. Therefore, using only 9.78MB of learning parameter, the SSPQ50 model guarantees 73.6% accuracy. This is an improved performance by 1.46% in terms of accuracy and 3.62MB in terms of model size compared to the MobileNetV2 lightweight model. The source code is available at https://github.com/Eunhui-Kim/SSPQ.

참고문헌 (34)

Custom-Op feng 2019
Tensorpack wu 2016
Proc Adv Neural Inf Process Syst Normalization helps training of quantized LSTM hou 2019 7346
10.1007/978-3-030-58539-6_26
10.5244/C.30.87
10.1109/CVPR.2018.00951
Proc Adv Neural Inf Process Syst Constructing fast network through deconstruction of convolution jeon 2018 5951
10.1109/CVPR.2019.00741
10.1109/CVPR.2017.243
arXiv 1905 11946 EfficientNet: Rethinking model scaling for convolutional neural networks tan 2019
arXiv 1510 00149 [cs] Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding han 2015
Proc Adv Neural Inf Process Syst Binaryconnect: Training deep neural networks with binary weights during propagations courbariaux 2015 3123
Proc Adv Neural Inf Process Syst Binarized neural networks hubara 2016 4107
ArXiv 1503 02531 Distilling the knowledge in a neural network hinton 2015
Proc Int Conf Learn Represent (ICLR) And the bit goes down: Revisiting the quantization of neural networks stock 2020
Proc Eur Conf Comput Vis (ECCV) Xnor-Net: ImageNet classification using binary convolutional neural networks rastegariy 2016 525
10.1109/CVPR.2016.90
arXiv 1606 06160 [cs] DoReFa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients zhou 2016
KIISE Communication Recent trends in lightweight technology for deep neural networks kim 2020 38 18
Proc Eur Conf Comput Vis (ECCV) Encoder-decoder with atrous separable convolution for semantic image segmentation chen 2018 801
10.1007/978-3-030-58610-2_34
10.1007/978-3-319-46493-0_38
Proc Adv Neural Inf Process Syst ImageNet classification with deep convolutional neural networks krizhevsky 2012 1097
arXiv 1704 04861 MobileNets: Efficient convolutional neural networks for mobile vision applications howard 2017
10 Technological Trends in 2020 research 2020
10.1109/CVPR.2018.00716
Bianco, Simone, Cadene, Remi, Celona, Luigi, Napoletano, Paolo. Benchmark Analysis of Representative Deep Neural Network Architectures. IEEE access : practical research, open solutions, vol.6, 64270-64277.

상세보기
10.1109/CVPR.2018.00474
arXiv 1611 01600 Loss-aware binarization of deep networks hou 2016
Coursera video lectures Neural networks for machine learning hinton 2012 264
Proc Adv Neural Inf Process Syst Metaquant: Learning to quantize by learning to penetrate non-differentiable quantization chen 2019 3916
arXiv 1802 08635 Loss-aware weight quantization of deep networks hou 2018
10.1109/CVPR.2019.00748
10.1007/978-3-030-01267-0_44

LOADING...

활용도 분석정보

상세보기

다운로드

내보내기

활용도 Top5 논문

해당 논문의 주제분야에서 활용도가 높은 상위 5개 콘텐츠를 보여줍니다.
더보기 버튼을 클릭하시면 더 많은 관련자료를 살펴볼 수 있습니다.

원문 보기

AccessON 원문보기

원문 URL 링크

DOI : 10.1109/ACCESS.2020.3038164 [무료]
IEEE : 저널 > 논문 [무료]
AccessON : 저널

*원문 PDF 파일 및 링크정보가 존재하지 않을 경우 KISTI DDS 시스템에서 제공하는 원문복사서비스를 사용할 수 있습니다.

오픈액세스(OA) 유형

GOLD

오픈액세스 학술지에 출판된 논문

저작권 관리 안내

내보내기 메뉴

내보내기 구분

파일저장
인쇄
메일전송

구성항목

기본정보
상세정보

관리번호, 논문명, 저널/프로시딩명, 저자 , 발행년, 권, 호, 시작페이지, 끝페이지, 발행기관

저장형식

Text(ASCII format)
Excel format
RefWorks Direct Export
RIS format (for Reference Manager, ProCite, EndNote), Scholar's Aids, Mendeley

메일정보

받는사람 (필수): @
보내는사람 (선택): @
제목
내용: KISTI 검색결과 이메일 서비스

안내

총 건의 자료가 검색되었습니다.

다운받으실 자료의 인덱스를 입력하세요. (1-10,000)

검색결과의 순서대로 최대 10,000건 까지 다운로드가 가능합니다.

데이타가 많을 경우 속도가 느려질 수 있습니다.(최대 2~3분 소요)

다운로드 파일은 UTF-8 형태로 저장됩니다.
파일의 내용이 제대로 보이지 않을실 때는 웹브라우저 상단의 보기 -> 인코딩 -> 자동선택 여부를 확인하십시오.

Text(ASCII format)
Excel format

표제어: PCR

동의어: Packet Collision Rate

용어 설명 출처 목록 (6)

용어 설명: PCR은 세균 특이성이 있는 primer를 이용하여 적은 수의 세균이 있을지라도 쉽게 검출할 수 있는 유용한 방법이며, 이를 이용하여 구강 내 치면세균막이나 타액에서 직접 세균을 검출할 수 있게 되었다[8].

AI-Helper ※ AI-Helper는 을 사용합니다.

AI-Helper

안녕하세요, AI-Helper입니다. 좌측 "선택된 텍스트"에서 텍스트를 선택하여 요약, 번역, 용어설명을 실행하세요.
※ AI-Helper는 부적절한 답변을 할 수 있습니다.

연합인증